Giter Club home page Giter Club logo

microcks-backstage-provider's Introduction

Microcks Backstage provider

GitHub Workflow Status Version License Project Chat

This is a plugin for synchronizing Microcks content into Backstage.io catalog.

The microcks-backstage-provider has a special entity provider for discovering catalog API entities defined in one or many Microcks instances. If you're using Microcks to discover APIs from your Git repositories, auto-publish smart mock endpoints and realize contract testing of your components, this provider can synchronize the API definitions from Microcks into your Backstage intance.

Screenshots

Your Services and API from different Microcks instances can be discovered and imported into your Backstage catalog.

Labels on your API in Microcks are be translated onto Backstage systems and owners. The plugin add links to access the API mock endpoints and conformance tests results.

Install

After having created a new backstage app like described in Backstage.io' Getting Started, enter the app directory and run this command to add the Microcks Entity provider as a new backend plugin:

yarn add --cwd packages/backend @microcks/microcks-backstage-provider@^0.0.5

Configure

Microcks Backstage provider allows configuration of one or many providers using the app-config.yaml configuration file of Backstage. Use a microcksApiEntity marker to start configuring them.

Each children key (here dev in the following sample) configures a provider for a specific lifecyle phase of your Entities so you may discover API entities from different Microcks instances.

At your configuration, you add a provider config per liefcycle:

# app-config.yaml

catalog:
  providers:
    microcksApiEntity:
      dev:
        baseUrl: https://microcks.acme.com
        serviceAccount: microcks-serviceaccount
        serviceAccountCredentials: ab54d329-e435-41ae-a900-ec6b3fe15c54
        systemLabel: domain
        ownerLabel: team 
        addLabels: false # optional; copy any other labels from Microcks in Backstage catalog - default is true
        addOpenAPIServerUrl: true # optional; add a server in OpenAPI spec to point to Microcks sandbox URL - default is false
        schedule: # optional; same options as in TaskScheduleDefinition
          # supports cron, ISO duration, "human duration" as used in code
          frequency: { minutes: 2 }
          # supports ISO duration, "human duration" as used in code
          timeout: { minutes: 1 }  

Configuration is pretty straightforward but your may check Microcks documentation on Service Accounts for details.

Once you've done that, you'll also need to declare the Microcks as a backend module or an available entity provider. This operation differs depending on the version of Backstage you're using.

For Backstage versions with the new Backend system

Typically for application created via the CLI starting with Backstage 1.24

You'll need to add the Microcks backend module in the packages/backend/src/index.ts file, before the backend.start() directive. Use this snippet below:

// microcks catalog provider
backend.add(import('@microcks/microcks-backstage-provider'));

For Backstage versions with the old Backend plugin system

You need add the segment below to packages/backend/src/plugins/catalog.ts:

/* packages/backend/src/plugins/catalog.ts */

import { MicrocksApiEntityProvider } from '@microcks/microcks-backstage-provider';

[...]
  const builder = await CatalogBuilder.create(env);

  /** ... other processors and/or providers ... */
  builder.addEntityProvider(
    MicrocksApiEntityProvider.fromConfig(env.config, {
      logger: env.logger,
      scheduler: env.scheduler
    }),
  );

  const { processingEngine, router } = await builder.build();
[...]

Troubleshoot

After having started your Backstage app, you should see some lines like those below in logs:

[1] 2023-01-04T16:07:47.630Z catalog info Keycloak authentication is not enabled, using a fake token. type=plugin target=MicrocksApiEntityProvider:dev
[1] 2023-01-04T16:07:47.699Z catalog info Discovered ApiEntity API Pastry - 2.0 - 2.0.0 type=plugin target=MicrocksApiEntityProvider:dev
[1] 2023-01-04T16:07:47.708Z catalog info Discovered ApiEntity Account Service - 1.1.0 type=plugin target=MicrocksApiEntityProvider:dev
[1] 2023-01-04T16:07:47.712Z catalog info Discovered ApiEntity HelloService - v1 type=plugin target=MicrocksApiEntityProvider:dev
[1] 2023-01-04T16:07:47.728Z catalog info Discovered ApiEntity User signed-up API - 0.1.1 type=plugin target=MicrocksApiEntityProvider:dev
[1] 2023-01-04T16:07:47.734Z catalog info Discovered ApiEntity User signed-up API - 0.1.20 type=plugin target=MicrocksApiEntityProvider:dev
[1] 2023-01-04T16:07:47.740Z catalog info Discovered ApiEntity User signed-up API - 0.1.30 type=plugin target=MicrocksApiEntityProvider:dev
[1] 2023-01-04T16:07:47.745Z catalog info Discovered ApiEntity User signed-up Avro API - 0.1.2 type=plugin target=MicrocksApiEntityProvider:dev
[1] 2023-01-04T16:07:47.749Z catalog info Applying the mutation with 7 entities type=plugin target=MicrocksApiEntityProvider:dev

Build and release

After a fresh git clone, you can install dependencies, compile typescript and build a new distribution with those commands:

$ yarn install && yarn export-dynamic && yarn tsc && yarn run build

Once happy with result, just push versioned package on npmjs.com with:

$ npm publish

Develop locally

After having created a new backstage app like described in Backstage.io' Getting Started, just edit the packages/backend/src/plugins/catalog.ts file and apply same configuration as above insdtallation directive but with relative path to your local clone of this repository:

/* packages/backend/src/plugins/catalog.ts */

import { MicrocksApiEntityProvider } from '../../../../../github/microcks-backstage-provider';

Just launch your local backstage app using yarn dev at the root folder level.

microcks-backstage-provider's People

Contributors

fmenesesg avatar hguerrero avatar lbroudoux avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

microcks-backstage-provider's Issues

Add proper exception handling and avoid to apply mutation on error

Reason/Context

As of today, the plugin only manages the "happy path". Connection exceptions are not correctly handled and mutation of entities can be applied when no entity is fetched (that lead to improper removal of still-existing entities). We had to fix this for better troubleshooting and to avoid phantom entities' symptom.

Description

The connectors (to Microcks and Keycloak) should probably handle connection failures, throwing an exception with clear message.

The reconciliation loop must also track/catch such exceptions, inform the administrator though a log message and avoid applying mutations in that situation.

Implementation ideas

No response

In the 'Try It Out' feature of the 'Definition' Tab the requestUrl is wrong

Describe the bug

I added a open-api spec to my microcks instance that I am running locally. On the microcks side the url for one of the mocked routes is http://localhost:8080/rest/Petstore+API/1.0.0/my/pets
image

Using the microcks-backstage-provider I am able to register this API, but when I want to use the 'Try It Out' feature for the mocked route the requestUrl populated is wrong http://localhost:3000/my/pets
image

Expected behavior

The 'Try It Out' url should match the mock URL (not sure what to do in case of multiple mocks?)

Actual behavior

The 'Try It Out' url does not match the mock URL

How to Reproduce?

No response

Microcks version or git rev

0.0.2

Install method (docker-compose, helm chart, operator, docker-desktop extension,...)

yarn

Additional information

No response

Release 0.0.2 version of microcks-backstage-provider

Embeds following features:

  • Support of Keycloak authentication for Microcks instance where authentication is turned on,
  • Proper error handling and management of reconciliation when error happens,
  • Community and project health files updated.

Release package on npmjs.com and ensure to put a proper tag in Git.

Add support for Microcks with authentication

As of today, the plugin only supports connecting to a Microcks instance where authentication is disabled. Everything has been prepared within the plugin but it needs tests and adjustments.

Microcks plugin is using a non standard library for Base64

Describe the bug

When trying to deploy the plugin using dynamic loading with Red Hat Developer Hub, the Base64 library is not found, breaking the plugin.

Expected behavior

Use either the platform-supported handling of Base64 or use the dependency from backstage.

Actual behavior

backstage error an error occured while loading dynamic backend plugin '@microcks/microcks-backstage-provider' from 'file:///opt/app-root/src/dynamic-plugins-root/microcks-microcks-backstage-provider-0.0.3' Cannot find module 'js-base64'

How to Reproduce?

Try to use the dynamic plugin with the microcks package 0.0.3 in Red Hat Developer Sandbox

Microcks version or git rev

0.0.3

Install method (docker-compose, helm chart, operator, docker-desktop extension,...)

Backstage

Additional information

No response

Config Loader issue on Backstage v1.17

Describe the bug

Microcks config schema is considered invalid on Backstage version 1.17

Expected behavior

Be able to use microcks plugin with backstage v1.17 version

Actual behavior

Microck plugin start with error, because of Config Loader CHANGELOG

How to Reproduce?

Create and backstage 1.17 instance or upgrade an existing one
Configure the Microcks plugin as the Readme file says
Start the backstage instance
See error

Microcks version or git rev

1.7.1

Install method (docker-compose, helm chart, operator, docker-desktop extension,...)

podman-compose

Additional information

No response

Support GraphQL API discovery and synchronization to Backstage

Reason/Context

As Backstage has now an API type for GraphQL APIs, we need to add their synchronization too.

Description

Allow listing of GraphQL APIs and retrieval of GraphQL Schemas from Microcks for creating the corresponding entities in Backstage.

Implementation ideas

No response

Make the plugin dynamic

Reason/Context

There is a proposal to add dynamic plugin loading. It requires minor fixes to be able to use this feature, like in RH Developer Hub

Description

Following the RFC for dynamic loading, there is now an implementation in the RH Developer Hub to dynamically load these kinds of plugins. Adding it to the Microcks plugin could make it easier to adopt.

Implementation ideas

Follow up the example from the 3scale plugin

  1. Add dynamic folder
  2. Add index.ts and alpha.ts
  3. Import those new files and export the module

Add community and update project health file

Describe the bug

Replicate code of conduct, contribution guide, and so on from the organization repository. Update README with latest information and commit package-lock.json file.

Attach any resources that can help us understand the issue.

See organization repository files.

Red Hat Developer Hub does not support empty owner and system labels

Describe the bug

When trying to deploy the backstage plugin, if there are no labels in the service, the plugin tries to put an empty label. This is not supported, hence the service is not added to the catalog.

Expected behavior

Microcks adds a default owner and system labels.

Actual behavior

No service is imported

How to Reproduce?

. Install plugin
. Load service
. Wait for plugin task

Microcks version or git rev

1.8,1

Install method (docker-compose, helm chart, operator, docker-desktop extension,...)

backstage plugin + uber jar

Additional information

No response

Support for new BackEnd system

Reason/Context

As of 1.24, backstage promotes the new back end system as the default backend in the backstage create app cli command.

The Microcks plugin does not support it yet natively.

Description

Provide a BackendModule that enables the activation of the Microcks plugin in the form of :

backend.add(import('@microcks/microcks-backstage-backend'));

in packages/backend/src/index.ts

Implementation ideas

I have started to work on this plugin, to use as an internal plugin but it does not work yet.

I have created the microcksProvider.ts file with

import {coreServices, createBackendModule} from "@backstage/backend-plugin-api";
import {MicrocksApiEntityProvider} from "@microcks/microcks-backstage-provider";
import { catalogProcessingExtensionPoint } from '@backstage/plugin-catalog-node/alpha';
import {loggerToWinstonLogger} from "@backstage/backend-common";

export default createBackendModule({
  pluginId: 'microcks',
  moduleId: 'microcksProvider',
  register(reg) {
    reg.registerInit({
      deps: {
        catalog: catalogProcessingExtensionPoint,
        config: coreServices.rootConfig,
        logger: coreServices.logger,
        scheduler: coreServices.scheduler,
      },
      async init({ catalog, config, logger, scheduler }) {
        catalog.addEntityProvider(
          MicrocksApiEntityProvider.fromConfig(config, {
            logger: loggerToWinstonLogger(logger),
            scheduler: scheduler,
            schedule: scheduler.createScheduledTaskRunner({
              frequency: { minutes: 2 },
              timeout: { minutes: 1 },
            }),
          })
        )
      },
    });
  },
});

I got inspired by the alpha file done for red hat https://github.com/microcks/microcks-backstage-provider/blob/main/src/dynamic/alpha.ts

And included the module in the index.ts like so:

backend.add(import('./plugins/microcksProvider'));

It compiles but fails to start properly...

[1] /mnt/c/work/ASD/backstage/node_modules/@backstage/backend-app-api/src/wiring/BackendInitializer.ts:67
[1]           throw new Error(
[1]                 ^
[1] Error: Illegal dependency: Module 'microcksProvider' for plugin 'microcks' attempted to depend on extension point 'catalog.processing' for plugin 'catalog'. Extension points can only be used within their plugin's scope.     
[1]     at BackendInitializer.getInitDeps_fn (/mnt/c/work/ASD/backstage/node_modules/@backstage/backend-app-api/src/wiring/BackendInitializer.ts:67:17)
[1]     at <anonymous> (/mnt/c/work/ASD/backstage/node_modules/@backstage/backend-app-api/src/wiring/BackendInitializer.ts:261:40)
[1]     at processNode (/mnt/c/work/ASD/backstage/node_modules/@backstage/backend-app-api/src/lib/DependencyGraph.ts:234:28)
[1]     at Array.map (<anonymous>)
[1]     at processMoreNodes (/mnt/c/work/ASD/backstage/node_modules/@backstage/backend-app-api/src/lib/DependencyGraph.ts:226:40)
[1]     at _DependencyGraph.parallelTopologicalTraversal (/mnt/c/work/ASD/backstage/node_modules/@backstage/backend-app-api/src/lib/DependencyGraph.ts:242:11)
[1]     at <anonymous> (/mnt/c/work/ASD/backstage/node_modules/@backstage/backend-app-api/src/wiring/BackendInitializer.ts:259:22)
[1]     at Array.map (<anonymous>)
[1]     at BackendInitializer.doStart_fn (/mnt/c/work/ASD/backstage/node_modules/@backstage/backend-app-api/src/wiring/BackendInitializer.ts:237:20)
[1]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[1]     at BackendInitializer.start (/mnt/c/work/ASD/backstage/node_modules/@backstage/backend-app-api/src/wiring/BackendInitializer.ts:149:5)
[1]     at BackstageBackend.start (/mnt/c/work/ASD/backstage/node_modules/@backstage/backend-app-api/src/wiring/BackstageBackend.ts:42:11)
[1]
[1] Node.js v20.9.0

Validate URL for backslashes at the end

Describe the bug

When setting the URL in the config to something like http://mydomain.com/ there is an error as it can't load the Keycloak configuration correctly.

We need to check if there is already a / and either remove it or not add an additional one.

Expected behavior

There is no difference on the behavior of an URL with or without trailing backlash

Actual behavior

It fails when setting the URL to something with a trailing backlash

How to Reproduce?

  1. Add Microcks provider
  2. Configure with trailing backslash
  3. See the error in the log

Microcks version or git rev

1.9.0

Install method (docker-compose, helm chart, operator, docker-desktop extension,...)

backstage

Additional information

No response

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.