The release.yml that we add to a scaffolded plugin is a bit outdated.

• Update the instructions in the final step (Publish to Grafana.com) to match the current workflow
• Caching strategy of yarn and node modules looks outdated. (https://github.com/actions/setup-node/blob/main/docs/advanced-usage.md#caching-packages-data)
• Make sure action versions are up to date

Currently we don't test that the generated apps can be built after they are generated.

We should test types and lint

From README.md:

yarn create @grafana/plugin
yarn install
yarn dev

after running yarn create a new directory is created (seems to be named after org - plugin - plugin type). The next two instructions, yarn install and yarn dev, don't do anything if you don't change into the newly created directory first.

If you choose to not generate a backend for app and datasource plugins, create-plugin stills generates a plugin.json with the required fields and this will cause the plugin to fail loading in grafana.

Modify package.json to not use docker to run cypress.

When generating an app plugin with a backend it gets scaffolded with a datasource plugin backend instead of an app plugin backend. We should probably change this to be scaffolded with something like:

https://github.com/grafana/incident/tree/main/plugin/pkg

Package Name

create-plugin

What happened?

originally reported here grafana/azure-data-explorer-datasource#499

When importing modules relative to typescript baseUrl webpack fails to compile.

This works correctly when running tsc standalone but not when running webpack build.

What you expected to happen

import { analyzeQueries, trackADXMonitorDashboardLoaded } from 'tracking';

should work the same as

import { analyzeQueries, trackADXMonitorDashboardLoaded } from './tracking';

when tracking.ts is a file located in ${baseUrl}/tracking.ts

How to reproduce it (as minimally and precisely as possible)

Do a relative import from any ts file.
Run webpack build
See it fail

Environment

All platforms

Additional context

No response

test migrating some cloud grafana plugins (without committing the result upstream) to validate migrate is doing the correct work.

Such as:

• https://github.com/grafana/x-ray-datasource
• https://github.com/grafana/azure-data-explorer-datasource
• https://github.com/grafana/athena-datasource
• https://github.com/grafana/redshift-datasource
• https://github.com/grafana/timestream-datasource

We need to bundle and publish this package to NPM to make it easy for the community to use this. 🚀

• Define minimum nodejs version
• Bundle this package using whichever tools do the job (tsc, babel, esbuild, swc, rollup, parcel etc)
• Add a github workflow / use drone to run steps (e.g. test, build, etc) and publish this package to NPM.

Package Name

create-plugin

What happened?

When running yarn create @grafana/plugin, the generated folder does not have a .gitignore file, even if it should as it's present in templates/common.

This happens with every template (app, datasource and panel).

What you expected to happen

.gitignore should be present in the generated folder.

How to reproduce it (as minimally and precisely as possible)

╰─❯ yarn create @grafana/plugin
yarn create v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Installed "@grafana/[email protected]" with binaries:
      - create-plugin
[#######################################################################################################################] 308/308?
? What is going to be the name of your plugin? test
? What is the organization name of your plugin? test
? How would you describe your plugin? 
? What kind of plugin would you like?  app
? Do you want a backend part of your plugin? No
? Do you want to add Github CI and Release workflows? No
? Do you want to add a Github workflow for automatically checking "Grafana API compatibility" on PRs? No
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/.eslintrc
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/.prettierrc.js
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/Dockerfile
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/jest-setup.js
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/jest.config.js
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/README.md
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/tsconfig.json
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/types/custom.d.ts
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/webpack/constants.ts
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/webpack/utils.ts
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.config/webpack/webpack.config.ts
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.eslintrc
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.nvmrc
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/.prettierrc.js
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/CHANGELOG.md
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/cypress/integration/01-smoke.spec.ts
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/docker-compose.yaml
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/jest-setup.js
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/jest.config.js
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/LICENSE
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/package.json
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/src/img/logo.svg
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/src/README.md
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/tsconfig.json
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/README.md
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/src/components/App/App.test.tsx
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/src/components/App/App.tsx
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/src/components/App/index.tsx
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/src/components/AppConfig/AppConfig.test.tsx
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/src/components/AppConfig/AppConfig.tsx
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/src/components/AppConfig/index.tsx
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/src/module.ts
✔  ++ /home/giuseppe/grafana/plugins/test-test-app/src/plugin.json
✔  +- /home/giuseppe/grafana/plugins/test-test-app/README.md
✔  +- /home/giuseppe/grafana/plugins/test-test-app/README.md

╰─❯ ls -d test-test-app/.*
test-test-app/.config  test-test-app/.eslintrc  test-test-app/.nvmrc  test-test-app/.prettierrc.js

Environment

System:
    OS: Linux 6.0 Manjaro Linux
    CPU: (24) x64 AMD Ryzen 9 5900X 12-Core Processor
    Memory: 26.46 GB / 31.29 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 16.18.0 - ~/.nvm/versions/node/v16.18.0/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v16.18.0/bin/yarn
    npm: 8.19.2 - ~/.nvm/versions/node/v16.18.0/bin/npm
  Browsers:
    Chromium: 106.0.5249.119
    Firefox: 105.0.3
  npmPackages:
    @grafana/data: 9.1.2 => 9.1.2 
    @grafana/e2e: 9.1.2 => 9.1.2 
    @grafana/e2e-selectors: 9.1.2 => 9.1.2 
    @grafana/eslint-config: ^2.5.0 => 2.5.2 
    @grafana/runtime: 9.1.2 => 9.1.2 
    @grafana/tsconfig: ^1.2.0-rc1 => 1.2.0-rc1 
    @grafana/ui: 9.1.2 => 9.1.2

Additional context

Additional information from @tolzhabayev as discussed on Slack:

Can confirm that it’s not part of the published templates in the package https://registry.npmjs.org/@grafana/create-plugin/-/create-plugin-0.3.2.tgz
Reason is npm being too smart about it npm/npm#3763
I guess we should do something like name it gitignore and rename to .gitignore when generating. Like https://github.com/facebook/create-react-app/blob/c20ccecfa1cf130a47d37908dc54959c618ce8ea/packages/react-scripts/scripts/init.js#L257-L271

Package Name

create-plugin

What happened?

When trying to build a plugin using create-plugin that has a react library that expects to be able to load CSS things go 💥.

See error message:

../node_modules/prismjs/themes/prism-tomorrow.min.css 1.28 KiB [built] [1 error]

ERROR in ../node_modules/prismjs/themes/prism-tomorrow.min.css 1:10
Module parse failed: Unexpected token (1:10)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
> code[class*=language-],pre[class*=language-]{color:#ccc;background:0 0;font-family:Consolas,Monaco,'Andale Mono','Ubuntu Mono',monospace;font-size:1em;text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;line-height:1.5;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none}pre[class*=language-]{padding:1em;margin:.5em 0;overflow:auto}:not(pre)>code[class*=language-],pre[class*=language-]{background:#2d2d2d}:not(pre)>code[class*=language-]{padding:.1em;border-radius:.3em;white-space:normal}.token.block-comment,.token.cdata,.token.comment,.token.doctype,.token.prolog{color:#999}.token.punctuation{color:#ccc}.token.attr-name,.token.deleted,.token.namespace,.token.tag{color:#e2777a}.token.function-name{color:#6196cc}.token.boolean,.token.function,.token.number{color:#f08d49}.token.class-name,.token.constant,.token.property,.token.symbol{color:#f8c555}.token.atrule,.token.builtin,.token.important,.token.keyword,.token.selector{color:#cc99cd}.token.attr-value,.token.char,.token.regex,.token.string,.token.variable{color:#7ec699}.token.entity,.token.operator,.token.url{color:#67cdcc}.token.bold,.token.important{font-weight:700}.token.italic{font-style:italic}.token.entity{cursor:help}.token.inserted{color:green}

What you expected to happen

For the build to succeed and the css to be loaded in the browser.

How to reproduce it (as minimally and precisely as possible)

1. Scaffold a plugin
2. Try to import css file (or use a library that expects a css loader).
3. Run 'yarn build'
4. See error pasted above.

Environment

System:
    OS: macOS 12.6
    CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
    Memory: 631.02 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.17.1 - ~/.nvm/versions/node/v16.17.1/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 8.15.0 - ~/.nvm/versions/node/v16.17.1/bin/npm
  Browsers:
    Brave Browser: 106.1.44.112
    Chrome: 106.0.5249.119
    Chrome Canary: 86.0.4239.0
    Firefox: 104.0.2
    Firefox Nightly: 81.0a1
    Safari: 16.0
    Safari Technology Preview: 16.4
  npmPackages:
    @grafana/data: 9.2.1 => 9.2.1
    @grafana/e2e: 9.1.2 => 9.1.2
    @grafana/e2e-selectors: 9.1.2 => 9.1.2
    @grafana/eslint-config: ^2.5.0 => 2.5.2
    @grafana/experimental: 0.0.2-canary.35 => 0.0.2-canary.35
    @grafana/runtime: 9.2.1 => 9.2.1
    @grafana/tsconfig: ^1.2.0-rc1 => 1.2.0-rc1
    @grafana/ui: 9.2.1 => 9.2.1

Additional context

First reported here: grafana/azure-data-explorer-datasource#499

Having merged #119 I think we should refactor how we're introducing the npm dependencies of create-plugin during a plugin migration. Right now we depend on the package.json template for grafana dependencies and then delete certain grafana dependencies that might be in undesired locations (devDeps for example). This made the code required to not change grafana dependencies in the linked PR feel like a band aid.

When using snapshots (or any tests with Date), the runtime TZ of the machine is used by jest.

adding

process.env.TZ = 'UTC';

before the module.export will force the tests to generate UTC timestamps, and will work inside git workflows/drone and locally.

Some plugins are using drone, some have no CI at all, and some may have "outdated" git workflows.

Should we create the CI workflows for migrations that do not have a drone config (or any config at all?)

We could detect:

• drone.yml
• .drone.yml
• .circleci
• others?

Pull Request: #9

The plugins create-plugin currently scaffolds depend on the nested dependencies in @grafana/toolkit for builds resulting in deprecation warnings and because of "latest" dist-tag could cause scripts to break.

yarn build
yarn run v1.22.17
$ node --trace-deprecation ./node_modules/webpack/bin/webpack.js -c ./webpack/webpack.prod.conf.js
Starting type checking service...
(node:35510) [DEP_WEBPACK_MODULE_ERRORS] DeprecationWarning: Module.errors was removed (use getErrors instead)
    at successfulTypeScriptInstance (/Users/jackwestbrook/Projects/grafana-plugin-examples/examples/basic-panel/node_modules/ts-loader/dist/instances.js:119:28)
    at Object.getTypeScriptInstance (/Users/jackwestbrook/Projects/grafana-plugin-examples/examples/basic-panel/node_modules/ts-loader/dist/instances.js:34:12)
    at Object.loader (/Users/jackwestbrook/Projects/grafana-plugin-examples/examples/basic-panel/node_modules/ts-loader/dist/index.js:17:41)
    at LOADER_EXECUTION (/Users/jackwestbrook/Projects/grafana-plugin-examples/examples/basic-panel/node_modules/loader-runner/lib/LoaderRunner.js:132:14)
    at runSyncOrAsync (/Users/jackwestbrook/Projects/grafana-plugin-examples/examples/basic-panel/node_modules/loader-runner/lib/LoaderRunner.js:133:4)
    at iterateNormalLoaders (/Users/jackwestbrook/Projects/grafana-plugin-examples/examples/basic-panel/node_modules/loader-runner/lib/LoaderRunner.js:250:2)
    at Array.<anonymous> (/Users/jackwestbrook/Projects/grafana-plugin-examples/examples/basic-panel/node_modules/loader-runner/lib/LoaderRunner.js:223:4)
    at runCallbacks (/Users/jackwestbrook/Projects/grafana-plugin-examples/examples/basic-panel/node_modules/webpack/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:27:15)
    at /Users/jackwestbrook/Projects/grafana-plugin-examples/examples/basic-panel/node_modules/webpack/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:200:4
    at /Users/jackwestbrook/Projects/grafana-plugin-examples/examples/basic-panel/node_modules/graceful-fs/graceful-fs.js:123:16

The ☝️ above warning is coming from ts-loader which is a dependency of @grafana/toolkit 👇

yarn why ts-loader      
[1/4] 🤔  Why do we have the module "ts-loader"...?
[2/4] 🚚  Initialising dependency graph...
warning Resolution field "[email protected]" is incompatible with requested version "rxjs@^6.6.3"
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "[email protected]"
info Reasons this module exists
   - "@grafana#toolkit" depends on it
   - Hoisted from "@grafana#toolkit#ts-loader"

We should declare the dependencies a plugin requires in the package.json.

Documentation:

• How to migrate from toolkit doc - #190
• Update https://grafana.com/docs/grafana/latest/developers/plugins/
• Update https://grafana.com/tutorials/build-a-panel-plugin/ - https://github.com/grafana/tutorials/pull/195/files
• Update https://grafana.com/tutorials/build-a-data-source-plugin/ - https://github.com/grafana/tutorials/pull/195/files
• Update https://grafana.com/tutorials/build-a-panel-plugin/ - https://github.com/grafana/tutorials/pull/195/files
• Update https://grafana.com/tutorials/build-a-data-source-plugin/ - https://github.com/grafana/tutorials/pull/195/files
• Update https://grafana.com/tutorials/build-a-data-source-backend-plugin/ - https://github.com/grafana/tutorials/pull/195/files
• Update https://grafana.com/tutorials/build-a-panel-plugin-with-d3/ - https://github.com/grafana/tutorials/pull/195/files
• Update https://grafana.com/docs/grafana/latest/developers/plugins/sign-a-plugin/ - grafana/grafana#58593
• Update https://grafana.com/grafana/plugins/ (at the bottom) - https://github.com/grafana/website/pull/10709

Announcement:

• In our internal slack channels
• In our community slack channel
• https://community.grafana.com/c/plugin-development/plugin-announcements/47
• Optional: Blog post about why, linking to the "how to migrate" doc
• https://grafana.com/docs/grafana/latest/developers/plugins/migration-guide/

• The datasource.ts should implement the DataSourceWithBackend if the user opted in for backend when scaffolding. This is so we get the default implementations of health checks etc.

keywords is a required field based on our plugin.json schema.

When running this in terminal I'm getting some deprecated dependency warnings. Would nice to clean those up.

npx @grafana/create-plugin migrate
Need to install the following packages:
  @grafana/create-plugin
Ok to proceed? (y) y
npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-resolve#deprecated

Challenge: detect if the current go environment is capable of running these commands. If so, try to run:

Run go get -u github.com/grafana/grafana-plugin-sdk-go
Run go mod tidy

to both update the go sdk and set go the dependencies

ideas to detect a go environment:

• Run go version and detect if it returns a version and exits 0
• Check for ENV variables
• Run go env and detect if the environment is correct (GOPATH)

This is a question about if it makes sense to remove the possibility to scaffold a datasource plugin without a backend. Let me elaborate a bit on this.

If you make sure to write your datasource with a backend it will automatically be available to run queries for alerting, expressions, remote plugins, store secret credentials etc.

So there are multiple benefits of cater for datasource with backend and try to deprecate the "datasource without backend" support.

Once #72 in merged we need to address the following things:

• Define what to do with version parameter of ManifestInfo and adjust the code accordingly #82
• Rename repo to grafana/plugin-tools
• Change the references in this PR's readme and other files from create-plugin to plugin-tools #83
• #118
• Improve sign-plugin documentation #83

Package Name

create-plugin

What happened?

Whilst trying a migration I spotted some console errors during tests which we squashed in toolkit here

What you expected to happen

For tests to pass (which they do) without the console error noise.

How to reproduce it (as minimally and precisely as possible)

1. Clone https://github.com/grafana/athena-datasource
2. Run create-plugin migrate
3. Set react and react-dom to 17.0.1 in the package.json
4. Run 'yarn'
5. Run 'yarn test'

Environment

System:
    OS: macOS 12.6
    CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
    Memory: 26.59 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.17.1 - ~/.nvm/versions/node/v16.17.1/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 8.15.0 - ~/.nvm/versions/node/v16.17.1/bin/npm
  Browsers:
    Chrome: 106.0.5249.119
    Firefox: 104.0.2
    Safari: 16.0
  npmPackages:
    @grafana/aws-sdk: 0.0.37 => 0.0.37
    @grafana/data: 8.2.1 => 8.2.1
    @grafana/e2e: 8.2.1 => 8.2.1
    @grafana/e2e-selectors: 8.2.1 => 8.2.1
    @grafana/eslint-config: ^2.5.0 => 2.5.2
    @grafana/experimental: ^0.0.2-canary.39 => 0.0.2-canary.39
    @grafana/runtime: 8.2.1 => 8.2.1
    @grafana/tsconfig: ^1.2.0-rc1 => 1.2.0-rc1
    @grafana/ui: 8.2.1 => 8.2.1

Additional context

No response

Package Name

create-plugin

What happened?

Generated links in plugin.json incorrectly points to the starter template, example:
https://github.com/grafana/grafana-plugin-examples/blob/784d7d0eddc1c2cea49ab175cd180364a39a69fe/examples/datasource-http-backend/src/plugin.json#L20-L29

What you expected to happen

Either no links to be generated or the links to be generated based on the GitHub repository/plugin being generated.

How to reproduce it (as minimally and precisely as possible)

N/A

Environment

N/A

Additional context

Discussed here: grafana/grafana-plugin-examples#103 (comment)

Based on conversations / feedback within the design doc we’re favouring Proposal 4: Auto as the tool to automate publishing packages.

It suggests shipping every PR change however due to the limitation related to merging whilst another is publishing we want to label PRs with a “release” label to workaround the issue

• Ask if should create the directory if $CWD is not empty
• Validate name and organization ids

We should not prompt to confirm to delete

Package Name

create-plugin

What happened?

Whilst migrating the clock-panel I noticed that the scaffolded docker-compose.yaml had incorrect strings based on the output from normalize_id. I think it best when migrating a plugin that we don't try to generate the plugin id for these files but pull it from the plugin.json where it's already defined.

What you expected to happen

For plugin id in the scaffolded files to match the original id.

How to reproduce it (as minimally and precisely as possible)

1. clone [email protected]:grafana/clock-panel.git
2. Run npx @grafana/create-plugin migrate
3. Answer yes to all prompts.
4. Open docker-compose.yaml. The container_name and volume mounts are incorrect. container_name: 'rafanaabs-lock-panel'

Environment

System:
    OS: macOS 12.6
    CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
    Memory: 237.28 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.17.1 - ~/.nvm/versions/node/v16.17.1/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 8.15.0 - ~/.nvm/versions/node/v16.17.1/bin/npm
  Browsers:
    Brave Browser: 106.1.44.112
    Chrome: 106.0.5249.119
    Chrome Canary: 86.0.4239.0
    Firefox: 104.0.2
    Firefox Nightly: 81.0a1
    Safari: 16.0
    Safari Technology Preview: 16.4
  npmPackages:
    @grafana/data: 8.0.0 => 9.1.2 
    @grafana/e2e: 9.1.2 => 9.1.2 
    @grafana/e2e-selectors: 9.1.2 => 9.1.2 
    @grafana/eslint-config: ^2.5.0 => 2.5.2 
    @grafana/runtime: 8.0.0 => 9.1.2 
    @grafana/tsconfig: ^1.2.0-rc1 => 1.2.0-rc1 
    @grafana/ui: 8.0.0 => 9.1.2

Additional context

No response

Yarn (and I'm guessing all package managers) sort dependencies alphabetically after adding another package to a project (e.g. yarn add @types/lodash). During the migration of a plugin we "add" dependencies into the package.json but the result isn't sorted.

We have had people ask on slack and in Github to make toolkit work within a mono-repo. Rather than invest time in getting toolkit to support mono-repos I think adding mono-repo support to this package would get quite a bit of buy in from plugin developers.

We could likely use this monorepo of plugins as reference to add this feature. All plugins within it are "scaffolded" or "migrated" by this project. They have then been manually tweaked to build within a mono-repo. However right now tests and e2e tests have not been catered for. We can probably ignore the webpack config extensions to each plugin too.

When running webpack we should not remove the built backend binaries from the dist folder because that will require the backend to be rebuilt everytime

The problem

Currently it's pretty hard to make sure that our templates are free of ESLint or Prettier errors, as the templates can contain templating-language specific expressions, which makes it almost impossible to run any linter on them before they would be generated.

A proposed solution

We could run ESLint and Prettier on the generated file during generation time, for example as a Plop action.

When migrating a plugin we shouldn't tamper with the versions of @grafana dependencies that already exist in a plugins package.json. Doing so will force plugin developers to have to fix all sorts of things (type errors, breaking changes in APIs, changes to e2e-selectors) for the tasks to work again. It will likely cause their plugin to stop working in the minimum version of Grafana that it was previously working in which I doubt a plugin developer wants or expects from the migration.

When running the migration, I ended up with a new docker-compose.yml file, and it used the plugin "name" instead of the "id" in a couple of locations.

It also detects that I had a docker-compose.yaml file and wanted to delete it.

Maybe detect if the repo is using .yaml and skip the migration for it? or rename/merge in changes?

• Improve the post-migration message in create-plugin to link to the migration docs.
• Expand the current migration docs to expand on the following topics:
  • Custom tsconfig
  • Custom webpack
  • How to update the configuration

• The react components should be grouped in a src/components folder.
  -webpack configs should be placed in a .webpack folder. -> we have our own .config now.
• We should not scaffold the plugin.json with the >=8.0.0 version but instead do ^8.0.0
• Getting started section of README.md should append the necessary partial rather than -- APPEND GETTING STARTED HERE --

Currently generated scripts have no signing command, our CI relies on this command being there

• https://github.com/grafana/plugin-tools/blob/main/packages/create-plugin/templates/github/ci/.github/workflows/release.yml#L80

We should adjust the generated command to include sign and to utilize the new sign package

When using toolkit create-plugin:migrate this gets generated:

    "build": "TS_NODE_PROJECT=\"./.config/webpack/tsconfig.webpack.json\" webpack -c ./.config/webpack/webpack.config.ts --env production",
    "dev": "TS_NODE_PROJECT=\"./.config/webpack/tsconfig.webpack.json\" webpack -w -c ./.config/webpack/webpack.config.ts --env development",

but that doesn't seem to be valid in windows - you can't set environment variables that way in windows. I tried seeing if webpack pays attention to config but it doesn't seem to.

 "scripts": {
    "build": "webpack -c ./.config/webpack/webpack.config.ts --env production",
    "dev": "webpack -w -c ./.config/webpack/webpack.config.ts --env development",
     /* ... */
  },
  "config": {
    "TS_NODE_PROJECT": "./.config/webpack/tsconfig.webpack.json"
  },

Initially this seemed to work as it runs the right thing:

yarn run v1.21.1
$ webpack -w -c ./.config/webpack/webpack.config.ts --env development

However, after that there are a lot of other errors. They all seem to be related to the fact that config didn't actually set TS_NODE_PROJECT...

I noticed webpack has an --env parameter, so I tried this:

    "dev": "webpack -w --env TS_NODE_PROJECT=./.config/webpack/tsconfig.webpack.json -c ./.config/webpack/webpack.config.ts --env development",

but this didn't work. Reading the docs on that that sets a webpack environment variable, which is not the same as an OS environment variable.

Chasing through the cause of this, I found this old webpack issue that completely describes the problem and suggests suggest two ideas:

• Install cross-env and set the variable that way, also suggested as a solution here
• Manipulate the environment variable somewhere else, like in webpack.config.ts

That webpack ticket is followed up with some webpack documentation that suggests the cross-env solution is the right one, though I suspect since grafana generates the webpack.config.ts they might have other ideas that don't bring in another dependency.

Some plugins have custom scripts that use grafana-toolkit. We can detect them and prompt the user to delete them or replace them with alternatives.

Original scripts of a migrated plugin:

The diff of a migrated file.

Similar to #108 other dependencies like react-colorful or esm-browser are causing problems because they need to be transpiled before jest can loaded.

Context

When jest load tests it also needs to load the code dependencies from node_modules. Those dependencies sometimes need transformation before jest can load them otherwise nodejs can't run them. e.g.: because they are written in typescript or they are ESM that some nodejs version don't understand.

Jest has a configuration list of regex that tells it which packages should not transform, by default jest won't transform any module inside node_modules.

This previous PR created an exception for the package ol: open layers but there are much more packages that will have this problem and fail to run inside jest.

Suggested solutions:

• Create and maintain a list of packages that should be transformed by jest and keep the list up to date
• Ask jest to transform all packages regardless if they need transformation or not
• Let the users maintain the list of packages on their own

There is an convention that the plugin should be named:

• graph-panel - panels should end with "-panel".
• time-datasource- datasources should end with "-datasource".
• extra-app - apps end with "-app".

Would be nice to get this automatically added when you scaffold without ending in a situation where we get names like graph-panel-panel because the user types graph-panel as the name.

"id": {
  "type": "string",
  "description": "Unique name of the plugin. If the plugin is published on grafana.com, then the plugin id has to follow the naming conventions.",
  "pattern": "^[0-9a-z]+\\-([0-9a-z]+\\-)?(app|panel|datasource)$"
},

Do we need to add the plugin type on the end for this plugin schema pattern?

Originally posted by @jackw in #2 (comment)

Example checks:

• Go environment
• nodejs environment
• docker
• etc...

We don't support backend for panel plugins and the current generated one doesn't work because the plugin json is incomplete.

It would be helpful to add an e2e step to the generated CI workflows to run e2e tests as a part of the pipeline.

e.g. https://github.com/grafana/grafana-plugin-examples/blob/master/examples/panel-basic/package.json#L8

Maybe make it optional somehow(?) to skip the step if there is no tests provided or run always and make a no-op command out of it if irrelevant or no tests provided.

Toolkit's webpack config currently searches all subdirectories in a plugins src directory looking for entry points (any file named module.ts, module.js or module.tsx). This gives it support for bundling nested plugins. Do we want to add this functionality to the create-plugin webpack config?

The final config passed to webpack would need to look something like the following:

// webpack.config.js
context: path.join(process.cwd(), SOURCE_DIR),
entry: {
  module: './module.ts',
  'redis-cli-panel/module': './redis-cli-panel/module.ts',
  'redis-cpu-panel/module': './redis-cpu-panel/module.ts',
  'redis-gears-panel/module': './redis-gears-panel/module.ts',
  'redis-keys-panel/module': './redis-keys-panel/module.ts',
  'redis-latency-panel/module': './redis-latency-panel/module.ts',
},

The alternative would be to promote the idea of monorepos of plugins where each plugin is built and published separately and the app plugin would then rely on dependencies.plugins prop in plugin.json to install all it's "nested" dependencies.

Things I noticed when using the scaffolder to create examples in this repo: https://github.com/grafana/grafana-plugin-examples/tree/mckn/basic-ds-example/examples/static-datasource

The scaffolded project...

• ...contains a yarn.lock should we exclude that and let it be generated when running yarn?
• ...has @latest as version tag for the Grafana dependencies. Shouldn't we generate the exact version instead?
• ...when running webpack commands I get the following error (node:32889) [DEP_WEBPACK_MODULE_ERRORS] DeprecationWarning: Module.errors was removed (use getErrors instead) (Use node --trace-deprecation ... to show where the warning was created)
• add cypress/videos to .gitignore

Lint Plugin action should be switched to using plugincheck2

Source code:
https://github.com/grafana/create-plugin/blob/main/templates/github/ci/.github/workflows/release.yml#L120-L126

To consider:

• Should we use a special -config in the Lint Plugin flow e.g. https://github.com/grafana/plugin-validator/blob/master/config/publishing.yaml?

previous plugin had

"devDependencies": {
"@types/lodash": "4.14.185",
}

after migrate was run, package.json contained:
"dependencies": {
"@types/lodash": "last",
}

along with the existing devDependencies entry, and yarn gives a warning

warning package.json: "dependencies" has dependency "@types/lodash" with range "latest" that collides with a dependency in "devDependencies" of the same name with version "4.14.185"

We should probably move this to devDependencies and look for a duplicate in dependencies.
Also - is "latest" correct, or should it match the targeted version of Grafana?

the generated template volume map includes a comma at the end, resulting in:

   - ./dist:/var/lib/grafana/plugins/dotdiagram-panel,

which then errors out on docker-compose up