Since not everyone cares about Orgs we should gracefully fail if Orgs are not readable rather then preventing the user from accessing the dashboards.

Makes it a bit easier to work with and allows for go install github.com/netsage-project/gdg@latest to work.

Previous behavior would have one pattern for make install and another for the native go tools. This brings a bit more consistency to the project though may break backward compatibility with previous releases.

When reading data and writing data supporting symlinks would be nice to have.

aka. ln -s dashboards/prod dashboards/qa would not work currently and will error out because the 'folder' qa does not exist.

Optionally adding support for importer.yml to follow symlinks would be good as well.

It would be interesting to explore the ability to generate a report between:

• Server A and Server B
• backup folder and Server
• two backup locations to generate a diff.

This is a bit of a more advanced feature, and initially looking at focusing it on Dashboards/DataSources.

Add the ability to only import a single dashboard or single datassource.

Hi @safaci2000
after I set a wrong context, gdg can't set a correct one.
$ gdg ctx set -c qa
$ gdg ctx list
┌──────┬────┐
│ CONTEXT │ ACTIVE │
├──────┼────┤
│ production │ false │
│ qa │ true │
└──────┴───┘
$ gdg ctx set -c abc
$ gdg ctx list
time="2021-06-24T10:07:40+02:00" level=error msg="Context is not found. Please check your config"
$ gdg ctx set -c qa
time="2021-06-24T10:07:47+02:00" level=error msg="Context is not found. Please check your config"

With the last command I can't set a correct context, so I think it's need to verify the context correctness.
The only way I found to go on it's to edit importer.yml.

It would be great if we had this to manage Teams as well as Users
see: https://grafana.com/docs/grafana/latest/http_api/team/

The number of tests and test coverage is very limited, we should add more tests in general.

The switch to interface allows for more mocking as needed. We should expand the coverage we currently have.

Our Grafana is running behind a self signed certificat.

I get this error while executing dash list

time="2021-11-29T13:39:10+01:00" level=warning msg="Error getting organizations: Get \"https://xxx.xxx/api/orgs\": local error: tls: no renegotiation"
time="2021-11-29T13:39:10+01:00" level=fatal msg="Failed to retrieve dashboardsGet \"https://xxx.xxx/api/search?type=dash-db\": local error: tls: no renegotiation"

Creating a new context using the wizard with a camel case entry like 'FooBar' causes the app to fail.

Hi @safaci2000
I think it would be useful to import and export a single dashboard.
A use case can be to copy a single dashboard for one organization to another when you have a lot organization and don't want, or can't, copy all dashboards every time.

gdg ctx set -c context_a
gdg dashboard import my-dash
gdg ctx set -c context_b
gdg dashboard export my-dash

Any idea is appreciated, I'm waiting of your feedback to start coding.

Hi @safaci2000
can you use https://github.com/grafana-tools/sdk instead of your forked repo https://github.com/netsage-project/sdk ?
I think it's better for security reasons

It would be nice if we could specify environment variables in the configuration file importer.yml

This will enable us to isolate secrets (tokens and passwords), and safely commit the configuration file to version control.

Current work for grafana is looking at generating go-client from swagger models. grafana/grafana#47827 Seems Alpha still.

Likely we should migrate to using this: https://github.com/grafana/grafana-api-golang-client as it's officially supported.

I'd like to do a quick POC and make sure it supports all of our needs. If so we can start migrating to it.

Otherwise, we can maintain our own version based on the swagger code. esnet/ has a repo that does just that.

• Add support for New API
• Migrate Orgs CRUD
• Migrate Dashboard CRUD
• Migrate Datasources
• Migrate Alert Notifications
• Migrate Users
• Migrate Folder CRUD
• ServerInfo

Hi all,

I have multiple grafana instances with many folders and wanna sync my folders from one dc to another. Can we have an option to import a folder from grafana at DC1 to my local folder dashboards/DC1 and then switch context to DC2 and export dashboards/DC1 to grafana at DC2? let me also say that I don't want to change output_path of dc2 to dashboards/DC1. I need it for example as an command line argument.

Thanks in advance.

GoReleaser supports releasing .deb, .rpm, etc.. using their automated pipeline. Look at potentially adding support for it via:

https://goreleaser.com/customization/nfpm/

Create a wizard to generate a new configuration and save it to the default importer.yml

Hi @safaci2000
Can we support multiple Grafana like wizzy?
E.g: switch to Grafana dev

./bin/grafana-dashboard-manager set context grafana dev

Thank you,

Explore Uploading/Pulling data from GS/S3.

Potential useful implementation/ abstraction to support multiple cloud providers: https://github.com/graymeta/stow

Currently if we have 7 DataSources that all use the same credentials, we'd have to have 7 entries for each one of them.

It would be nice if we can map the DS based on the URL and other meta data rather than the name if they're all pulling from the same source.

Summary

AFAIK currently there is no way to list / export / import / delete alerts set in Grafana. We would like to start exporting them so that we can have a back up in git.

Features

I would like to implement the following commands:

./gdg alerts list
./gdg alerts import
,/gdg alerts export
./gdg alerts clear

Also, add a new block into the configuration file / parser. Something like:

contexts:
  all:
    dashboards_output: "dashboards"
    alerts_output: "alerts"

When exporting / importing, we would consider all of the alerts, just like with datasources. Meaning it would not be possible to export / import specific alerts.

Notes

• API documentation https://grafana.com/docs/grafana/latest/http_api/alerting/
• I very much won't mind implementing this, but I will have time for this at best at the start of November. So I am creating an issue to first ask whether there are any plans for this, or whether by any chance you dont have a partial code for it already. 🤷

Apple binary are a bit cranky due to missing a cipher key. Look into getting gorelease to sign the binaries as they are released and tagged.

Is your feature request related to a problem? Please describe.
We should document properly how to install the GDG binary.

Describe the solution you'd like
The generated docs as well as possible the README should call out how to install gdg

Thank you for this project!

When listing/importing dashboards from a Grafana instance with more than 1000 dashboards, you only get the dashboards within the watched folder that were returned in the first 1000 results. As result, in my case I only imported 37 dashboards instead of all 225.

I have no immediate need for this feature, but hopefully it could safe others some time or trouble in the short term.

Support Organizations CRUD via API.

Are there any plans to add the ability to backup to aws s3

Is your feature request related to a problem? Please describe.
Since Datasources are not able to be place in Orgs or folders. The only way to import / manage a subset is by utilizing regex or type filters.

Describe the solution you'd like
We should allow for one or multiple types of regex patterns to apply. Ideally chained using '|' (pipe) character. Also support for a list of support data types: aka only import elasticsearch and Google Sheets and ...etc. exclude all others.

Describe alternatives you've considered
Organizations and Folders none are currently supported.

For those that really only are looking for backup/restore functionality it would be really helpful to be able export and import all dashboards irregardless of filters. Managing an up-to-date list of folders just seem like unecessary work.

Sorry if this is already possible, all my attempts with various blank or wildcard or no filters/watched folders have resulted in either General folder or nothing being exported.

Currently the CI/CD is trying to publish documentation. Create a new task that builds but does not push to validate the docs without requiring credential access.

The ability to sync folders is also fully supported by the SDK and should be easy enough to implement.

If grafana has an invalid certificate there is no way to use GDG with the grafana instance

Import, Export, clear should be added as well to make it a complete CRUD.

Optionally adding a set to set the remote Org to a new value.

https://goreleaser.com/customization/homebrew/ HomeBrew support

Ability to import/export single dashboards by slug instead of entire grafana instance or entire folder.

Is your feature request related to a problem? Please describe.
Gdg must be compiled and cannot be installed via brew.

Describe the solution you'd like
Provide a tap to install gdg directly via brew on mac. See brew docs.

Describe alternatives you've considered

Additional context
Maybe I find time to create a PR, but would be interesting whether this was already thought of or why this does not yet exist.

Hi,

We are trying to export ds JSON file but getting the below error, so can you please help here

gdg@aa6ed4e8cc6c:/opt/gdg/bin$ gdg ds export
time="2022-03-29T05:22:57Z" level=info msg="Exporting datasources"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x86b426]

goroutine 1 [running]:
github.com/netsage-project/gdg/api.(*DashNGoImpl).ExportDataSources(0xc0002a96b0, {0xad3cb8, 0xc0002a9cb0})
/go/src/github.com/netsage-project/gdg/api/datasources.go:108 +0x546
github.com/netsage-project/gdg/cmd.glob..func21(0xde3700, {0x9f53d8, 0x0, 0x0})
/go/src/github.com/netsage-project/gdg/cmd/datasourcesExport.go:16 +0x83
github.com/spf13/cobra.(*Command).execute(0xde3700, {0xffe1d0, 0x0, 0x0})
/go/pkg/mod/github.com/spf13/[email protected]/command.go:860 +0x5f8
github.com/spf13/cobra.(*Command).ExecuteC(0xde4100)
/go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3bc
github.com/spf13/cobra.(*Command).Execute(...)
/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
github.com/netsage-project/gdg/cmd.Execute()
/go/src/github.com/netsage-project/gdg/cmd/root.go:40 +0x25
main.main()
/go/src/github.com/netsage-project/gdg/main.go:14 +0x4a

JSON file:-
{
"orgId": 1,
"name": "AlertManagerDS",
"type": "camptocamp-prometheus-alertmanager-datasource",
"typeLogoUrl": "public/app/plugins/datasource/prometheus/img/prometheus_logo.svg",
"access": "proxy",
"editable": "false",
"url": "http://xxxxxxxxxxxxxxxx.svc",
"jsonData": {
"severity_critical": "critical",
"severity_high": "error",
"severity_warning": "warning",
"severity_info": "information"
}

Apparently, Grafana does not plan to support importing library panels through POST API and they want people to use import dashboard API that is not documented.

grafana/grafana#44003
grafana/grafana#43525

It would be great if you could use that import API to support that feature.

I was planning on writing PR, but currently have busy schedule.

The Path of where the DataSource and Dashboards are pulled from should be simplified.

GDG should only configure the base path after which the app should construct the full path.

Aka. if prefix = "test" then the dashboards will be test/dashboards and datasources in test/datasources rather then needing the user to choose a path for both.

Additional Updates needed

• Needs Documentation updates.
• Needs ctx new update

It looks like sometime around 8.3.6 (first time I noticed it), Grafana changed the way the dashboards refer to a datasource. It now refers to them by uid instead of by name. The problem with this is that the uid seems to change across deployments meaning when you export the dashboards it can't find any of the data sources. You can see this by running gdg dash import against a Grafana instance running 8.3.6 then running gdg dash export.

Specifically, when you'd gdg dash import with previous Grafana versions, the data sources in the dashboard json looks something like following:

{
      "datasource": "Flow - Rollup by Edge ASNs (5m)",
      ...
}

With 8.3.6 it looks like the following:

{
      "datasource": {
        "type": "elasticsearch",
        "uid": "VuEGTpa7k"
      }
      ...
}

hi
The authentication with Auth Token doesn't seem to work, even when created with Admin permission.
Is it the API key I need to create or am I missing something here?
Thanks a lot!

Use the CI to do a release when a new tag has been pushed.

Create a way to easily switch from grafana-tools to netsage-project or another fork as needed.

go.mod's replace doesn't work with go install and other go patterns so ideally we'd regex replace or have a tool to manage it.

via your link here

Create a .md file for contributor guide and maybe add a github template for PRs.

Go install fails the checksum on the downloaded binary and the checksum provided by checksum database.

$ go install github.com/esnet/gdg@latest
go: downloading github.com/esnet/gdg v0.1.11
go: github.com/esnet/gdg@latest: github.com/esnet/[email protected]: verifying module: checksum mismatch
	downloaded: h1:CnDtASXmMzHJ9l2BX5zTnOpSpbQcVvko0pbW8vI/UrU=
	sum.golang.org: h1:0b0aapElYX2+d9RLvDoPOrWZZ1FbJoFwPtCLQVb1Tyc=
SECURITY ERROR
This download does NOT match the one reported by the checksum server.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.

https://go.dev/ref/mod#authenticating

https://grafana.com/docs/grafana/latest/http_api/other/

Add the ability to get the Server version. Based on the version we can set a min/max on API calls to ensure the correct call is used.

Add the ability to import/export a datasource by slug aka. single entry instead of the entire set.

Is your feature request related to a problem? Please describe.

Given two valid contexts, it would be nice to have the ability to synchronize a particular entity ie. dashboards between two contexts.

Source context could use grafana instance or the storage system as an input while the destination should always be a grafana insurance.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Example : gdg sync dashboards staging production
gdg sync datasources --backup staging production

As this is supported by the SDK it should be trivial to add support for a simple CRUD to pull and push and list notifications.

client.GetAllAlertNotifications
client.CreateAlertNotification

Look into using Hugo or similar to generate a pretty version of the markdown documentation pushed to gh-pages.