firstly thanks for this bosh release. got it deployed and hooked up to our graphite instance pretty easily!! I then tried to setup AWS cloudwatch as a datasource in Grafana as per http://docs.grafana.org/datasources/cloudwatch/

Unfortunately grafana produced a HTTP Error Internal Server Error in the browser and the following in the logs

2016/08/29 11:01:05 [cloudwatch.go:139 handleListMetrics()] [E] Unable to call AWS API: NoCredentialProviders: no valid providers in chain

the Grafana docs imply it's looking for AWS credentials in $HOME/.aws/credentials which i had assumed to be /home/vcap/.aws/credentials but that didn't work at all.

I'm not certain, but i believe the problem might be due to the grafana_ctl start/stop script launching the grafana-server process with the chpst command.
exec chpst -u vcap:vcap $PACKAGE_DIR/bin/grafana-server results in the grafana-server process running as user vcap, but its environment variables including $HOME are for the user running the grafana_ctl script (root). It therefore is trying to access /root/.aws/credentials for setting up a datastore.

Does this seem likely? Or have you managed to get the cloudwatch datastore working?

Thanks,
Michael

Hi Alex,

I am currently preparing a new boshrelease with grafana-2.6 (I'll create a pull request within the next week) and I am wondering which of the new options to pass through as properties to fill in.
I started with the grafana-2.6/conf/samples.ini file from the tar ball and inserted all the current configurable options we have in config.ini.erb.
The new options I would like to make configurable also are the ones in the [auth.proxy] section.

Anything else I should add immediately? Or would you prefer to discuss this when I have the pull request ready?

Best Regards,
Holger.

Hi @vito, any thoughts about upgrading this older grafana to a later 5.2*?

Thank you

In the latest Grafana release, 4.3.2 many issues have been fixed. https://github.com/grafana/grafana/releases

We are currently using an old bosh release of Grafana, which you have created.
We would highly appreciate, if you could create a bosh release of the latest Grafana version 4.3.2. https://github.com/grafana/grafana/releases

I really dislike to fork your repository https://github.com/vito/grafana-boshrelease. But if you tell me that you won’t do it anytime soon, I will do it.

Kind Regards

Grafana 4 has been released for literally hours! I don't think there are any relevant breaking changes in the changelog.

https://github.com/grafana/grafana/releases/tag/v4.0.0

Issue

We got bosh failing due a post-start task failure:

  Started updating job graphite > graphite/0 (51497a43-3b0c-4906-9103-30a1c4f07031). Failed: Action Failed get_task: Task 5ef4013e-2beb-4440-5600-550587d8da1e result: 1 of 1 post-start scripts failed. Failed Jobs: grafana. (00:02:05)

Error 450001: Action Failed get_task: Task 5ef4013e-2beb-4440-5600-550587d8da1e result: 1 of 1 post-start scripts failed. Failed Jobs: grafana.

Rerun the deployment works.

Root cause

The reason was that the DB migrations took a while, /var/vcap/sys/log/grafana/create-update-datasources.log:

2016/07/01 14:59:10 [I] Starting Grafana
2016/07/01 14:59:10 [I] Version: 3.0.3, Commit: v3.0.3, Build date: 2016-05-23 08:58:31 +0000 UTC
2016/07/01 14:59:10 [I] Configuration Info
Config files:
  [0]: /var/vcap/packages/grafana/conf/defaults.ini
  [1]: /var/vcap/jobs/grafana/config/config.ini
Paths:
  home: /var/vcap/packages/grafana
  data: /var/vcap/store/grafana
  logs: /var/vcap/sys/log/grafana
  plugins: /var/vcap/store/grafana/plugins

2016/07/01 14:59:10 [I] Database: sqlite3
2016/07/01 14:59:10 [I] Migrator: Starting DB migration
2016/07/01 14:59:10 [I] Migrator: exec migration id: create temp user table v1-7
2016/07/01 14:59:22 [I] Migrator: exec migration id: create index IDX_temp_user_email - v1-7
2016/07/01 14:59:34 [I] Migrator: exec migration id: create index IDX_temp_user_org_id - v1-7
2016/07/01 14:59:47 [I] Migrator: exec migration id: create index IDX_temp_user_code - v1-7
2016/07/01 14:59:59 [I] Migrator: exec migration id: create index IDX_temp_user_status - v1-7
2016/07/01 15:00:10 [I] Migrator: exec migration id: Add column updated_by in dashboard - v2
2016/07/01 15:00:24 [I] Migrator: exec migration id: Add column created_by in dashboard - v2
2016/07/01 15:00:43 [I] Migrator: exec migration id: Add column with_credentials
2016/07/01 15:00:58 [I] Migrator: exec migration id: create quota table v1
2016/07/01 15:01:08 [I] Migrator: exec migration id: create index UQE_quota_org_id_user_id_target - v1
2016/07/01 15:01:18 [I] Migrator: exec migration id: create plugin_setting table
2016/07/01 15:01:29 [I] Migrator: exec migration id: create index UQE_plugin_setting_org_id_plugin_id - v1
2016/07/01 15:01:44 [I] Migrator: exec migration id: create session table
2016/07/01 15:01:44 [I] Migrator: exec migration id: Drop old table playlist table
2016/07/01 15:01:44 [I] Migrator: exec migration id: Drop old table playlist_item table
2016/07/01 15:01:44 [I] Migrator: exec migration id: create playlist table v2
2016/07/01 15:01:44 [I] Migrator: exec migration id: create playlist item table v2
2016/07/01 15:01:44 [I] Migrator: exec migration id: drop preferences table v2
2016/07/01 15:01:44 [I] Migrator: exec migration id: drop preferences table v3
2016/07/01 15:01:44 [I] Migrator: exec migration id: create preferences table v3
2016/07/01 15:01:44 [I] Plugins: Scan starting
2016/07/01 15:01:44 [W] Plugins: Plugin dir /var/vcap/store/grafana/plugins does not exist
2016/07/01 15:01:44 [I] Plugins: Plugin dir /var/vcap/store/grafana/plugins created
2016/07/01 15:01:44 [I] Listen: http://0.0.0.0:3000

So the the post-start task failed, even with the 3 retries with 10 seconds wait in the middle

# cat /var/vcap/sys/log/grafana/create-update-datasources.log 

Cannot connect to Grafana. Retrying ...

Cannot connect to Grafana. Retrying ...

Cannot connect to Grafana. Retrying ...

Error: Cannot connect to Grafana after 3 retries.

Expected

It should not fail :)

Workaround

rerun deployment. As the DB is already migrated, it will pass.

Proposed solutions

We can either:

• Increase the retries to 10 retries, for instance
• wait for the port to be open
• Simple ignore this issue :)

Hi @vito,

we would like to use your bosh-release and to contribute. But as a prerequisite we first need a license. We created a pull request with a apache license 2.0 (#1) for your convenience (other licenses might be
OK for us too, but apache one of the easiest for us to consume and it is also used in other cloudfoundry projects).

A question: Could this project be moved to one of the cloudfoundry organisations? (perhaps cloudfoundry-community?) That would make it easier for us to pass our own corporate legal requirements.

Best regards,
Holger.

Allowing plugins to be specified via the deployment manifest would be great. We use a plugin for displaying histogram data, and if we do a bosh redeploy that causes the VM to be recreated, we need to manually re-add the plugin.