datadog / kvexpress Goto Github PK

It's likely how I've programmed it, but I'd like it to just return nil and keep on moving.

It's pretty ugly right now.

We sometimes need to use them.

Right now there's no token awareness at all.

Add flag and exec that command.

Right now there's no docs.

Would make it easier for people to start.

So that files can land with different owners / groups.

Just to see - don't see any performance problems at the moment.

Don't want to hang waiting.

You end up in this state:

[staging]root@i-c972d41e:/etc/consul-template/output# ls -al
total 48
drwxr-x--- 2 root root  4096 Nov  3 22:39 .
drwxr-x--- 7 root root  4096 Aug  7 16:33 ..
-rw-r--r-- 1 root root 16516 Nov  3 22:39 config_hosts
-rw-r----- 1 root root 16335 Nov  3 22:39 config_hosts.compare
-rw-r----- 1 root root    22 Nov  3 22:39 config_hosts.last

Where - the files didn't match - but the checksum inside the KV store matched:

[staging]root@i-c972d41e:/etc/consul-template/output# shasum -a 256 config_hosts.compare
deab902e0ab2f432f0ebc68035992f10a743059222cbea991f3345ba195c4dad  config_hosts.compare

So it didn't update the KV store - but it leaves the .last file in the wrong state.

Replaces kvexpress.rb.

Isn't needed and doesn't add anything in this context.

Investigate.

Sent to dogstatsd.

Much easier than sshing into the web interface. Finding the token to look at the KV store and setting it there.

Right now - it's set to always connect.

That way we can easily do that programmatically from capistrano or consul exec

Right now I'm doing it manually which is error prone.

Will take a look at it later and continue the rest for now.

Example output:

Oct 30 15:48:58 i-c972d41e kvexpress:  2015/10/30 15:48:58 in: file='compare' full_path='/etc/consul-template/output/etc/consul-template/output/config_consul_config.ini.compare'
Oct 30 15:48:58 i-c972d41e kvexpress:  2015/10/30 15:48:58 in: file='last' full_path='/etc/consul-template/output/etc/consul-template/output/config_consul_config.ini.last'

Like an actual command in the root rather than how I've done it?

Seems sort of hackish.

There's none at the moment.

Would be nice to have some automated ones.

To grab a file from a URL and place it as a file.

So that we could grab files as a replacement if necessary.

Make it smarter.

Setup the connection once per binary invocation.

Maybe we can obfuscate it a bit - and show beginning and end?

Do we want to bother to do anything with it in kvexpress?

Right now we're ignoring it.

Would this be a problem when deploying new nodes?

We "say" stop - but new node still needs the files.

Needs that command.

That way - when we hit stop:

1. Nothing goes in.
2. Nothing comes out.

Not needed when we go to production.

Too much output.

So that we can make sure we're not getting too big for Consul's KV store. The current limit is 512KB.

10/22/15 10:08:06.000 PM kvexpress[78395]: 2015/10/22 22:08:06 out: file_wrote='true' location='file' permissions='416'

I want them to show 0640

For some of the values:

1. Datadog API and App keys.
2. Connection parameters that may never change.

Resurrect this branch:

https://github.com/darron/kvexpress/tree/darron/config-file

So that not everything restarts at the same time?

Or do we make it so that restarts are nice and smooth? (Probably better.)

Sticking this here to think about it.

Want to know when that's happening.

Of course - without a config file that already has the APP key and API key it's less than useful.

If a file already exists, and it's the same stuff. Don't write it.

Would make it easier to adjust as needed.

To deal with possible race condition where:

1. We're watching the "data" key.
2. We update the "data" key - but not the "checksum" or the "updated" keys.
3. Watches fire - and they checksum against the old version.

A possible alternative solution:

May need to instead watch the "updated" key as that's updated last.

Be able to create a specific version for distribution.

To allow for some code reuse between the out and in commands.

Testing on staging.

Not sure if it's related to the 0.6 upgrade.

For times when you've got it stopped but still want to be able to pull stuff out.

So that we can put larger files in AND/OR use less network traffic.

That way we can put stuff there too - would be easier.

With the --sorted flag - if there are lots of blank lines then it keeps them all.

Need to remove them - don't want to shell out to uniq to do it.

Which is quite nice - helps with all sorts of debugging - but will need to be tuned when we're in production.

Make a way to do that.