When there's a large number of credentials it's difficult to find a credential in the dropdown list when mapping a credential with a service. This input should be type to complete in a way that filters the dropdown. Maybe something like select2 or some other plugin for xeditable that works for this?

If a user attempts google authentication, but picks the incorrect domain confidant should redirect the user to an error page and log them out. Currently it just sends back 403s and gives them the interface, which is quite confusing.

The current google auth docs punt on anything related to setting up the application in Google, but those steps aren't really straightforward. We should have some basic docs here, ideally with some screenshots.

It's possible to completely setup KMS auth policy via IAM policy, rather than grants. It's actually a lot easier to maintain and it has a better user experience from the workflow perspective (create credentials, map to a new service in confidant, launch the service in AWS, no need to go back to confidant to create the grants).

We need to update the docs to describe the policy, though, since AWS's KMS docs for IAM policy are basically non-existent and we want very specific policy.

Right now Confidant requires auth to load basically anything, and gives back a 401 if auth fails. There's no way to log out either, except to clear your session token.

Rather than immediately starting google auth, we should show a dialog and provide a login button. When logged-in we should show a logout button.

It can be difficult to track a credential key back to its credential. The credential pairs are encrypted, so we can't have a search index for this. The most common use-case here, though, is to find a credential key when you already know the service. If we show the credential pair keys along with the credential names on the service page, people will be able to find the credential resource easily.

It seems swiftype eliminated their free plan for search and now the cheapest plan is $299/month. We need to find an alternative.

Right now it's possible for Confidant to generate its own tables, but it only occurs if you provide a dynamo url, which is generally only done in development. We should add a config option to control whether or not this occurs, so that even outside of AWS we can allow Confidant to generate its own tables.

Using a JSONAttribute in pynamodb, support extensible metadata on secrets. The use-case here is to keep the Confidant server dumb, but allow Confidant clients to be intelligent. For instance, we could have metadata like:

mode: 0600
path: /etc/mysecret
user: root
group: root

The server is simply storing data, but when the client fetches the secret, it can decide what it wants to do with the secret. In this case it would place the file in /etc/mysecret only accessible by root.

This is partially motivated by #1.

We should be able to handle sessions and auth without redis, especially since it's the only place we're using it.

The relatively positioned css works fine for desktop displays, but it makes things really awkward on mobile. We should likely have the sidebars be the initial view and when a resource is selected, it should hide the sidebar and show the resource, with a back button which hides the resource and shows the sidebar.

Confidant's web interface requires secrets, so we should ensure those don't need to live in the clear. Easiest option here is:

1. Add a script that can use Confidant's at-rest KMS key that'll read from a json (or yaml) file (or stdin) encrypt/base64 the contents and output a file (or write to stdout)
2. Modify settings.py to read the encrypted contents, decrypt it, then load it as a dict.
3. Read the settings from the dict.

From the Security Model page:

What an authenticated user can achieve
A user can view all secrets.
A user can view all secret to service mappings.
A user can create new revisions of a secret.
A user can create new revisions of a service mapping.

It would be grand to be able to have classes of users:

• admins, who have all power
• users, who are tied to the enumerated services by the admins (possibly through farming out to some LDAP system or other RBAC provider).

IOW, applying the principle of least privileges to the users.

I've followed http://lyft.github.io/confidant/advanced/contributing/#quickstart-for-testing-or-development on Fedora 23 and the application is failing to start due to SELinux:

type=AVC msg=audit(1459447090.791:1040): avc:  denied  { write } for  pid=12660 comm="java" name="dynamo" dev="dm-1" ino=1469994 scontext=system_u:system_r:svirt_lxc_net_t:s0:c373,c563 tcontext=system_u:object_r:mnt_t:s0 tclass=dir permissive=0

The application does start with SELinux permissive. Please consider using auto labeling which has been merged in docker 1.7+. http://www.projectatomic.io/blog/2015/06/using-volumes-with-docker-can-cause-problems-with-selinux/

selinux-policy-targeted-3.13.1-158.11.fc23.noarch
docker-1.9.1-6.git6ec29ef.fc23.x86_64

In the KMS payload for authentication, if scope exists, limit the authorization scope based on the subset of the user's scope and the scope provided in the token:

{'scope':{'Actions':['get_ssh_keys']}}

Using the insecure quickstart configuration in the docs and version c7ca4e8 on master:

New service, named "whiffleball", no creds...

confidant_1 |   File "/srv/confidant/confidant/authnz.py", line 156, in decorated
confidant_1 |     return f(*args, **kwargs)
confidant_1 |   File "/srv/confidant/confidant/authnz.py", line 106, in decorated
confidant_1 |     if check_csrf_token():
confidant_1 |   File "/srv/confidant/confidant/authnz.py", line 97, in check_csrf_token
confidant_1 |     if g.auth_type == 'kms':
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/werkzeug/local.py", line 343, in __getattr__
confidant_1 |     return getattr(self._get_current_object(), name)
confidant_1 | AttributeError: '_AppCtxGlobals' object has no attribute 'auth_type'
confidant_1 | 172.17.0.1 - - [18/Apr/2016:17:31:31 +0000] "PUT /v1/services/whiffleball HTTP/1.1" 500 - "-" "-"

New credentials, one row:

confidant_1 |   File "/srv/confidant/confidant/authnz.py", line 156, in decorated
confidant_1 |     return f(*args, **kwargs)
confidant_1 |   File "/srv/confidant/confidant/authnz.py", line 106, in decorated
confidant_1 |     if check_csrf_token():
confidant_1 |   File "/srv/confidant/confidant/authnz.py", line 97, in check_csrf_token
confidant_1 |     if g.auth_type == 'kms':
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/werkzeug/local.py", line 343, in __getattr__
confidant_1 |     return getattr(self._get_current_object(), name)
confidant_1 | AttributeError: '_AppCtxGlobals' object has no attribute 'auth_type'
confidant_1 | 172.17.0.1 - - [18/Apr/2016:18:02:40 +0000] "POST /v1/credentials HTTP/1.1" 500 - "-" "-"

This uses the service.env defined in the documentation.

We added support for DYNAMODB_CREATE_TABLE in 1.1. It would be very useful to also have this in 1.0, so that confidant can auto-create its tables.

The table definitions for dynamodb in the docs don't match up well with what AWS has in the console when you create tables and indexes. We should use similar info and we should show screenshots (and/or a screencast) of the process.

Title says it all :)

Hello

I configure oauth2 and obtain OAuth 2.0 credentials: client ID and Client secret.
But in Google Developers Console needed to specify a redirect URI:
"Authorized redirect URIs
For use with requests from a web server. This is the path in your application that users are redirected to after they have authenticated with Google. The path will be appended with the authorization code for access. Must have a protocol. Cannot contain URL fragments or relative paths. Cannot be a public IP address."

Which URI will be correct for user sending back to Confidant app?
Thanks

The repo isn't currently running tests on PRs. We should setup travis to run the tests on PRs.

For the autoscale group to launch correctly, it's necessary to know the elasticache IP address in the cloud-init.

One option here is to add a call in the cloud-init that looks up the elasticache IP address and injects it as an environment variable. Another (likely better option) is to fix issue #12, which would let us remove elasticache completely.

We're currently not doing anything with the payload of the KMS auth token. We need to standardize what should be in the payload and a good first starting point is an optional encryption token:

{'token': 'a_fernet_token'}

If token is in that payload, it should be used to encrypt the returned secrets from the get_service endpoint. The idea here is to protect against a couple threats:

1. In-transit theft of the KMS token or the returned secrets.
2. At-rest protection of the secrets on the client side. The secrets could be stored encrypted with a KMS data-key that's also encrypted on-disk. To access the secrets it would be necessary to make a KMS call to decrypt the data-key, which could be used to decrypt the secrets.

Let's say for instance we have multiple AWS accounts: sandbox, primary, security. We'll put confidant into the security account, and we'll store its KMS keys there. In the KMS key policy we'll allow security, development and production to use the authnz key, so that they can call get_service. The issue is at this point we're fully trusting the IAM policy of all accounts to properly limit access to services, whether or not those services are in their own account.

To support multi-account we'll need to add some information to the service, which allows us to scope the service to an account. A scoping that can work is: service -> account -> kms key. We'll create a KMS key for each account, then we'll add settings to KMS keys for each account. Our default will be backwards compatible, which is to allow the service to be fetched by any account (which will be our AUTH_KEY setting).

For now we'll only consider these settings for service auth. User auth will continue using the USER_AUTH_KEY and won't be scoped by account.

Fix the HTML and CSS as a followup to the release of confidant. Some lyft specific CSS classes and styles were used, which makes the frontend look a little odd in some cases.

Right now the process for building and deploying docs is manual. We should build docs and publish them whenever a merge occurs.

The api is currently undocumented. We should add swagger markup.

Confidant should create and store SSH host keys for each IAM role, so that they can be used as stable keys for autoscale groups. Should be delivered from the get_service route (/v1/services/id). Maybe this should be opt-in per service. Should be configurable feature so that it can be disabled by default.

Web auth is currently limited to google auth with a redis based session storage manager. This should be modular.

If a credential isn't mapped to a service, it won't be usable anywhere. People who are new to confidant often make this mistake, so let's show a warning dialog at the top of the credential if it isn't mapped to a service.

It would be nice if the Configuration section could be broken up into smaller readable sections. The way it stands now I am personally finding it hard to get the information I am looking for quickly.

Perhaps if it were something like :

Basics
Basics -> Configuration
Basics -> Configuration -> Environments
Basics -> Configuration -> Environments -> Development
Basics -> Configuration -> Environments -> Production

Maybe more granular would help the end user find/read about how to configure something more easily?

Also, when configuring for production...I am finding this process to be rather difficult. I have minimal AWS experience, so I am not sure exactly what needs to be done when setting up the IAM policies and the DynamoDB table that Confidant uses. Could you create a video showing how to setup Confidant in a production environment or at the very least provide a list of detailed steps to do so?

Thanks!

I am getting a 500 when i am trying to create a new credential with a single key that contains a multiline value:

Key: test
Value:

hello
world

Logs says the POST to save the credentials returns 200 OK. The subsequent GET returns 500. (See logs below).

If i create the credential containing a key and a single line value and later update it with a multiline value, it works as expected.

I am running the latest container image in the testing configuration that is documented here: http://lyft.github.io/confidant/advanced/contributing/#quickstart-for-testing-or-development

confidant_1 | 172.17.42.1 - - [06/Nov/2015:16:17:24 +0000] "POST /v1/credentials HTTP/1.1" 200 265 "http://localhost/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:39.0) Gecko/20100101 Firefox/39.0"
confidant_1 | 172.17.42.1 - - [06/Nov/2015:16:17:25 +0000] "GET /v1/credentials HTTP/1.1" 200 4246 "http://localhost/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:39.0) Gecko/20100101 Firefox/39.0"
confidant_1 | 172.17.42.1 - - [06/Nov/2015:16:17:25 +0000] "GET /v1/credentials/ea5cb10c685c4aa7b6030bb983d80b47/services HTTP/1.1" 200 20 "http://localhost/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:39.0) Gecko/20100101 Firefox/39.0"
confidant_1 | [2015-11-06 16:17:25 +0000] [10] [ERROR] Error handling request
confidant_1 | Traceback (most recent call last):
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/async.py", line 52, in handle
confidant_1 |     self.handle_request(listener_name, req, client, addr)
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/ggevent.py", line 159, in handle_request
confidant_1 |     super(GeventWorker, self).handle_request(*args)
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/async.py", line 105, in handle_request
confidant_1 |     respiter = self.wsgi(environ, resp.start_response)
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1836, in __call__
confidant_1 |     return self.wsgi_app(environ, start_response)
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/guard.py", line 62, in __call__
confidant_1 |     return self.application(environ, _start_response)
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1820, in wsgi_app
confidant_1 |     response = self.make_response(self.handle_exception(e))
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1403, in handle_exception
confidant_1 |     reraise(exc_type, exc_value, tb)
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
confidant_1 |     response = self.full_dispatch_request()
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
confidant_1 |     rv = self.handle_user_exception(e)
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
confidant_1 |     reraise(exc_type, exc_value, tb)
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
confidant_1 |     rv = self.dispatch_request()
confidant_1 |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
confidant_1 |     return self.view_functions[rule.endpoint](**req.view_args)
confidant_1 |   File "/srv/confidant/confidant/authnz.py", line 109, in decorated
confidant_1 |     return f(*args, **kwargs)
confidant_1 |   File "/srv/confidant/confidant/routes/v1.py", line 312, in get_credential
confidant_1 |     _credential_pairs = json.loads(_credential_pairs)
confidant_1 |   File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
confidant_1 |     return _default_decoder.decode(s)
confidant_1 |   File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
confidant_1 |     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
confidant_1 |   File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
confidant_1 |     obj, end = self.scan_once(s, idx)
confidant_1 | ValueError: Invalid control character at: line 1 column 16 (char 15)
confidant_1 | 172.17.42.1 - - [06/Nov/2015:16:17:25 +0000] "GET /v1/credentials/ea5cb10c685c4aa7b6030bb983d80b47 HTTP/1.1" 500 - "-" "-"

We should show a warning header/banner if you've intentionally disabled encryption (for e.g. in development or trying out Confidant via Docker).

An example from Stripe's OAuth flow (though Confidant's should be more alarming and explicit about the lack of encryption):

By default, Confidant assumes you are running behind an ssl terminator like Amazon's ELB. So Gunicorn is not configured to use ssl and direct, encrypted connections are not possible. That said, this really isn't a bug per se and updating the docs to communicate the assumption that Confidant is running behind an ssl terminator might be the best solution.

Thank you,
Bill

Currently it's a bit difficult to run the unit tests because it requires access to IAM and KMS, which requires valid IAM credentials (and costs money). We should mock out KMS and should use moto for IAM.

It's possible that you don't want services created in confidant. This is especially true if you're running multiple confidant instances that are segregated by environment (development/staging/prod). In these cases maybe you'd like to disallow staging and development environment services from being created in production confidant so that users don't get confused. We should add a service blacklist regex so that we can say "^.staging.$" and "^.development.$" are disallowed.

Anyone standing up confidant behind load balancers will likely need to set --forwarded-allow-ips=* or similar in order for wsgi to trust the X-Forwarded-* headers and get the real client's address and scheme.

We need to do the following to cut a new release:

1. Finish up some work on the client
2. Add scoped account support to service history view
3. Update the docs
4. Merge 1.1 into master
5. Make a release notes doc with features added per release.
6. Make an announcement to the list.

A common use of confidant's credentials pairs is to inject them into shell environments. However, credential pair keys aren't limited to strings that are valid for shell environment variable strings. We don't want to limit this because there may be people using confidant in a way that doesn't inject into the shell, and then there's no reason to limit the keys. We do, however, want to want people if they're using keys that may be invalid for their use-case.

So, we should add a warning dialog to the UI that tells people they're using a credential pair key that may limit its usability.

Essentially what is reported here, Although it only effects cryptography==1.0.1.

pyca/cryptography#2973

Downgrading SSL packages to '1.0.1k-3+deb8u5' resolves the issue.

Setting up Confidant should be simplified, perhaps with a configuration management script that creates the correct KMS and IAM resources.

gunicorn is running behind an AWS ELB.

I enabled google+ and the people API

using the allowed redirect uri's:
https://confidant.example.com
https://confidant.example.com/v1/login

400. That’s an error.
Error: redirect_uri_mismatch

The redirect URI in the request, http://confidant.example.com.au/, does not match the ones authorized for the OAuth client. Visit https://console.developers.google.com/apis/credentials/oauthclient/585383940121-etq6mgot8mmd7q4bi.apps.googleusercontent.com?project=58540121 to update the authorized redirect URIs.

I checked and http scheme uri is not anywhere in the credential configuration.

If I add the unused http:// scheme URI's to the allowed redirect URI's

http://confidant.example.com
http://confidant.example.com/v1/login

I get prompted an Oauth privacy screen I click allow and it posts

POST /o/oauth2/approval?hd=example.com.au&as=-72bf7555&pageId=none&xsrfsign=APd5Au-3o4Q-eIcsrShTb HTTP/1.1

And responds with a HTTP/2.0 302 Found to http://confidant.example.com instead of https.

Currently when a service or credential is disabled, it's effectively just hidden from the interface. We return the status via the enabled attribute in get_credential and get_service, which leaves it up to clients on how to handle disabled resources. We should consider some approaches to further improvements to this on the server side:

1. Filter disabled credentials from get_service returns, with a parameter arg to not filter.
2. 404 get_service for get_service calls, with a parameter to allow disabled services.

The history view is pretty rough when looking at diffs for credentials with large keys. At minimum we should rework the diff view, but the entire workflow may need an overhaul.

When generating a new credential, auto-populate the 'value' field with a random string. Could have controls for e.g. key length, allow special characters, etc.,

This is pretty open-ended since I'm not sure what the security implications of this are.

To ease development and testing of Confidant, add config options to mock out KMS so that Confidant can be completely started in docker without any AWS dependencies.

I am attempting to run the development environment described here:
http://lyft.github.io/confidant/advanced/contributing/#quickstart-for-testing-or-development

I clone the confidant repo, copy the dev config to a service.env (sourced it for good measure) then I ran docker-compose up which resulted in running containers but no functioning app at http://localhost:80

confidant_1 | [2015-11-09 15:56:14 +0000] [1] [INFO] Starting gunicorn 19.3.0
confidant_1 | [2015-11-09 15:56:14 +0000] [1] [INFO] Listening at: http://0.0.0.0:80 (1)
confidant_1 | [2015-11-09 15:56:14 +0000] [1] [INFO] Using worker: gevent
confidant_1 | [2015-11-09 15:56:14 +0000] [12] [INFO] Booting worker with pid: 12
confidant_1 | [2015-11-09 15:56:14 +0000] [15] [INFO] Booting worker with pid: 15
confidant_1 | [2015-11-09 15:56:44 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:12)
confidant_1 | [2015-11-09 15:56:44 +0000] [12] [INFO] Worker exiting (pid: 12)
confidant_1 | [2015-11-09 15:56:44 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:15)

This is inside a Ubuntu 14.04 VM

It's possible to create service mappings without the associated IAM role. This is by design so that it's possible to create credentials and map them to a service before its deployed. The issue here is that when the service is deployed, the grants need to exist for the service to get its credentials. It's possible for an end-user to go to the service page and update the grants, but this should be automated if possible.

We can create a scheduled lambda that runs at a user definable interval that will get all grants from the KMS auth key, then compare them against the service mappings. If a service mapping exists, but grants do not, we can check to see if an IAM role exists and create grants if so.

At minimum we should have a client in Go. It may also be a good idea to write a very basic client in bash, for maximum compatibility.

In general we're open to clients in any language, but want to initially target languages that are frequently used for systems programming, since clients can cache returned values into ramdisks for use in configuration management systems, or directly in applications by reading the cache and json loading the data.

The dist directory is not correctly being generated from the Gruntfile, so we can't use the minified directory as the static asset directory.