The credmon and Flask apps should log the reason (if given) for file read/write errors rather than just saying that writing/reading/refreshing tokens failed.

We should provide some WSGI configuration or a Docker implementation for the OAuth Credmon Webserver.

For example, say a user wants multiple Box tokens, and so requests tokens named box_1 and box_2. When the OAuthCredmonWebserver sends the user out to Box's authorize endpoint, they will get sent back to something like "/return/box", not "/return/box_1". Thus, methods that lookup the provider info in the key file or session object will fail because they will be looking for a provider named "box", per the return decorator's argument /return/<provider>. We should tag the session variable with the outgoing custom token name so that it can be looked up upon return from the OAuth authorize endpoint.

In the deploy script, an admin should be able to specify which web server they want to use, where the credential directory should be, and what subdirectory (if any) the webserver should host the Flask app under. Thinking some of this should be done interactively (with an option to skip and use defaults) rather than having a ton of command line options.

Also, the Apache part as is probably only works in RHEL-dervied distros, where we assume Apache configs are in /etc/httpd. How can we make this more universal?

Client secrets, tokens, session state... don't do it!!!

Key files should be cleaned up after users have logged in to all of their OAuth providers rather than waiting for the condor_credd to do it.

Per experience with getting this working in CHTC, the install script is seeming less and less ideal. For distro packages, let's put commented-out configs in /etc/condor/config.d (or whatever makes sense for the distro) and the WSGI app in /var/www/wsgi-scripts/scitokens-credmon. For both wheel and distro packages, let's provide an examples directory for all configs, where wheel installers can grab all configs from this directory and distro installers can grab web server configs from this directory.

Currently only the OAuth Credmon implementation is included, we should also add the Self-Signed Certs Credmon (code already exists).

The credmon continues to lock up. Derek noticed that the condor cgroup may be running up against file open limits. Maybe this is causing one of the credmon threads to silently die?

Currently, the Flask app logs to the same files as the credmon by default. The logging should really end up in the httpd's logs. Flask provides a handler for this.

From the code:

We should have appropriate command line parsing for the credmon

Code should be ready to go for when https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=7462 is resolved. Might be good to actually poll the Schedd for what version is running (don't check the Python bindings' version, which could be different) if we need to condition on the version of HTCondor that is running. But really we should lobby for some boolean to be added to the condor_config that we can check.

The credmons both operate out of the same directory on files with the same extensions (.use and .top). This means, if the local issuer is configured, both credmons are poking through all of the credentials. The credmons should differentiate which tokens they work on. It seems like the best option right now is to use the presence of .meta files to discern which tokens are OAuth and which are local.

The credmon is meant to be forked by condor, not run by regular users, therefore it should be in $(SBIN)

The webserver should never run as root, but we should secure user's tokens by owning them as root (or HTCondor's real uid). The credmon should be forked by the credd, so it will be running as the correct uid. Current idea: The webserver writes tokens to a temp directory and then pings the credmon (how?) to move and own the tokens into the SEC_CREDENTIAL_DIRECTORY.

We should really include proper packaging of the credmon itself in order to make this straightforward to install "alongside" a schedd.

Thoughts that come to mind:

• RPM packages (see #14)
• Drop condor config files necessary to enable the credmon. Only enable if admin specifies it in the config file
• WSGI configuration necessary for integration in apache.
• Generate new signing keys on first start of the credmon.
• Export .well-known directory for OAuth2 auto-discovery.
• Change Flask app to not function unless condor config changed by sysadmin.
• Split configuration files to "must be edited" and "infrequently edited".

The end goal is that yum install scitokens-credmon - with very minimal config changes - should result in a working local issuer.