caravanacloud / cloud-janitor Goto Github PK

CJ will download and install some commands, however, this is currently hardcoded in java.
Build a task that can lookup in configuration the proper package to download.
It would also be interesting to allow package managers install.

Use ec2 compute optimizer and/or cloudwatch to detect when an instance is overprovisioned and generating waste.

Use "latest" for generated files and folder names instead of / besides timestamp, so that actions can fetch easily

Extract generated secrets from the command line output and save them for future use

Create the initial documentation site. Suggestted tooling: mkdocs material
https://squidfunk.github.io/mkdocs-material/

Remove secrets and undesired content from standard output and logged messages.

What other tasks would be interesting to add to the tree?

Add the correct jreleaser configuration to publish packages to homebrew, so that user can install cloud-janitor with "brew install"

Check all resources that are flagged as wasteful and notify owners

Output redaction is curently hardcoded in java and quite static.
Improve the output redaction so that it can be configurable, allowing the user to add words or expressions to filter.
It would be interesting to consider the posibility of auto-detecting secrets to redact in the future.

Test auto-detection of regions in aws-nuke

Currently AWS Authentication is performed entirely by the AWS SDK, using the default credentials chain.
Supporting multiple AWS Profiles would let users clean multiple AWS accounts with a single run.

Create a configuration element to let tasks be configured according to their "scope", for example:
multi-provider, multi-account, multi-region (ex: inventory all my cloud resources)
single-provider, multi-account, multi-region (ex: create attribution trails and tables)
single-provider, single-account, multi-region (ex: cleanup a single AWS region)
single-provider, single-account, single-region (ex: transcribe a video)

Review linter warnings

Lookup resources that have both attribution and waste tags and notify the appropriate user.

Load parameters (cluster size, aws region, ...) and secrets (ssh key, db password, ...) from a cloud service (AWS SSM Parameter Store)

Create a configuration element to let tasks be chained in sequence without running the application again.
For example, the task "aws-transcribe-translate" could call both "aws-transcribe" and "aws-translate"

Build a task that detects if ec2 instances are overprovisioned by reading metric data from cloudwatch and/or the compute optimizer service. Tag underutilized resources.

All tests are manual for now.
Adding automated tests would help prevent bugs and improve user confidence.

Delete CloudFormation resources

Create a few initial test cases

Instead of programatically generating templates, create a TaskTemplate annotation to define parameters and defauts.

Add support for S3 on aws-cleanup task

We currently use "sleep" delays to wait for resource termination / release before cascading deletes.
Using waiter methods could reduce the wait times to the minimal necessary.
See: https://docs.aws.amazon.com/cli/latest/reference/ec2/wait/instance-stopped.html

https://github.com/jobrunr/jobrunr

Ommit stack traces unless in development

Add unit tests to improve coverage.
Initial goal is 5%

add jreleaser configuration to release to homebrew