redhatqe / cloudwash Goto Github PK
View Code? Open in Web Editor NEWA cloud resources cleanup utility tool
License: Apache License 2.0
A cloud resources cleanup utility tool
License: Apache License 2.0
From configuration and brief check of source I don't see any chance how to clean resources across all regions in AWS. Also configuration allows only define one region for ec2 and the most weird part is username and password for ec2... AWS allows you to use api account with id and key so this tool should be able to reuse it.
Subscription QE has asked for the CSV Report of resources in-use.
$subject
Hey, the mkdir ~/cloudwash && cd ~/cloudwash
command in the main installation instructions does not appear to work. When pip installs all packages, it did it into $HOME/.local for me on Fedora 36. So I am going to delete that directory.
I do not use virtual environments that often so my experience with this is small. But it works fine, I just wanted to let you know that README might need an update.
Also what puzzled me was how to setup config if I would like to use the official container. The example YAML is baked in, I am not sure how I would use swach there (perhaps I would need another commit with configuration). That is weird.
Anyways, thanks for the tool. Gonna test it out now.
Hello,
my Azure personal account is protected, just like all my cloud accounts, with 2FA. Cloudwash is unable to use that and it would be useless as I want to automate the process.
Is it possible to configure Azure account via client/secret id/value, like applications are normally supposed to be configured?
Thanks.
I'm working on creating a jenkins pipeline for cloudwash to clean all providers we have but I'm getting that error and I don't know how to solve it. It is working fine for AWS.
Running swach for gce
[Pipeline] sh
10:47:22
10:47:22 <<<<<<< Running the cleanup script in DRY RUN mode >>>>>>>
10:47:22 The GCE providers settings are initialized and validated !
10:47:22 Traceback (most recent call last):
10:47:22 File "/opt/app-root/bin/swach", line 8, in <module>
10:47:22 sys.exit(cleanup_providers())
10:47:22 File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1130, in __call__
10:47:22 return self.main(*args, **kwargs)
10:47:22 File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1055, in main
10:47:22 rv = self.invoke(ctx)
10:47:22 File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1657, in invoke
10:47:22 return _process_result(sub_ctx.command.invoke(sub_ctx))
10:47:22 File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1404, in invoke
10:47:22 return ctx.invoke(self.callback, **ctx.params)
10:47:22 File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 760, in invoke
10:47:22 return __callback(*args, **kwargs)
10:47:22 File "/opt/app-root/lib64/python3.8/site-packages/click/decorators.py", line 26, in new_func
10:47:22 return f(get_current_context(), *args, **kwargs)
10:47:22 File "/opt/app-root/lib64/python3.8/site-packages/cloudwash/cli.py", line 53, in gce
10:47:22 gceCleanup(vms=vms, discs=discs, nics=nics, _all=_all, dry_run=is_dry_run)
10:47:22 File "/opt/app-root/lib64/python3.8/site-packages/cloudwash/providers/gce.py", line 14, in cleanup
10:47:22 with compute_client("gce") as gce_client:
10:47:22 File "/usr/lib64/python3.8/contextlib.py", line 113, in __enter__
10:47:22 return next(self.gen)
10:47:22 File "/opt/app-root/lib64/python3.8/site-packages/cloudwash/client.py", line 29, in compute_client
10:47:22 service_account=json.loads(settings.providers.gce.service_account),
10:47:22 File "/usr/lib64/python3.8/json/__init__.py", line 357, in loads
10:47:22 return _default_decoder.decode(s)
10:47:22 File "/usr/lib64/python3.8/json/decoder.py", line 337, in decode
10:47:22 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
10:47:22 File "/usr/lib64/python3.8/json/decoder.py", line 355, in raw_decode
10:47:22 raise JSONDecodeError("Expecting value", s, err.value) from None
10:47:22 json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
This is my settings file for GCE. Is there any mistake from my side?
+ cat /opt/app-root/src/cloudwash/settings.yaml
DELETE_VM: ''
SLA_MINUTES: 180
PROVIDERS:
GCE:
PROJECT_ID: XXXXX-qe
SERVICE_ACCOUNT: [email protected]
EXCEPT_VM_LIST: []
EXCEPT_VM_STOP_LIST: []
AZURE: null
EC2: null
Hi!
Just to be sure that you are aware of, with the latest version of cloudwash I started to get some future warning for all regions in AWS like this:
/opt/app-root/lib64/python3.8/site-packages/botocore/client.py:625: FutureWarning: The sqs client is currently using a deprecated endpoint: us-west-2.queue.amazonaws.com. In the next minor version this will be moved to sqs.us-west-2.amazonaws.com. See https://github.com/boto/botocore/issues/2705 for more details.
It seems like in the next version of boto will be modified so it should be taken into account. I do not know if it is an automatic change or not, but just raising a possible future issue.
Requester: jabia (for Image Builder project)
Issue: Add VMWare to the list of supported cloud. Image builder team performs some tests on this cloud and having this option would enable us to use cloudwash
Urgency: Together with #72 and #74, they are must have features for Image Builder to start using cloudwash. In the meantime, the team will continue using cloud-cleaner.
Willing to Contribute: Yes, although the priority is not very high so we might struggle to find the time to do it.
Requester: Sharon Brown
Issue: Would like to have an exception list for instances we don't want to delete. but currently, we have an exception list to stop the instance but not completely to escape it.
Urgency: Not mentioned but looks like a promising future.
Willing to Contribute: Not mentioned!
A possible solution to include some IaC framework could be Pulumi, pulumi would offer resources management in a declarative way as Terraform would do.
The key benefit would be use python as language to integrate the IaC framework instead of orchestrate invokations for tf scripts (which should be define using terraform syntax)
Some additional resources:
Small presentation on Pulumi:
Some project on pulumi (golang):
Requester: jabia (for Image Builder project)
Issue: Add tagging system. Add an option to use tags as the filtering system instead of a name pattern. AFAIK, aws, azure, gcp and vmware all have a tagging feature.
Urgency: Together with #72 and #73, they are must have features for Image Builder to start using cloudwash. In the meantime, the team will continue using cloud-cleaner.
Willing to Contribute: Yes, although the priority is not very high so we might struggle to find the time to do it.
One of the multiple advantages is that clous apis let you list only resources with a specific tag, so the cleaning is more efficient.
Can you add support for deleting EC2 Mac Instances in AWS?
When running the cleanup script in DRY mode for Azure resources, an AttributeError occurs due to a missing attribute in the settings configuration. Specifically, the 'AWS' attribute is not found within the 'Settings' object, leading to the termination of the cleanup process.
> python cloudwash/cli.py -d azure --all
<<<<<<< Running the cleanup script in DRY mode >>>>>>>
The AZURE providers settings are initialized and validated !
Resources from the region and resource group: eastus/SatelliteQEResources
Traceback (most recent call last):
File "/Users/okhatavk/satellite/cloudwash/cloudwash/cli.py", line 143, in <module>
cleanup_providers()
File "/Users/okhatavk/satellite/cloudwash/env/lib/python3.11/site-packages/cloudwash/entities/resources/discs.py", line 44, in list
if settings.aws.criteria.disc.unassigned:
^^^^^^^^^^^^
File "/Users/okhatavk/satellite/cloudwash/env/lib/python3.11/site-packages/dynaconf/base.py", line 145, in __getattr__
value = getattr(self._wrapped, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/okhatavk/satellite/cloudwash/env/lib/python3.11/site-packages/dynaconf/base.py", line 328, in __getattribute__
return super().__getattribute__(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Settings' object has no attribute 'AWS'
This issue in Develop
Branch
Hi,
I have been using Cloudwash for couple of months.
And till now I have observed that it needs a detailed logging structure.
because as I was writing the Tekton pipeline for cleanup of Azure, AWS and GCP it always showed me this line at first and then it went on with the actual error.
(app-root) bash-5.2$ swach -d azure --all
<<<<<<< Running the cleanup script in DRY RUN mode >>>>>>>
The AZURE providers settings are initialized and validated !
and even if it wasn't able to fetch the secrets still it was saying "The AZURE providers settings are initialized and validated !" which i think is confusing.
So, now that I am testing my cron job locally with the help of minikube kubectl and everything is setup, resolved all of the error,
but now it just gives me above pasted LOC. and not the expected deletable resources.
So, I expect it to show me the full log.
Thanks !
Create a Dockerfile in root to deploy the cloudwash container! And create an image on Quay for new version create in cloudwash.
Currently there is no way to identify the current version for the tool. It would be great to add a flag to show the current version:
swach -v
With this option we can be sure about the version we are using.
As we are looking for a tool to manage the clean-up in the QE department. It is important to add IBM cloud to the list of cloud support as this is the main provider for test platforms in QE.
Hope to deal with this in CY22Q4 or at the start of CY23.
After the adoption of the tool in QE for AWS, Azure, and GCP and adjust the tool to QE-wide needs.
Request from Subscription QE to fetch th CSV of resources thats in use !!
$subject
Hello,
I've run the tool to clean up EC2 instances. It run successfully, however the tool mentioned "azure" instead of "aws" in the log message.
<<<<<<< Running the cleanup script in ACTION mode >>>>>>>
The EC2 providers settings are initialized and validated !
Stopped [] and removed ['testcloudwash-xkmdw-master-1', 'testcloudwash-xkmdw-master-2', 'testcloudwash-xkmdw-master-0', 'testcloudwash-xkmdw-worker-us-east-1a-jf4l2', 'testcloudwash-xkmdw-worker-us-east-1a-ff7gn'] VMs from Azure Cloud.
Removed following and all unused nics from Azure Cloud.
['eni-078c16e35914e3189', 'eni-004e575b611e2e356', 'eni-06c75dd20d94f45fa']
Removed following and all unused discs from Azure Cloud.
[]
Removed following and all unused pips from Azure Cloud.
[]
Currently, there is
FROM quay.io/fedora/python-311:latest
I suggest to change it to
registry.access.redhat.com/ubi9/python-311:latest
According to documentation currently the settings.yaml file should be filled and placed at ~/cloudwash/settings.yaml
. I was expecting that would be the path where cloudwash looks for the file. Instead it is looking for the file at current path where the tool is been executed.
Using the image on a different context like Tekton
the execution will end on error:
<<<<<<< Running the cleanup script in DRY RUN mode >>>>>>>
Traceback (most recent call last):
File "/opt/app-root/bin/swach", line 8, in <module>
sys.exit(cleanup_providers())
File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/app-root/lib64/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/opt/app-root/lib64/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/opt/app-root/lib64/python3.8/site-packages/cloudwash/cli.py", line 87, in ec2
validate_provider([ctx.command.name](http://ctx.command.name/))
File "/opt/app-root/lib64/python3.8/site-packages/cloudwash/config.py", line 13, in validate_provider
f"{provider}.{setting_key}" for setting_key in settings.to_dict()["PROVIDERS"].get(provider)
This is because the working dir (from where swach will be executed) for the tekton task is /tekton/home
and the file is at /tekton/home/cloudwash/settings.yaml
So it would be great if a new option for passing the settings file path would be added:
swach --settings-file /custom_path/settings.yaml
Repo is lacking contribution guide for this project.
Each of these cloud provider which cloudwash already supports, have specific format and standard location of the config files.
For example:
I belive that the cloudwash could take it from those locations as well ...
Requester: Prabodh Phuse
Issue: provision of deleting OCP cluster on AWS, which is a sum of various services and resources ...ec2, vpc, S3...etc
Urgency: Not mentioned but looks a promising future. But since cluster is a virtual resource of a cloud we need to rethink if cloudwash should get into virtual resources cleanup.
Willing to Contribute: Not mentioned!
$subject
Implement pre-commit with the hooks for clean and neat code.
$subject
Maybe this comes from an intent to have the default behavior pick all images for removal if the `delete_pattern` is empty?
Either way, it needs to be adjusted so delete_pattern
is applied correctly when present - I would recommend actually using re
here and actually accept a pattern string. Using just startswith
will immediately limit the use and dictate that any teams wanting to use this tool have to prefix image names.
re.compile
can be used to create the regex pattern object (and to parse the pattern from config), and then you can use Pattern.search
function in the list comprehension to indicate whether the given pattern is anywhere in the string. match
can be used too, but will be more explicit and will require the caller to specify .*
in their pattern to match anywhere in the string. I think either function is fine here, but the docblock/readme/config comment should explain which is in use.
Originally posted by @mshriver in #101 (comment)
cloudwash removes the resources(nics and pips) just freed by deleted VM(that makes them eligible for unused resource) but this includes the resources which were not asked to delete after VM delete.
Handle that in cloudwash to skip the resources just freed after VM deletion this works against the Cloud VM deletion behavior and should be configurable !
Requester: jabia (for Image Builder project)
Issue: Add an option to use negative logic when cleaning resources. This means removing every resource that does NOT comply with the filter criteria
Urgency: Together with #73 and #74, they are must have features for Image Builder to start using cloudwash. In the meantime, the team will continue using cloud-cleaner.
Willing to Contribute: Yes, although the priority is not very high so we might struggle to find the time to do it.
Having the option to negate the logic of the cleaning method can be very useful to reduce cloud costs.
Can you add support for deleting Red Hat OpenShift Service on AWS?
Requester: Nikhil Kathole
Issue: Support for cleaning up more resources on AWS like policies, roles, s3 buckets for AWS from multiple accounts.
Urgency: Whenever we can, the existing script works for him.
Willing to Contribute: Sure, let us know once repo setup.
Secrets Management: git-crypt (as a part of CI config repo)
Independent Tool or APIs Would like to use the tool directly since the separate Jenkins job is running cloud cleanup. We would also love to see it as a service so that we can move away from maintaining Jenkins jobs or cron.
We create new resource group, which name is based version and platform, i.e. eap-8.0.0.Beta-redhat-20220713-w2k19-openjdk-17-group, ... which may be different per test run. We follow suggestion that resources in the resource group share same lifecycle hence once the test run is finished resource group may be deleted.
Sometimes test execution get stuck / fails / ... and test resources may remain up costing us money. It is inconvenient to determine all resource groups that should be cleaned for separate periodical cloudwash run. I would like to have a support for option to delete all resource groups (meeting SLA minutes criteria) except for a list of predefined resource groups (containing monitoring resources, ... )
Create a Cloudwash service to store information about users account resources in centralised DB
Cloudwash should run in two modes .
Agent to collect information from the cloud provider and push data to centralised DB.
- Build a centralised repository to data matrix published by local cloudwash agents
- Data matrix includes the information about all the resources from individual user cloud provided accounts.
As a sandbox to locally fetch and display data in terminal .
Separate config clients definition from current singular compute_client
that takes arguments for all providers.
Since the parameters to compute client differs its better to separate them.
Requester: Sheron Brown
Issue: would like to see it run on multiple accounts (as on all QE accounts in the clouds). And run every 1 hour, with distinguished the SLA of deleting after 2/3 hours/days. and even be triggered after the failure of deployment of OCP for example!
Urgency: As soon as possible, we have a big urgent to save cost in the cloud this year.
Willing to Contribute: hopeful yes
Secrets Management: Contact Prabodh Phusey !
Independent Tool or APIs I liked the idea of that tool to stand alone - so maybe in the end it can become a service.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.