Giter Club home page Giter Club logo

cloud-nuke's Introduction

Maintained by Gruntwork.io

cloud-nuke

This repo contains a CLI tool to delete all resources . cloud-nuke was created for situations when you might have an account you use for testing and need to clean up leftover resources so you're not charged for them. Also great for cleaning out accounts with redundant resources. Also great for removing unnecessary defaults like default VPCs and permissive ingress/egress rules in default security groups.

In addition, cloud-nuke offers non-destructive inspecting functionality that can either be called via the command-line interface, or consumed as library methods, for scripting purposes.

The currently supported functionality includes:

AWS

Cloud-nuke suppports ๐Ÿ”Ž inspecting and ๐Ÿ”ฅ๐Ÿ’€ deleting the following AWS resources:

Resource Family Resource type
App Runner Service
EC2 Auto scaling groups
EC2 Elastic Load Balancers (v1 and v2)
EC2 EBS Volumes
EC2 Unprotected EC2 instances
EC2 AMIS
EC2 Snapshots
EC2 Elastic IPs
EC2 Launch Configurations
EC2 IPAM (Amazon VPC IP Address Manager)
EC2 IPAM Pool
EC2 IPAM Scope
EC2 IPAM Custom Allocation
EC2 IPAM BYOASN
EC2 IPAM Resource Discovery
EC2 Internet Gateway
EC2 Network ACL
EC2 Egress only internet gateway
EC2 Endpoint
EC2 Security Group
EC2 Network Interface
Certificate Manager ACM Private CA
Direct Connect Transit Gateways
Elasticache Clusters
Elasticache Parameter Groups
Elasticache Subnet Groups
Elastic Beanstalk Applications
ECS Services
ECS Clusters
EKS Clusters
DynamoDB Tables
Lambda Functions
SQS Queues
S3 Buckets
S3 Access Points
S3 Object Lambda Access Points
S3 Multi Region Access Points
VPC Default VPCs
VPC Default rules in the un-deletable default security group
VPC NAT Gateways
IAM Users
IAM Roles (and any associated EC2 instance profiles)
IAM Service-linked-roles
IAM Groups
IAM Policies
IAM Customer-managed policies
IAM Access analyzers
IAM OpenID Connect providers
Secrets Manager Secrets
CloudWatch Dashboard
CloudWatch Log groups
CloudWatch Alarms
OpenSearch Domains
KMS Custgomer managed keys (and associated key aliases)
GuardDuty Detectors
Macie Member accounts
SageMaker Notebook instances
Kinesis Streams
Kinesis Firehose
API Gateway Gateways (v1 and v2)
EFS File systems
SNS Topics
CloudTrail Trails
ECR Repositories
Config Service recorders
Config Service rules
RDS RDS databases
RDS Neptune
RDS Document DB instances
RDS RDS parameter group
RDS RDS Proxy
Security Hub Hubs
Security Hub Members
Security Hub Administrators
SES SES configuration set
SES SES email template
SES SES Identity
SES SES receipt rule set
SES SES receipt filter
AWS Certificate Manager Certificates
CodeDeploy Applications
Route53 Hosted Zones
Route53 CIDR collections
Route53 Traffic Policies
NetworkFirewall Network Firewall
NetworkFirewall Network Firewall Policy
NetworkFirewall Network Firewall Rule Group
NetworkFirewall Network Firewall TLS inspection configuration
NetworkFirewall Network Firewall Resource Policy
VPCLattice VPC Lattice Service
VPCLattice VPC Lattice Service Network
VPCLattice VPC Lattice Target Group

WARNING: The RDS APIs also interact with neptune and document db resources. Running cloud-nuke aws --resource-type rds without a config file will remove any neptune and document db resources in the account.

NOTE: AWS Backup Resource: Resources (such as AMIs) created by AWS Backup, while owned by your AWS account, are managed specifically by AWS Backup and cannot be deleted through standard APIs calls for that resource. These resources are tagged by AWS Backup and are filtered out so that cloud-nuke does not fail when trying to delete resources it cannot delete.

BEWARE!

When executed as cloud-nuke aws, this tool is HIGHLY DESTRUCTIVE and deletes all resources! This mode should never be used in a production environment!

When executed as cloud-nuke defaults-aws, this tool deletes all DEFAULT VPCs and the default ingress/egress rule for all default security groups. This should be used in production environments WITH CAUTION.

Telemetry

As of version v0.29.0 cloud-nuke sends telemetry back to Gruntwork to help us better prioritize bug fixes and feature improvements. The following metrics are included:

  • Command and Arguments
  • Version Number
  • Timestamps
  • Resource Types
  • Resource Counts
  • A randomly generated Run ID
  • AWS Account ID

We never collect

  • IP Addresses
  • Resource Names

Telemetry can be disabled entirely by setting the DISABLE_TELEMETRY environment variable on the command line.

As an open source tool, you can see the exact statistics being collected by searching the code for telemetry.TrackEvent(...)

Install

Download from releases page

  1. Download the latest binary for your OS on the releases page.
  2. Move the binary to a folder on your PATH. E.g.: mv cloud-nuke_darwin_amd64 /usr/local/bin/cloud-nuke.
  3. Add execute permissions to the binary. E.g.: chmod u+x /usr/local/bin/cloud-nuke.
  4. Test it installed correctly: cloud-nuke --help.

Install via package manager

Note that package managers are third party. The third party cloud-nuke packages may not be updated with the latest version, but are often close. Please check your version against the latest available on the releases page. If you want the latest version, the recommended installation option is to download from the releases page.

  • macOS: You can install cloud-nuke using Homebrew: brew install cloud-nuke.

  • Linux: Most Linux users can use Homebrew: brew install cloud-nuke.

  • Windows: You can install cloud-nuke using winget: winget install cloud-nuke

Usage

Simply running cloud-nuke aws will start the process of cleaning up your cloud account. You'll be shown a list of resources that'll be deleted as well as a prompt to confirm before any deletion actually takes place.

In AWS, to delete only the default resources, run cloud-nuke defaults-aws. This will remove the default VPCs in each region, and will also revoke the ingress and egress rules associated with the default security group in each VPC. Note that the default security group itself is unable to be deleted.

Nuke or inspect resources using AWS Profile

When using cloud-nuke aws, or cloud-nuke inspect-aws, you can pass in the AWS_PROFILE env variable to target resources in certain regions for a specific AWS account. For example the following command will nuke resources only in ap-south-1 and ap-south-2 regions in the gruntwork-dev AWS account:

AWS_PROFILE=gruntwork-dev cloud-nuke aws --region ap-south-1 --region ap-south-2

Similarly, the following command will inspect resources only in us-east-1

AWS_PROFILE=gruntwork-dev cloud-nuke inspect-aws --region us-east-1

Nuke or inspect resources in certain regions

When using cloud-nuke aws, or cloud-nuke inspect-aws, you can use the --region flag to target resources in certain regions. For example the following command will nuke resources only in ap-south-1 and ap-south-2 regions:

cloud-nuke aws --region ap-south-1 --region ap-south-2

Similarly, the following command will inspect resources only in us-east-1

cloud-nuke inspect-aws --region us-east-1

Including regions is available within:

  • cloud-nuke aws
  • cloud-nuke defaults-aws
  • cloud-nuke inspect-aws

Exclude resources in certain regions

When using cloud-nuke aws or cloud-nuke inspect-aws, you can use the --exclude-region flag to exclude resources in certain regions from being deleted or inspected. For example the following command does not nuke resources in ap-south-1 and ap-south-2 regions:

cloud-nuke aws --exclude-region ap-south-1 --exclude-region ap-south-2

Similarly, the following command will not inspect resources in the us-west-1 region:

cloud-nuke inspect-aws --exclude-region us-west-1

--region and --exclude-region flags cannot be specified together i.e. they are mutually exclusive.

Excluding regions is available within:

  • cloud-nuke aws
  • cloud-nuke defaults-aws
  • cloud-nuke inspect-aws

Excluding Resources by Age

You can use the --older-than flag to only nuke resources that were created before a certain period, the possible values are all valid values for ParseDuration For example the following command nukes resources that are at least one day old:

cloud-nuke aws --older-than 24h

Excluding resources by age is available within:

  • cloud-nuke aws
  • cloud-nuke inspect-aws

List supported resource types

You can use the --list-resource-types flag to list resource types whose termination is currently supported:

cloud-nuke aws --list-resource-types

Listing supported resource types is available within:

  • cloud-nuke aws
  • cloud-nuke inspect-aws

Terminate or inspect specific resource types

If you want to target specific resource types (e.g ec2, ami, etc.) instead of all the supported resources you can do so by specifying them through the --resource-type flag:

cloud-nuke aws --resource-type ec2 --resource-type ami

will search and target only ec2 and ami resources. The specified resource type should be a valid resource type i.e. it should be present in the --list-resource-types output. Using --resource-type also speeds up search because we are searching only for specific resource types.

Similarly, the following command will inspect only ec2 instances:

cloud-nuke inspect-aws --resource-type ec2

Specifying target resource types is available within:

  • cloud-nuke aws
  • cloud-nuke inspect-aws

Exclude terminating specific resource types

Just like you can select which resources to terminate using --resource-type, you can select which resources to skip using --exclude-resource-type flag:

cloud-nuke aws --exclude-resource-type s3 --exclude-resource-type ec2

This will terminate all resource types other than S3 and EC2.

--resource-type and --exclude-resource-type flags cannot be specified together i.e. they are mutually exclusive.

Specifying resource types to exclude is available within:

  • cloud-nuke aws
  • cloud-nuke inspect-aws

Dry run mode

If you want to check what resources are going to be targeted without actually terminating them, you can use the --dry-run flag

cloud-nuke aws --resource-type ec2 --dry-run

Dry run mode is only available within:

  • cloud-nuke aws

With Timeout

If you want to set up a timeout option for resources, limiting their execution to a specified duration for nuking, use the --timeout flag:

cloud-nuke aws --resource-type s3 --timeout 10m

This will attempt to nuke the specified resources within a 10-minute timeframe.

Using cloud-nuke as a library

You can import cloud-nuke into other projects and use it as a library for programmatically inspecting and counting resources.

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/aws/aws-sdk-go/aws"
	nuke_aws "github.com/gruntwork-io/cloud-nuke/aws"
	nuke_config "github.com/gruntwork-io/cloud-nuke/config"
	"github.com/gruntwork-io/cloud-nuke/externalcreds"
)

func main() {
	// You can scan multiple regions at once, or just pass a single region for speed
	targetRegions := []string{"us-east-1", "us-west-1", "us-west-2"}
	excludeRegions := []string{}
	// You can simultaneously target multiple resource types as well
	resourceTypes := []string{"ec2", "vpc"}
	excludeResourceTypes := []string{}
	// excludeAfter is parsed identically to the --older-than flag
	excludeAfter := time.Now()
	// an optional start time- can pass null if the filter is not required
	includeAfter := time.Now().AddDate(-1, 0, 0)

	// an optional execution timeout duration
	timeout := time.Duration(10 * time.Second)

	// Any custom settings you want
	myCustomConfig := &aws.Config{}
	myCustomConfig.WithMaxRetries(3)
	myCustomConfig.WithLogLevel(aws.LogDebugWithRequestErrors)
	// Optionally, set custom credentials
	// myCustomConfig.WithCredentials()

	// Be sure to set your config prior to calling any library methods such as NewQuery
	externalcreds.Set(myCustomConfig)
	// this config can be configured to add include/exclude rule to filter the resources- for all resources pass an empty struct
	nukeConfig := nuke_config.Config{}

	// NewQuery is a convenience method for configuring parameters you want to pass to your resource search
	query, err := nuke_aws.NewQuery(
		targetRegions,
		excludeRegions,
		resourceTypes,
		excludeResourceTypes,
		&excludeAfter,
		&includeAfter,
		false,
		&timeout,
	)
	if err != nil {
		fmt.Println(err)
	}

	// GetAllResources still returns *AwsAccountResources, but this struct has been extended with several
	// convenience methods for quickly determining if resources exist in a given region
	accountResources, err := nuke_aws.GetAllResources(context.Background(), query, nukeConfig)
	if err != nil {
		fmt.Println(err)
	}
	// You can call GetRegion to examine a single region's resources
	usWest1Resources := accountResources.GetRegion("us-west-1")
	// Then interrogate them with the new methods:
	// Count the number of any resource type within the region
	countOfEc2InUsWest1 := usWest1Resources.CountOfResourceType("ec2")
	fmt.Printf("countOfEc2InUsWest1: %d\n", countOfEc2InUsWest1)
	// countOfEc2InUsWest1: 2
	fmt.Printf("usWest1Resources.ResourceTypePresent(\"ec2\"):%b\n", usWest1Resources.ResourceTypePresent("ec2"))
	// usWest1Resources.ResourceTypePresent("ec2"): true
	// Get all the resource identifiers for a given resource type
	// In this example, we're only looking for ec2 instances
	resourceIds := usWest1Resources.IdentifiersForResourceType("ec2")
	fmt.Printf("resourceIds: %s", resourceIds)
	// resourceIds:  [i-0c5d16c3ef28dda24 i-09d9739e1f4d27814]
}

Config file

You can also specify which resources to terminate with more granularity via using config files. The config file is a YAML file that specifies which resources to terminate. The top level keys of the config file are the resource types, and the values are the rules for which resources to terminate.

Filtering Features

For each resource type, you can specify either include or exclude rules. Each rule can be one of the following filters mentioned below. Here is an example:

s3:
  include:
    ...
  exclude:
    ...

Names Regex Filter

Now given the following config, the s3 buckets that will be nuked are further filtered to only include ones that match any of the provided regular expressions. So a bucket named alb-app-access-logs would be deleted, but a bucket named my-s3-bucket would not.

s3:
  include:
    names_regex:
      - ^alb-.*-access-logs$
      - .*-prod-alb-.*

Similarly, you can adjust the config to delete only IAM users of a particular name by using the IAMUsers key. For example, in the following config, only IAM users that have the prefix my-test-user- in their username will be deleted.

IAMUsers:
  include:
    names_regex:
      - ^my-test-user-.*

Now consider the following contrived example:

s3:
  include:
    names_regex:
      - ^alb-.*-access-logs$
      - .*-prod-alb-.*
  exclude:
    names_regex:
      - public
      - prod

The intention is to delete all the s3 buckets that match the include rules but not the exclude rules. Filtering is commutative, meaning that you should get the same result whether you apply the include filters before or after the exclude filters.

The result of these filters applied in either order will be a set of s3 buckets that match ^alb-.*-access-logs$ as long as they do not also contain public or prod. The rule to include s3 buckets matching .*-prod-alb-.* is negated by the rule to exclude those matching prod.

Time Filter

You can also filter resources by time. The following config will delete all s3 buckets that were created after 2020-01-01T00:00:00Z.

s3:
  include:
    time_after: '2020-01-01T00:00:00Z'

Similarly, you can delete all s3 buckets that were created before 2020-01-01T00:00:00Z by using the time_before

s3:
  include:
    time_before: '2020-01-01T00:00:00Z'

Tag Filter

You can also exclude resources by tags. The following config will exclude all s3 buckets that have a tag with key foo and value true (case-insensitive).

s3:
  exclude:
    tag: 'foo'

Timeout

You have the flexibility to set individual timeout options for specific resources. The execution will pause until the designated timeout is reached for each resource.

s3:
  timeout: 10m

  ........

s3:
  timeout: 5s

By default, it will use the exclusion default tag: cloud-nuke-excluded to exclude resources. Note: it doesn't support including resources by tags.

What's supported?

To find out what we options are supported in the config file today, consult this table. Resource types at the top level of the file that are supported are listed here.

resource type config key names_regex time tags timeout
acm ACM โœ… (Domain Name) โœ… (Created Time) โŒ โœ…
acmpca ACMPCA โŒ โœ… (LastStateChange or Created Time) โŒ โœ…
ami AMI โœ… (Image Name) โœ… (Creation Time) โŒ โœ…
apigateway APIGateway โœ… (API Name) โœ… (Created Time) โŒ โœ…
apigatewayv2 APIGatewayV2 โœ… (API Name) โœ… (Created Time) โŒ โœ…
accessanalyzer AccessAnalyzer โœ… (Analyzer Name) โœ… (Created Time) โŒ โœ…
asg AutoScalingGroup โœ… (ASG Name) โœ… (Created Time) โœ… โœ…
app-runner-service AppRunnerService โœ… (App Runner Service Name) โœ… (Created Time) โŒ โœ…
backup-vault BackupVault โœ… (Backup Vault Name) โœ… (Created Time) โŒ โœ…
cloudwatch-alarm CloudWatchAlarm โœ… (Alarm Name) โœ… (AlarmConfigurationUpdated Time) โŒ โœ…
cloudwatch-dashboard CloudWatchDashboard โœ… (Dashboard Name) โœ… (LastModified Time) โŒ โœ…
cloudwatch-loggroup CloudWatchLogGroup โœ… (Log Group Name) โœ… (Creation Time) โŒ โœ…
cloudtrail CloudtrailTrail โœ… (Trail Name) โŒ โŒ โœ…
codedeploy-application CodeDeployApplications โœ… (Application Name) โœ… (Creation Time) โŒ โœ…
config-recorders ConfigServiceRecorder โœ… (Recorder Name) โŒ โŒ โœ…
config-rules ConfigServiceRule โœ… (Rule Name) โŒ โŒ โœ…
dynamodb DynamoDB โœ… (Table Name) โœ… (Creation Time) โŒ โœ…
ebs EBSVolume โœ… (Volume Name) โœ… (Creation Time) โœ… โœ…
elastic-beanstalk ElasticBeanstalk โœ… (Application Name) โœ… (Creation Time) โŒ โœ…
ec2 EC2 โœ… (Instance Name) โœ… (Launch Time) โœ… โœ…
ec2-dedicated-hosts EC2DedicatedHosts โœ… (EC2 Name Tag) โœ… (Allocation Time) โŒ โœ…
ec2-dhcp-option EC2DhcpOption โŒ โŒ โŒ โœ…
ec2-keypairs EC2KeyPairs โœ… (Key Pair Name) โœ… (Creation Time) โœ… โœ…
ec2-ipam EC2IPAM โœ… (IPAM name) โœ… (Creation Time) โœ… โœ…
ec2-ipam-pool EC2IPAMPool โœ… (IPAM Pool name) โœ… (Creation Time) โœ… โœ…
ec2-ipam-resource-discovery EC2IPAMResourceDiscovery โœ… (IPAM Discovery Name) โœ… (Creation Time) โœ… โœ…
ec2-ipam-scope EC2IPAMScope โœ… (IPAM Scope Name) โœ… (Creation Time) โœ… โœ…
ec2-subnet EC2Subnet โœ… (Subnet Name) โœ… (Creation Time) โœ… โŒ
ec2-endpoint EC2Endpoint โœ… (Endpoint Name) โœ… (Creation Time) โœ… โœ…
ecr ECRRepository โœ… (Repository Name) โœ… (Creation Time) โŒ โœ…
ecscluster ECSCluster โœ… (Cluster Name) โŒ โŒ โœ…
ecsserv ECSService โœ… (Service Name) โœ… (Creation Time) โŒ โœ…
ekscluster EKSCluster โœ… (Cluster Name) โœ… (Creation Time) โœ… โœ…
elb ELBv1 โœ… (Load Balancer Name) โœ… (Created Time) โŒ โœ…
elbv2 ELBv2 โœ… (Load Balancer Name) โœ… (Created Time) โŒ โœ…
efs ElasticFileSystem โœ… (File System Name) โœ… (Creation Time) โŒ โœ…
eip ElasticIP โœ… (Elastic IP Allocation Name) โœ… (First Seen Tag Time) โœ… โœ…
elasticache Elasticache โœ… (Cluster ID & Replication Group ID) โœ… (Creation Time) โŒ โœ…
elasticacheparametergroups ElasticacheParameterGroups โœ… (Parameter Group Name) โŒ โŒ โœ…
elasticachesubnetgroups ElasticacheSubnetGroups โœ… (Subnet Group Name) โŒ โŒ โœ…
guardduty GuardDuty โŒ โœ… (Created Time) โŒ โœ…
iam-group IAMGroups โœ… (Group Name) โœ… (Creation Time) โŒ โœ…
iam-policy IAMPolicies โœ… (Policy Name) โœ… (Creation Time) โŒ โœ…
iam-role IAMRoles โœ… (Role Name) โœ… (Creation Time) โŒ โœ…
iam-service-linked-role IAMServiceLinkedRoles โœ… (Service Linked Role Name) โœ… (Creation Time) โŒ โœ…
iam IAMUsers โœ… (User Name) โœ… (Creation Time) โœ… โœ…
internet-gateway InternetGateway โœ… (Gateway Name) โœ… (Creation Time) โœ… โœ…
egress-only-internet-gateway EgressOnlyInternetGateway โœ… (Gateway name) โœ… (Creation Time) โœ… โœ…
kmscustomerkeys KMSCustomerKeys โœ… (Key Name) โœ… (Creation Time) โŒ โŒ
kinesis-stream KinesisStream โœ… (Stream Name) โŒ โŒ โœ…
kinesis-firehose KinesisFirehose โœ… (Delivery Stream Name) โŒ โŒ โœ…
lambda LambdaFunction โœ… (Function Name) โœ… (Last Modified Time) โŒ โœ…
lc LaunchConfiguration โœ… (Launch Configuration Name) โœ… (Created Time) โŒ โœ…
lt LaunchTemplate โœ… (Launch Template Name) โœ… (Created Time) โŒ โœ…
macie-member MacieMember โŒ โœ… (Creation Time) โŒ โœ…
msk-cluster MSKCluster โœ… (Cluster Name) โœ… (Creation Time) โŒ โœ…
nat-gateway NatGateway โœ… (EC2 Name Tag) โœ… (Creation Time) โœ… โœ…
network-acl NetworkACL โœ… (ACL Name Tag) โœ… (Creation Time) โœ… โœ…
network-interface NetworkInterface โœ… (Interface Name Tag) โœ… (Creation Time) โœ… โœ…
oidcprovider OIDCProvider โœ… (Provider URL) โœ… (Creation Time) โŒ โœ…
opensearchdomain OpenSearchDomain โœ… (Domain Name) โœ… (First Seen Tag Time) โŒ โœ…
redshift Redshift โœ… (Cluster Identifier) โœ… (Creation Time) โŒ โœ…
rds-cluster DBClusters โœ… (DB Cluster Identifier ) โœ… (Creation Time) โœ… โœ…
rds DBInstances โœ… (DB Name) โœ… (Creation Time) โœ… โœ…
rds-parameter-group RdsParameterGroup โœ… (Group Name) โŒ โŒ โœ…
rds-subnet-group DBSubnetGroups โœ… (DB Subnet Group Name) โŒ โŒ โœ…
rds-proxy RDSProxy โœ… (proxy Name) โœ… (Creation Time) โŒ โœ…
s3 s3 โœ… (Bucket Name) โœ… (Creation Time) โœ… โœ…
s3-ap s3AccessPoint โœ… (Access point Name) โŒ โŒ โœ…
s3-olap S3ObjectLambdaAccessPoint โœ… (Object Lambda Access point Name) โŒ โŒ โœ…
s3-mrap S3MultiRegionAccessPoint โœ… (Multi region Access point Name) โœ… (Creation Time) โŒ โœ…
security-group SecurityGroup โœ… (Security group name) โœ… (Creation Time) โœ… โŒ
ses-configuration-set SesConfigurationset โœ… (Configuration set name) โŒ โŒ โœ…
ses-email-template SesEmailTemplates โœ… (Template Name) โœ… (Creation Time) โŒ โœ…
ses-identity SesIdentity โœ… (Identity -Mail/Domain) โŒ โŒ โœ…
ses-receipt-rule-set SesReceiptRuleSet โœ… (Receipt Rule Set Name) โœ… (Creation Time) โŒ โœ…
ses-receipt-filter SesReceiptFilter โœ… (Receipt Filter Name) โŒ โŒ โœ…
snstopic SNS โœ… (Topic Name) โœ… (First Seen Tag Time) โŒ โœ…
sqs SQS โœ… (Queue Name) โœ… (Creation Time) โŒ โœ…
sagemaker-notebook-smni SageMakerNotebook โœ… (Notebook Instnace Name) โœ… (Creation Time) โŒ โœ…
secretsmanager SecretsManager โœ… (Secret Name) โœ… (Last Accessed or Creation Time) โŒ โœ…
security-hub SecurityHub โŒ โœ… (Created Time) โŒ โœ…
snap Snapshots โŒ โœ… (Creation Time) โœ… โœ…
transit-gateway TransitGateway โŒ โœ… (Creation Time) โŒ โœ…
transit-gateway-route-table TransitGatewayRouteTable โŒ โœ… (Creation Time) โŒ โœ…
transit-gateway-attachment TransitGatewaysVpcAttachment โŒ โœ… (Creation Time) โŒ โœ…
vpc VPC โœ… (EC2 Name Tag) โœ… (First Seen Tag Time) โŒ โŒ
route53-hosted-zone Route53HostedZone โœ… (Hosted zone name) โŒ โŒ โŒ
route53-cidr-collection Route53CIDRCollection โœ… (Cidr collection name) โŒ โŒ โŒ
route53-traffic-policy Route53TrafficPolicy โœ… (Traffic policy name) โŒ โŒ โŒ
network-firewall NetworkFirewall โœ… (Firewall name) โœ… (First Seen Tag Time) โœ… โŒ
network-firewall-policy NetworkFirewallPolicy โœ… (Firewall Policy name) โœ… (First Seen Tag Time) โœ… โŒ
network-firewall-rule-group NetworkFirewallRuleGroup โœ… (Firewall Rule group name) โœ… (First Seen Tag Time) โœ… โŒ
network-firewall-tls-config NetworkFirewallTLSConfig โœ… (Firewall TLS config name) โœ… (First Seen Tag Time) โœ… โŒ
network-firewall-resource-policy NetworkFirewallResourcePolicy โœ… (Firewall Resource Policy ARN) โŒ โŒ โŒ
vpc-lattice-service VPCLatticeService โœ… (VPC Lattice service ARN) (Creation Time) โŒ โœ…
vpc-lattice-service-network VPCLatticeServiceNetwork โœ… (VPC Lattice service network ARN) (Creation Time) โŒ โœ…
vpc-lattice-target-group VPCLatticeTargetGroup โœ… (VPC Lattice target group ARN) (Creation Time) โŒ โœ…

Resource Deletion and 'IsNukable' Check Option

Supported Resources for 'IsNukable' Check

For certain resources, such as AMI, EBS, DHCP Option, and others listed below, we support an option to verify whether the user has sufficient permissions to nuke the resources. If not, it will raise error: INSUFFICIENT_PERMISSION error.

Supported resources:

  • AMI
  • EBS
  • DHCP Option
  • Egress only Internet Gateway
  • Endpoints
  • Internet Gatway
  • IPAM
  • IPAM BYOASN
  • IPAM Custom Allocation
  • IPAM Pool
  • IPAM Resource Discovery
  • IPAM Scope
  • Key Pair
  • Network ACL
  • Network Interface
  • Subnet
  • VPC
  • Elastic IP
  • Launch Template
  • NAT Gateway
  • Network Firewall
  • Security Group
  • SnapShot
  • Transit Gateway

Unsupported Resources

Please note that the eligibility check for nukability relies on the DryRun feature provided by AWS. Regrettably, this feature is not available for all delete APIs of resource types. Hence, the 'eligibility check for nukability' option may not be accessible for all resource types

How to Use

Once you created your config file, you can run a command like this to nuke resources with your config file:

cloud-nuke aws --resource-type s3 --config path/to/file.yaml

CLI options override config file options

The options provided in the command line take precedence over those provided in any config file that gets passed in. For example, say you provide --resource-type s3 in the command line, along with a config file that specifies ec2: at the top level but doesn't specify s3:. The command line argument filters the resource types to include only s3, so the rules in the config file for ec2: are ignored, and ec2 resources are not nuked. All s3 resources would be nuked.

In the same vein, say you do not provide a --resource-type option in the command line, but you do pass in a config file that only lists rules for s3:, such as cloud-nuke aws --config path/to/config.yaml. In this case all resources would be nuked, but among s3 buckets, only those matching your config file rules would be nuked.

Be careful when nuking and append the --dry-run option if you're unsure. Even without --dry-run, cloud-nuke will list resources that would undergo nuking and wait for your confirmation before carrying it out.

Log level

By default, cloud-nuke sends most output to the Debug level logger, to enhance legibility, since the results of every deletion attempt will be displayed in the report that cloud-nuke prints after each run.

However, sometimes it's helpful to see all output, such as when you're debugging something.

You can set the log level by specifying the --log-level flag as per logrus log levels:

cloud-nuke aws --log-level debug

OR

LOG_LEVEL=debug cloud-nuke aws

Default value is - info. Acceptable values are debug, info, warn, error, panic, fatal, trace as per logrus log level parser.

Nuking only default security group rules

When deleting defaults with cloud-nuke defaults-aws, use the --sg-only flag to delete only the default security group rules and not the default VPCs.

cloud-nuke defaults-aws --sg-only

Note for nuking VPCs

When nuking VPCs cloud-nuke will attempt to remove dependency resources underneath the VPC

Supported VPC sub-resources

  • Internet Gateways
  • Egress Only Internet Gateways
  • Elastic Network Interfaces
  • VPC Endpoints
  • Subnets
  • Route Tables
  • Network ACLs
  • Security Groups
  • DHCP Option Sets (Will be dissociated from VPC, not deleted. Must be cleaned up separately)
  • Elastic IPs (Supported as a separate resource that gets cleaned up first. If you are filtering what gets nuked, Elastic IPs may prevent VPCs from destroying.)

All other resources that get created within VPCs must be cleaned up prior to running cloud-nuke on VPC resources.

VPC resources may not be entirely cleaned up on the first run. We believe this is caused by an eventual consistency error in AWS.

If you see errors like InvalidParameterValue: Network interface is currently in use. We recommend waiting 30 minutes and trying again.

Happy Nuking!!!

Credentials

AWS

In order for the cloud-nuke CLI tool to access your AWS, you will need to provide your AWS credentials. You can use one of the standard AWS CLI credential mechanisms.

Running Tests

go test -v ./...

Contributing

cloud-nuke is an open source project, and contributions from the community are very welcome! Please check out the Contribution Guidelines and Developing cloud-nuke for instructions.

Developing cloud-nuke

Running Locally

To run cloud-nuke locally, use the go run command:

go run main.go

Running tests

Note: Many of the tests in the aws folder run against a real AWS account and will create and destroy actual resources. DO NOT hit CTRL+C while the tests are running, as this will prevent them from cleaning up properly. We are not responsible for any charges you may incur.

Before running the tests, you must configure your AWS credentials.

To run all the tests:

go test -v ./...

To run only the tests in a specific package, such as the package aws:

cd aws
go test -v

And to run a specific test, such as TestListAMIs in package aws:

cd aws
go test -v -run TestListAMIs

And to run a specific test, such as TestLambdaFunction_GetAll in package aws/resources:

cd aws/resources
go test -v -run TestLambdaFunction_GetAll

Use env-vars to opt-in to special tests, which are expensive to run:

# Run acmpca tests
TEST_ACMPCA_EXPENSIVE_ENABLE=1 go test -v ./...

Formatting

Every source file in this project should be formatted with go fmt.

Releasing new versions

We try to follow the release process as deifned in our Coding Methodology.

Choosing a new release tag

If the new release contains any new resources that cloud-nuke will support, mark it as a minor version bump (X in v0.X.Y) to indicate backward incompatibilities.

This is because since version v0.2.0 cloud-nuke has been configured to automatically include new resources (so you have to explicitly opt-out). This is inherently not backward compatible, because users with CI practices around cloud-nuke would be surprised by new resources that are suddenly being picked up for deletion! This surprise is more alarming for resources that are actively in use for any account, such as IAM Users.

Therefore please mark your release as backward incompatible and bump the minor version (X in v0.X.Y) when it includes support for nuking new resources, so that we provide better signals for users when we introduce a new resource.

To release a new version

Go to the Releases Page and create a new release. The CircleCI job for this repo has been configured to:

  1. Automatically detect new tags.
  2. Build binaries for every OS using that tag as a version number.
  3. Upload the binaries to the release in GitHub.

See .circleci/config.yml for details.

Nukable error statuses

You'll encounter any of the following statuses when attempting to nuke resources, and here's what each status means:

  • error:INSUFFICIENT_PERMISSION : You don't have enough permission to nuke the resource.
  • error:DIFFERENT_OWNER : You are attempting to nuke a resource for which you are not the owner.

License

This code is released under the MIT License. See LICENSE.txt.

cloud-nuke's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cloud-nuke's Issues

Introducing you to AWSweeper (a similar tool)

Hi there,

it looks like the problem of trying to clean out Cloud accounts is present everywhere :-) -- I also started working on it last year, so I wanted show you what I've got (a tool named AWSweeper).

Maybe it's helpful for you to reuse some code, fork it and drive it, or contribute (not sure if it's smart for me to continue AWSweeper as a one-man show, which it currently is). Anyway, here are some thoughts behind the tool I wanted to share with you:

  • It currently supports deletion of 29 resource types (but there are so many more). Therefore, I followed a generic approach (via reflection) to easily support deletion of more types out of the box, where only some API information of go-aws-sdk routines to list and delete resources needs to be added to a config array (pointer to code).

  • It is built upon the existing delete methods of the AWS terraform provider (pointer to some code). I thought this might be helpful to get retries, detaching of some policies etc. from IAM resources, "forcing" of deletion where dependencies exist, and other stuff, for free.

  • Integration tests for each resource type using the Terraform testing framework (pointer to the tests).

  • I also started with a all-or-nothing-wipe-out approach, but then realised that it's handy to sometimes keep some resources (e.g. an IAM user + credentials to access the account). So, with AWSweeper one can filter resources by type, tags or ids described in a yaml config (see here). I have the idea to make filtering also more generic, i.e, allow filtering on all attributes available about a resource (such as creation date, etc), returned via the output struct of the Go API.

Thanks for reading & cheers,
Jan

cloud-nuke aws --exclude-resource-type s3 --dry-run NOT SUPPORTED

Hi,
When running:

cloud-nuke aws --exclude-resource-type s3 --dry-run
Incorrect Usage: flag provided but not defined: -exclude-resource-type

NAME:
cloud-nuke aws - BEWARE: DESTRUCTIVE OPERATION! Nukes AWS resources (ASG, ELB, ELBv2, EBS, EC2, AMI, Snapshots, Elastic IP, RDS).

USAGE:
cloud-nuke aws [command options] [arguments...]

OPTIONS:
--region value regions to include
--exclude-region value regions to exclude
--resource-type value Resource types to nuke
--list-resource-types List available resource types
--older-than value Only delete resources older than this specified value. Can be any valid Go duration, such as 10m or 8h. (default: "0s")
--dry-run Dry run without taking any action.
--force Skip nuke confirmation prompt. WARNING: this will automatically delete all resources without any confirmation

ERRO[2020-04-06T21:50:27Z] flag provided but not defined: -exclude-resource-type error="flag provided but not defined: -exclude-resource-type"

Like in the title, unfortunately, this is not working as described in the WIKI.
Thanks,
Agata

EC2 Instances from Automated Tests Are Left Around

I'm pretty consistently seeing EC2 Instances in every region with names like:

  • aws-nuke-test-3nOZAN
  • aws-nuke-test-iBLgDR

It looks like aws-nuke-test-3nOZAN was launched at February 20, 2018 at 7:49:30 AM UTC-7, so I'm pretty confident that aws-nuke tests aren't cleaning up after themselves. Either that, or tests are continually failing, but it doesn't look like that's the case, so the test clean up is probably the culprit.

To see for yourself, check out the PhxDevOps AWS account in any region.

aws profiles not supported?

Hi,
according to the documentation all "standard AWS CLI credential mechanisms" are supported by cloud-nuke. However, when I try to execute

cloud-nuke aws --profile my-profile-name

I get the following error: "flag provided but not defined: -profile"

Am I missing something?

Move to go modules

Right now go is with go 1.13.6, many repos have been moved to go modules which is the future go toolchain. So suggesting move this repo to also use go modules and deprecate dep.

cloud-nuke gcp

Hi,

I'm interested in using cloud-nuke for google cloud platform. I see it on the roadmap in the README. Are you accepting contributions for this feature?

Limit defaults-aws to security groups only

In the case of default VPCs that are still in use, a user should still be able to remediate for CIS by blanking the default security groups. An option on defaults-aws to exclude VPC destruction would allow this. Currently I do not see any AWS CIS requirements for removing all default VPCs. Only that the default security group restricts all traffic and others do not openly allow RDP or SSH.

Nuke VPC endpoint services

VPC endpoint services, not VPC endpoints directly, as they may be fronting NLBs spun up during testing. As part of deleting the VPC endpoint services, any existing endpoint connections would need to be rejected.

My reasoning for not targeting VPC endpoints is that, in my case, they are potentially tied into how the VPC is designed - such as allowing communication to AWS APIs without going onto the big-wide internet

String could not find any enabled regions

Hi,
I try to run cloud-nuke aws --resource-type ec2 --dry-run.
I have configured env variables:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_REGION.
I do not have aws cli installed.

When running I get the error message:
*errors.errorString could not find any enabled regions
/go/src/github.com/gruntwork-io/cloud-nuke/aws/aws.go:64 (0x85fd15e)
/go/src/github.com/gruntwork-io/cloud-nuke/aws/aws.go:77 (0x85fd1ae)
/go/src/github.com/gruntwork-io/cloud-nuke/commands/cli.go:128 (0x860fd5f)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/gruntwork-io/gruntwork-cli/errors/errors.go:93 (0x84d8343)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/urfave/cli/app.go:490 (0x84cabc2)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/urfave/cli/command.go:210 (0x84cbb48)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/urfave/cli/app.go:255 (0x84c93db)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/gruntwork-io/gruntwork-cli/entrypoint/entrypoint.go:21 (0x8611971)
/go/src/github.com/gruntwork-io/cloud-nuke/main.go:13 (0x8611b99)
/usr/local/go/src/runtime/proc.go:195 (0x807010d)
/usr/local/go/src/runtime/asm_386.s:1635 (0x8096351)
error="could not find any enabled regions"

I tried to follow #49
but I can't get the repo mentioned.
Could any one help me with this?
Thanks, Agata

What IAM permissions are required for cloud-nuke ?

Great tool! I'm wondering is there any documentation on the required permissions to run cloud-nuke?

currently I get

...is not authorized to perform: ecs:ListClusters on resource: *\n\tstatus code: 400,

which I can fix, but would be cool to know the prerequisite permissions in any case.

Contribution guideline should highlight dependency on dep, or update to go modules to simplify onboarding

While debugging #49 with @sugandh-pasricha - setting up a local environment to get cloud-nuke code gave the following error:

go get -v github.com/gruntwork-io/cloud-nuke
github.com/gruntwork-io/cloud-nuke/commands
# github.com/gruntwork-io/cloud-nuke/commands
commands/cli.go:22:5: app.Author undefined (type *cli.App has no field or method Author)
commands/cli.go:25:15: cannot use []cli.Command literal (type []cli.Command) as type []*cli.Command in assignment
commands/cli.go:31:24: cannot use cli.StringSliceFlag literal (type cli.StringSliceFlag) as type cli.Flag in array or slice literal:
        cli.StringSliceFlag does not implement cli.Flag (Apply method has pointer receiver)
commands/cli.go:35:24: cannot use cli.StringSliceFlag literal (type cli.StringSliceFlag) as type cli.Flag in array or slice literal:
        cli.StringSliceFlag does not implement cli.Flag (Apply method has pointer receiver)
commands/cli.go:39:24: cannot use cli.StringSliceFlag literal (type cli.StringSliceFlag) as type cli.Flag in array or slice literal:
        cli.StringSliceFlag does not implement cli.Flag (Apply method has pointer receiver)
commands/cli.go:43:17: cannot use cli.BoolFlag literal (type cli.BoolFlag) as type cli.Flag in array or slice literal:
        cli.BoolFlag does not implement cli.Flag (Apply method has pointer receiver)
commands/cli.go:47:19: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in array or slice literal:
        cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
commands/cli.go:52:17: cannot use cli.BoolFlag literal (type cli.BoolFlag) as type cli.Flag in array or slice literal:
        cli.BoolFlag does not implement cli.Flag (Apply method has pointer receiver)
commands/cli.go:62:17: cannot use cli.BoolFlag literal (type cli.BoolFlag) as type cli.Flag in array or slice literal:
        cli.BoolFlag does not implement cli.Flag (Apply method has pointer receiver)

I also tried and can confirm that go get github.com/gruntwork-io/cloud-nuke gives this error. This failure is because doing a go get gets github.com/urfav/cli which has a breaking change

~/go/src/github.com/urfave/cli# git describe --tags
v1.22.1-335-g0587424

because the pinned version is v1.20.0 as per https://github.com/gruntwork-io/cloud-nuke/blob/master/Gopkg.toml#L30

So if one wants to do local development they should ignore the above error and then go to the downloaded folder and do the following

cd gruntwork-io/cloudnuke
dep ensure -v
go build

which creates a local vendor folder.

Is there a way to ensure that go get github.com/gruntwork-io/cloud-nuke does not give the above error and gets the right versions of modules?

If not then we should have a "How to contribute" section in the README calling this out.

Ability to delete specific resources instead of all

cloud-nuke is great for cleaning up stale resources and saving $$. I wanted to run an idea through the team to know if this makes sense.

The current cli for cloud-nuke is

cloud-nuke aws

which when run gets all active AWS resources and then deletes them. This could be explicity called out by making the user specify an "all" argument e.g.

cloud-nuke aws all 

will delete all the supported resources. This means that the user can delete either all of the supported resources or specific ones by specifying

cloud-nuke aws ec2

which will delete only ec2 instances. The advantage of doing so are:

  1. Reducing search time.
  2. Targeting specific resources as someone might not want entire account deletion but only for specific components.
  3. Giving the user an ability to list out the currently supported resources from the cmdline instead of referring to the README.

Add region excludes to defaults-aws

In the case that 90% of the regions are unused but need to be CIS remediated before being disabled completely, it would help greatly to be able to nuke default-aws --exclude-region X in similar fashion to the standard nuke aws. Then at most the in use region can be manually remediated for CIS afterwards.

Feature Request - Removing AWS Config and AWS Config Rules

Just wondering if you are looking to in the future add support for removing all config rules.

Use Case would be if anyone has been playing with config rules and setup and forgot any rules they have configured and are being triggered will cause additional charges to users accounts.

It would be good if the tool could also clear these out so if someone wanted to start their account from scratch they can.

Thoughts anyone?

New feature: ability to delete IAM roles

To assist with hitting the IAM roles limit in AWS - we could use the ability to delete IAM roles - but to also provide a list of roles that should be preserved and not deleted.

Incomplete Execution Due to Missing Exception Handling

Please add exception handling to continue execution when an existing control prevents successful deletion of a resource.

$ ./cloud-nuke aws --log-level debug
INFO[2020-05-04T14:29:33-05:00] The following resources types will be nuked:
INFO[2020-05-04T14:29:33-05:00] - ami
INFO[2020-05-04T14:29:33-05:00] - asg
INFO[2020-05-04T14:29:33-05:00] - ebs
INFO[2020-05-04T14:29:33-05:00] - ec2
INFO[2020-05-04T14:29:33-05:00] - ecsserv
INFO[2020-05-04T14:29:33-05:00] - eip
INFO[2020-05-04T14:29:33-05:00] - ekscluster
INFO[2020-05-04T14:29:33-05:00] - elb
INFO[2020-05-04T14:29:33-05:00] - elbv2
INFO[2020-05-04T14:29:33-05:00] - lc
INFO[2020-05-04T14:29:33-05:00] - rds
INFO[2020-05-04T14:29:33-05:00] - s3
INFO[2020-05-04T14:29:33-05:00] - snap
INFO[2020-05-04T14:29:37-05:00] Retrieving active AWS resources in [eu-north-1, ap-south-1, eu-west-3, eu-west-2, eu-west-1, ap-northeast-2, ap-northeast-1, sa-east-1, ca-central-1, ap-southeast-1, ap-southeast-2, eu-central-1, us-east-1, us-east-2, us-west-1, us-west-2]
INFO[2020-05-04T14:29:37-05:00] Checking region [1/16]: eu-north-1
ERRO[2020-05-04T14:29:37-05:00] *awserr.requestError AccessDenied: User: arn:aws:sts::111122223333:role/admin/cloud-nuke-test is not authorized to perform: autoscaling:DescribeAutoScalingGroups with an explicit deny
	status code: 403, request id: 103e6933-9797-474d-b391-55ebfeb1f88d
/go/src/github.com/gruntwork-io/cloud-nuke/aws/asg.go:18 (0x18138da)
/go/src/github.com/gruntwork-io/cloud-nuke/aws/aws.go:207 (0x1817f83)
/go/src/github.com/gruntwork-io/cloud-nuke/commands/cli.go:204 (0x1831eaa)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/gruntwork-io/gruntwork-cli/errors/errors.go:93 (0x15f9d1b)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/urfave/cli/app.go:490 (0x15e89f2)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/urfave/cli/command.go:210 (0x15e9d65)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/urfave/cli/app.go:255 (0x15e6b58)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/gruntwork-io/gruntwork-cli/entrypoint/entrypoint.go:21 (0x1834167)
/go/src/github.com/gruntwork-io/cloud-nuke/main.go:13 (0x18343f7)
/usr/local/go/src/runtime/proc.go:195 (0x102b756)
	main: // A program compiled with -buildmode=c-archive or c-shared
/usr/local/go/src/runtime/asm_amd64.s:2337 (0x1057911)
	goexit: ???
  error="AccessDenied: User: arn:aws:sts::111122223333:role/admin/cloud-nuke-test is not authorized to perform: autoscaling:DescribeAutoScalingGroups with an explicit deny\n\tstatus code: 403, request id: 103e6933-9797-474d-b391-55ebfeb1f88d"

aws-nuke should pause when the --force flag is enabled

To give users a chance to hit CTRL+C in case of error, aws-nuke should pause for 10 seconds with the log output "Pausing for 10 seconds to give you a last chance to hit CTRL+C" whenever the --force flag is enabled.

Remove Default DHCP Options Set

I'm trying this tool and can cleanly remove defaults VPCs on all regions but the default DHCP options are not removed. Are there any reason not to remove DHCP option set? Thank you

Documentation for minimum IAM policy

There ought to be documentation for what the minimum IAM policy is to be able to run cloud nuke. This documentation would be valuable so that users can run cloud-nuke with a cron job and not worry about the user being overly permissive.

This was brought up in #106 but prematurely closed.

I know the README lists what services are targeted, but it does not list what specific permissions are needed. For example, does EC2 Auto Scaling need to be able to list everything or just DescribeAutoScalingGroup?

Adding support for non-dedicated accounts

Functional Description

Is there any way to support nuking environments that do not have fully dedicated accounts? Similar to #65 which added support for specific resource types, I'd like to delete resources:
(A) when the resources have a matching tag, and/or
(B) when the resource has an ID (ARN on AWS) which is contained within a specific list or tracked resource IDs

The latter case (B) would require that somewhere we are logging the IDs of resources which were created (not yet solved for) and would be needed in order to give an option for resources that do not support tags needed to implement (A). (For example: ECS clusters have an ARN but do not yet support tagging.)

Has this been considered or is there a way to implement this today? And if not, is it something you would consider in the roadmap and/or accept a PR for?

Sample Workflow

One option for an adapted workflow would look like:

  1. During the terraform design process, we apply a uniquely generated tag to apply to all (taggable) resources.
  2. For resources known to not be taggable (such as ECS) we log the ARNs into a text file or into state variables (or both)
  3. Nuke called twice, once with an arg like --nuke_tag=terraform-autotag-8fsCd and once with an arg like --resource_list=created_resources.txt

Additional Background

The specific use case is that (mostly for training labs and POCs), we don't always have the luxury of being able to create new AWS accounts, but we still want to have confidence that we can ALWAYS successfully destroy an environment after we are finished with it.

Thanks!

General UX Enhancement Ideas

Currently aws-nuke outputs its prompt as follows:

INFO[2018-02-18T12:02:57-07:00] Retrieving all active AWS resources
INFO[2018-02-18T12:04:05-07:00] The following AWS resources are going to be nuked:
INFO[2018-02-18T12:04:05-07:00] * ec2-i-023397578dd4e6940-eu-west-2

INFO[2018-02-18T12:04:05-07:00] * ec2-i-0d13286852fecf36f-eu-west-2

INFO[2018-02-18T12:04:05-07:00] * ebs-vol-095e6aabf8f382f20-eu-west-2

INFO[2018-02-18T12:04:05-07:00] * ebs-vol-0f6d2742b48945347-eu-west-2

INFO[2018-02-18T12:04:05-07:00] * ebs-vol-02af799dda2ba7e18-eu-west-2

INFO[2018-02-18T12:04:05-07:00] * asg-confluent-tools-OYr86p-0-sa-east-1

INFO[2018-02-18T12:04:05-07:00] * asg-confluent-tools-OYr86p-1-sa-east-1

INFO[2018-02-18T12:04:05-07:00] * asg-confluent-tools-OYr86p-2-sa-east-1
...

There are a few opportunities for improvement here:

  1. It would be helpful to see this output summarized by region.
  2. There's extra line between each line of output which causes more scrolling
  3. The INFO[2018-02-18T12:04:05-07:00] prefix doesn't add much value in this context. It'd be nice to have it without for the interactive prompt. Perhaps for automated runs it returns.
  4. Some summary stats (ASGs: 5, EC2 Instance: 5) by region and overall would be helpful.

None of these are critical, just suggestions for future improvements.

On the positive side, I really like the manual prompt the tool requires, however the prompt should include the AWS account number. Better yet, it'd be nice if there were some way to get the AWS Account Name (e.g. phxdevops).

Aws-nuke fails on Deleting EBS Volumes

While doing a fresh aws-nuke run, I received the following error:

INFO[2018-02-18T12:07:48-07:00] No Auto Scaling Groups to nuke in region eu-central-1
INFO[2018-02-18T12:07:48-07:00] No Elastic Load Balancers to nuke in region eu-central-1
INFO[2018-02-18T12:07:48-07:00] No V2 Elastic Load Balancers to nuke in region eu-central-1
INFO[2018-02-18T12:07:48-07:00] Terminating all EC2 instances in region eu-central-1
INFO[2018-02-18T12:07:49-07:00] Terminated EC2 Instance: i-0c61c9e0e5878f441
	INFO[2018-02-18T12:08:54-07:00] [OK] 1 instance(s) terminated in eu-central-1
INFO[2018-02-18T12:08:54-07:00] Deleting all EBS volumes in region eu-central-1
ERRO[2018-02-18T12:08:55-07:00] [Failed] VolumeInUse: Volume vol-0b43429069414b331 is currently attached to i-05333ad8fe9a89158
	status code: 400, request id: eeea65fe-4c56-414d-89b0-1878293db105

This makes me realize that part of deleting an EBS Volume means repeatedly checking that it has in fact been detached before initiating its termination.

Receiving an error="NoCredentialProviders:

I am trying this out for the first time and I was stopped in my tracks and don't know how to troubleshoot further. I use an AWS profile configuration to assume a role in an account. Is this supported? Am I overlooking something obvious?

guzzi:~ $ cloud-nuke --version
cloud-nuke version v0.1.13

guzzi:~ $ env | grep -i aws
AWS_PROFILE=secret-profile-name

guzzi:~ $ aws sts get-caller-identity
{
"UserId": "AROAI3SSNQ5VVVVMPHPQA:botocore-session-1579734394",
"Account": "xxxxxxxxxxxxx",
"Arn": "arn:aws:sts::xxxxxxxxxxxx:assumed-role/TeamRole/botocore-session-1579734394"
}

guzzi:~ $ cloud-nuke aws --region us-west-2 --dry-run
INFO[2020-01-22T15:14:22-08:00] Retrieving active AWS resources in [us-west-2]
INFO[2020-01-22T15:14:22-08:00] Checking region [1/1]: us-west-2
ERRO[2020-01-22T15:14:42-08:00] *awserr.baseError NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
/go/src/github.com/gruntwork-io/cloud-nuke/aws/asg.go:18 (0x16d48ea)
/go/src/github.com/gruntwork-io/cloud-nuke/aws/aws.go:204 (0x16d8468)
/go/src/github.com/gruntwork-io/cloud-nuke/commands/cli.go:149 (0x16eac46)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/gruntwork-io/gruntwork-cli/errors/errors.go:93 (0x15ef2cb)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/urfave/cli/app.go:490 (0x15ddfa2)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/urfave/cli/command.go:210 (0x15df315)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/urfave/cli/app.go:255 (0x15dc108)
/go/src/github.com/gruntwork-io/cloud-nuke/vendor/github.com/gruntwork-io/gruntwork-cli/entrypoint/entrypoint.go:21 (0x16ecc37)
/go/src/github.com/gruntwork-io/cloud-nuke/main.go:13 (0x16ecec7)
/usr/local/go/src/runtime/proc.go:195 (0x102b626)
/usr/local/go/src/runtime/asm_amd64.s:2337 (0x10577e1)
error="NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors"

Support wait for dependent resources

Example: we use Terraform to create a EBS-Instance and three ElPs and connect them.
cloud-nuke now runs into an issue: the EIPs can not be deleted while the EBS instance still exists, because EIPs can not be deleted while they are assigned.
Even running cloud-nuke twice in a row (we are trying to use it to automatically reset an account regularly) does not solve this problem, because the EBS-instance requires time to shut down - more time than cloud-nuke running twice.

Would cloud-nuke considering such dependencies be something that could be added in the future?
Since EBS-removal is already supported by cloud-nuke, how did you deal with such cases?

Add support for more complex matching rules

We have many clean-up use cases where we want to nuke all resources that match a specific pattern: e.g., nuke S3 buckets, IAM roles, and IAM profiles where the name matches one of a long list of specific regular expressions (i.e., those that come from our automated tests).

We should extend cloud-nuke with a way to filter resources using these match rules. Specifying them via CLI arguments is likely going to be inconvenient, so we may need some config format for this.

Add dry-run mode

Hi,

as you mentioned on #30 you should add a --dry-run mode.

I must confess I switch to awseeper because dry mode is missing

Regards

Nuke VPCs and related resources

A nice to have (but not critical) for the future would be the ability to nuke VPCs and all related resources:

  • The VPC itself
  • Subnets
  • Route tables
  • NACLs
  • VPC endpoints
  • NAT Gateways
  • ENIs / EIPs

Add support for deleting RDS snapshots

We are hitting limits in our testing account with having too many snapshots...

Of course, RDS snapshots are very sensitive things, so we should make sure to support careful filtering by tag, date, name, etc.

Allow specifying exclusion tags

The new s3 nuking capability has a built in mechanism to ignore any buckets that have the tag cloud-nuke-excluded. We should consider extending this to other resources.

See #101 (comment) for more context and suggestions.

Feature Request: protect resources from deletion based on a tag

Example use case:
Let's say you have a sandbox/dev account and want to use cloud-nuke to keep it clean of old artifacts, but on the other hand have resources that you're actively using and that may also be even older than your time threshold.

Solution:
Tag those resources with something like 'cloud-nuke'='protect'

Cloud-nuke should leave you with a 100% fresh account.

We recently received the following customer request:

So, I wanted a quick destruction of the environment created by terraform. cloud-nuke is great, but it leaves a lot of resources like rds, elastic ips, vpcs, buckets. sqs etc. I'd like to just go back to a state as if it was a fresh account.

For example I get a warning that elastic ips can't be deleted because they're attached somewhere. Specifically network interface which is not deleted by cloud nuke.

This is more of a meta-issue because updating cloud-nuke to actually get you to a 100% fresh account would mean having support for every available AWS resource, which is an aspirational goal but one that would be very difficult to achieve. Nevertheless, I wanted to record the feedback here so we have it written down.

Invalid response on prompt leads to immediate exit.

Cloud-nuke takes a while to identify all the resources it plans to delete in AWS and the finally displays this prompt:

Are you sure you want to nuke all listed resources? Enter 'nuke' to confirm: 

I accidentally entered yes instead of nuke, and the program immediately exited, which means I have to re-scan all AWS resources. Instead catch the error and give the user another chance to enter nuke. Also consider advising the user to use CTRL+C to cancel.

Test roles are leaking

Iam roles created on automated tests for this repo are not getting properly deleted and are accumulating inside our test AWS account.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.