This is an open-source tagging solution for AWS. Deploy AutoTag to Lambda using CloudTrail consumed through CloudWatch Events and have each of your resources tagged with the ARN of who created it. Optionally, resources can be tagged with when it was created and which AWS service invoked the request if one is provided. It was written by GorillaStack.
Read a blog post about the project.
Also see retro-tag for a solution to retrospectively tagging your resources using CloudTrail data.
Automatically tagging resources can greatly improve the ease of cost allocation and governance.
CloudWatch events delivers a near real-time stream of CloudTrail events as soon as a supported resource type is created. CloudWatch event rules triggers our AutoTag code to tag the resource. In this configuration the Lambda function is executed once each time it is triggered by the CloudWatch Event Rule (one event at a time). The CloudWatch Event Rule includes a pattern filter so it is only triggered by the supported events, meaning fewer Lambda invocations and lower operational costs.
The infrastructure consists of:
- S3 Bucket
- Main CloudFormation Stack (1 AWS region)
- Lambda Function
- IAM Role
- Collector CloudFormation Stack (All active AWS regions)
- CloudWatch Events Rule
- SNS Topic
Add pre-defined static tagging or custom tagging from the CloudTrail event. Using a JSON document, define one or more tags with either a hard-coded value or a value extracted from the CloudTrail event using variable substitution. Hard-coded tags will be applied to all supported AWS resources. When using variable substitution more than one variable can be provided in a single tag value, and if all of the substitutions in the field fail to be resolved the tag will not be written. That will allow for custom tags to be created using certain CloudTrail event fields that may not exist in all CloudTrail event types. Check out the CloudTrail Log Event Reference for the most common fields. Also, each AWS resource will have unique fields in the requestParameters and responseElements fields that can be used. Examples of specific AWS resource CloudTrail events can be found at CloudTrail Log File Examples or by searching in the CloudTrail event history.
Example:
{
"AutoTag_ManagedBy": "Site Reliability Engineering",
"AutoTag_UserIdentityType": "$event.userIdentity.type",
"AutoTag_UserName": "$event.userIdentity.userName",
"AutoTag_ClientInfo": "SourceIP: $event.sourceIPAddress - UserAgent: $event.userAgent",
"AutoTag_Ec2_ImageId": "$event.responseElements.instancesSet.items.0.imageId"
}You will need at least 1 AWS Account, and CloudTrail should be enabled.
We have documented two different ways to deploy the infrastructure to an AWS account. Since there are CloudFormation stacks that need to be deployed in multiple regions we've provided a script that uses the AWS CLI to deploy everything for you. The other deployment method uses CloudFormation StackSets to deploy across multiple regions.
This deploy script deploy_autotag.sh will create, delete, or update all of the AutoTag infrastructure for a single AWS account.
The script will attempt to auto-install its own dependencies: aws-cli, jq, npm, git, zip
The create command will start by creating a dedicated AutoTag S3 Bucket for storing code deployment packages in your AWS account. Then it will download or build the code package, and create both the main CloudFormation stack and the collector CloudFormation stacks. When executing the delete command all resources will be removed except the S3 bucket. Use the update-release command to update existing CloudFormation stacks to a specific release, update-master to update to the master branch (build required), or update-local to update to the local cloned git repo (build required).
The deploy script can use all of the credential providers that the AWS CLI allows, see Configure AWS CLI and take a look at the deployment examples. A separate set of CLI credentials can be provided by the argument --s3-profile for utilizing a single S3 bucket when deploying infrastructure across multiple AWS accounts. The script will also secure the S3 bucket by blocking all public access configuration, and add the required S3 bucket-policy statement to allow the cross-account GetObject access if necessary.
The script needs at minimum the IAM permissions described in this policy: deploy_iam_policy.json
Before using this IAM policy replace the 2 occurrences of my-autotag-bucket with the name of your actual AutoTag S3 bucket.
Usage: deploy_autotag.sh [options] <command>
Commands:
create Create the AutoTag infrastructure
delete Delete the AutoTag infrastructure
update-release Update the AutoTag infrastructure with a specific release version
update-master Update the AutoTag infrastructure with the latest from the master branch
update-local Update the AutoTag infrastructure with the local source code
Options:
-h --help Show this screen
-r --region The primary AWS region where the main CloudFormation stack will be deployed
-p --profile The main AWS credential profile
-s3bu --s3-bucket The S3 bucket where the code package will be uploaded
-s3pr --s3-profile A separate AWS credential profile to upload code packages to the S3 Bucket
-rv --release-version The release version to deploy, e.g. '0.5.2' or 'latest'
-lr --log-retention-days The number of days to retain the Lambda Function's logs (default: 90)
-ld --log-level-debug Enable the debug logging for the Lambda Function
-dct --disable-create-time Disable the 'CreateTime' tagging for all AWS resources
-dib --disable-invoked-by Disable the 'InvokedBy' tagging for all AWS resources
-ct --custom-tags Define custom tags in a JSON document
Follow these steps to prepare to run the create command.
- Select a primary AWS
--regionfor the S3 bucket and the Main CloudFormation stack - Pick a dedicated AutoTag
--s3-bucketname, e.g. 'acme-autotag' - Configure AWS credentials for the AWS CLI, see Configure AWS CLI
Download the latest version of deploy_autotag.sh, or find it in the root of the repository.
curl -LO https://raw.githubusercontent.com/GorillaStack/auto-tag/master/deploy_autotag.sh
chmod +x deploy_autotag.sh
Create the infrastructure with the latest release using either the default, $AWS_PROFILE, or instance AWS credentials profile.
./deploy_autotag.sh --region us-west-2 --s3-bucket my-autotag-bucket --release-version latest createCreate the infrastructure with the latest release using a named AWS credentials profile.
./deploy_autotag.sh -r us-west-2 -s3bu my-autotag-bucket --release-version latest --profile dev-acct createCreate the infrastructure using $AWS_ACCESS_KEY_ID and $AWS_SECRET_ACCESS_KEY.
export AWS_ACCESS_KEY_ID=XXX
export AWS_SECRET_ACCESS_KEY=YYY
./deploy_autotag.sh -r us-west-2 -s3bu my-autotag-bucket createCreate the infrastructure using a named AWS credentials profile (--profile), but with the S3 Bucket operations utilizing a separate AWS credential profile (--s3-profile). Use this feature to deploy across multiple accounts using a single S3 bucket.
./deploy_autotag.sh -r us-west-2 -s3bu my-autotag-bucket --profile dev-acct --s3-profile s3-acct createCreate the infrastructure with an additional custom tag with a static value, this tag will be applied globally across all of the supported AWS resources.
./deploy_autotag.sh -r us-west-2 -s3bu my-autotag-bucket create \
--custom-tags '{"AutoTag_ManagedBy": "Site Reliability Engineering"}'Create the infrastructure with an additional event-based custom tag, any key in the CloudTrail event is valid to use and it will be applied globally across all of the supported AWS resources unless the field does not exist in the CloudTrail event.
./deploy_autotag.sh -r us-west-2 -s3bu my-autotag-bucket create \
--custom-tags '{"AutoTag_UserIdentityType": "$event.userIdentity.type"}'Interpolation with text in the value is supported and more than one field from the event can be rendered in a single tag value.
./deploy_autotag.sh -r us-west-2 -s3bu my-autotag-bucket create \
--custom-tags '{"AutoTag_ClientInfo": "SourceIP: $event.sourceIPAddress - UserAgent: $event.userAgent"}'Update the infrastructure to the bleeding edge (master).
./deploy_autotag.sh -r us-west-2 -s3bu my-autotag-bucket update-masterUpdate the infrastructure to the latest git release.
./deploy_autotag.sh -r us-west-2 -s3bu my-autotag-bucket --release-version latest update-releaseUpdate the infrastructure to a specific git release - only works for releases >= 0.5.1.
./deploy_autotag.sh -r us-west-2 -s3bu my-autotag-bucket --release-version 0.5.2 update-releaseUpdate the infrastructure to the local git folder's current state.
git clone https://github.com/GorillaStack/auto-tag.git
cd auto-tag
./deploy_autotag.sh -r us-west-2 -s3bu my-autotag-bucket update-localDelete the infrastructure.
./deploy_autotag.sh -r us-west-2 deleteCloudFormation StackSet Deployment Method
Currently Auto-Tag, supports the following AWS resource types:
Tags Applied: C=Creator, T=Create Time, I=Invoked By
| Technology | Event Name | Tags Applied | IAM Deny Tag Support |
|---|---|---|---|
| AutoScaling Group | CreateAutoScalingGroup | C, T, I | Yes |
| ASG Instances w/ENI & Vol | RunInstances | C, T, I | Yes |
| Data Pipeline | CreatePipeline | C, T, I | No |
| DynamoDB Table | CreateTable | C, T, I | No |
| CloudWatch Alarm ? | PutMetricAlarm | C, T, I | ? |
| CloudWatch Events Rule ? | PutRule | C, T, I | ? |
| CloudWatch Log Group ? | CreateLogGroup | C, T, I | ? |
| EBS Volume | CreateVolume | C, T, I | Yes |
| EC2 AMI w/Snapshot * | CreateImage | C, T, I | Yes |
| EC2 AMI w/Snapshot * | CopyImage | C, T, I | Yes |
| EC2 AMI * | RegisterImage | C, T, I | Yes |
| EC2 Customer Gateway ? | CreateCustomerGateway | C, T, I | ? |
| EC2 DHCP Options ? | CreateDhcpOptions | C, T, I | ? |
| EC2 Elastic IP | AllocateAddress | C, T, I | Yes |
| EC2 ENI | CreateNetworkInterface | C, T, I | Yes |
| EC2 Instance w/ENI & Volume | RunInstances | C, T, I | Yes |
| EC2 / VPC Security Group | CreateSecurityGroup | C, T, I | Yes |
| EC2 Snapshot * | CreateSnapshot | C, T, I | Yes |
| EC2 Snapshot * | CopySnapshot | C, T, I | Yes |
| EC2 Snapshot * | ImportSnapshot | C, T, I | Yes |
| Elastic LB (v1 & v2) | CreateLoadBalancer | C, T, I | No |
| EMR Cluster | RunJobFlow | C, T, I | No |
| IAM Role | CreateRole | C, T, I | ? |
| IAM User | CreateUser | C, T, I | ? |
| Lambda Function ? | CreateFunction20150331 | C, T, I | ? |
| Lambda Function ? | CreateFunction20141111 | C, T, I | ? |
| OpsWorks Stack | CreateStack | C | No |
| OpsWorks Clone Stack * | CloneStack | C | No |
| OpsWorks Instances w/ENI & Vol | RunInstances | C, T, I | Yes |
| RDS Instance | CreateDBInstance | C, T, I | No |
| S3 Bucket | CreateBucket | C, T, I | No |
| NAT Gateway | CreateNatGateway | C, T, I | Yes |
| VPC | CreateVpc | C, T, I | Yes |
| VPC Internet Gateway | CreateInternetGateway | C, T, I | Yes |
| VPC Network ACL | CreateNetworkAcl | C, T, I | Yes |
| VPC Peering Connection | CreateVpcPeeringConnection | C, T, I | Yes |
| VPC Route Table | CreateRouteTable | C, T, I | Yes |
| VPC Subnet | CreateSubnet | C, T, I | Yes |
| VPN Connection | CreateVpnConnection | C, T, I | Yes |
| VPN Gateway ? | CreateVpnGateway | C, T, I | ? |
*=not tested by the test suite
NOTE: When tag-able resources are created using CloudFormation StackSets the "Creator" tag is NEVER populated with the ARN of the user who executed the StackSet, instead it is tagged with the less useful CloudFormation StackSet Execution Role's "assumed-role" ARN.
Use the following IAM policy to deny a user or role the ability to create, delete, and edit any tag starting with 'AutoTag_'. The ec2:CreateAction condition allows users to create EC2 instances with tags starting with 'AutoTag_', this enables the 'Launch More Like This' feature, in that case the tags will be overwritten after the instance is created.
{
"Sid": "DenyAutoTagPrefix",
"Effect": "Deny",
"Action": [
"ec2:CreateTags",
"ec2:DeleteTags",
"autoscaling:CreateOrUpdateTags",
"autoscaling:DeleteTags"
],
"Condition": {
"ForAnyValue:StringLike": {
"aws:TagKeys": "AutoTag_*"
},
"StringNotEquals": {
"ec2:CreateAction": [
"RunInstances"
]
}
},
"Resource": "*"
}NOTE: At the time of this writing the deny tag IAM condition (aws:TagKeys) is only available for resources in EC2 and AutoScaling, see the table above for a status of each resource.
If you have questions, feature requests or bugs to report, please do so on the issues section of our github repository.
If you are interested in contributing, please get started by forking our GitHub repository and submit pull-requests.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent