answers4aws / graffiti-monkey Goto Github PK

What file denotes the format of the Tag "Key". I'd like to change "instance_id" and "device" to something else, but can't find the exact file or files where this is set? I have tried changing it where I can see it, but doesn't seem to make any difference.

I would like to specify which profile in my .boto file to use.

I tried to execute the below command on windows and found that this is due to the python time formatting in windows (particularly %T)

reference: http://stackoverflow.com/questions/10807164/python-time-formatting-different-in-windows

C:\Users\sp>graffiti-monkey --region us-east-1 -c C:\Users\sp\De
sktop\test.yml

ValueError: Invalid format string
Logged from file core.py, line 85
Traceback (most recent call last):
File "c:\Python27\lib\logging__init__.py", line 859, in emit
msg = self.format(record)
File "c:\Python27\lib\logging__init__.py", line 732, in format
return fmt.format(record)
File "c:\Python27\lib\logging__init__.py", line 473, in format
record.asctime = self.formatTime(record, self.datefmt)
File "c:\Python27\lib\logging__init__.py", line 431, in formatTime
s = time.strftime(datefmt, ct)

Hi Peter,

I run graffiti-monkey and it mentioned that No AWS credentials found. Do I need to manually create a credential file or does it get the credential information from awscli credential file?

Thanks.

Regards,
Yew Wei

Would love to be able to easily configure which tags are propagated.

We have some "aws internal" tags we want to propagate to the volume and the snapshots:

aws:autoscaling:groupName

But Amazon didn't allow to set such Tags. So they have to be renamed (replace ":" with "-" for example). Is there a way to do that?

Hi Guys,

I am trying to use filter option for one particular instance or tag value buts not working. Can anyone help me on this please. want to test on one instance first in PROD before implementing in entire environment.

This script is fantastic, saved me a lot of time, thanks for all your work on it. It helped me tag all our volumes and snapshots and it would be great if it could also clean up all our old AMI's by tagging those also.

It's weird to install a v1 from github/source instead of the pypi that already exists but is outdated.

I see the project has been inactive since 2016 in terms of commits and closed issues/PRs.

There is boto3 PR, a fork to support lambda and I would like to see docker and kubernetes helm charts added. Is there any leadership to maintain this?

is it possible to run graffity-monkey in all regions or i have to execute again changing region flag?

Thanks, great tool!

Would like to see Graffiti Monkey have the ability to assume roles in other accounts, like Backup Monkey already does.

1. When i do graffiti-monkey -h it does NOT show below options.
   --volumes volume(s) to tag
   --snapshots snapshot(s) to tag
   --novolumes do not perform volume tagging
   --nosnapshots do not perform snapshot tagging
2. Using --volumes (or --volume) gives before error.
   graffiti-monkey: error: unrecognized arguments: --volumes vol-c73b4601
3. Config files sections not working at all.
   it setting up tags on all volumes / snapshot even when list is not empty.
   it is ignoring mandatory tag setting by "_volume_tags_to_be_set:"

Does anyone have a successful IAM policy for Graffiti Monkey they can share?

I've tried a few but I am clearing missing something as still getting 404 from boto.

I am trying to update tags on volumes and snapshots but getting rate limit exceeded error.

<?xml version="1.0" encoding="UTF-8"?> <Response><Errors><Error><Code>FilterLimitExceeded</Code><Message>The maximum number of filter values specified on a single call is 200</Message></Error></Errors> <RequestID>b7d7c7a5-e8ef-4ff4-addb-1b82783c034a</RequestID></Response>

Can there be an option added to copy AMI tags to their associated snapshot?

Python 2.7 will be deprecated on July 2021 and we are getting below error in Python 3.8 AWS Lambda Function.

`{

"errorMessage": "Unable to import module 'service': No module named 'graffiti_monkey'",
"errorType": "Runtime.ImportModuleError",
"stackTrace": []
}`

Is 'graffiti_monkey' supports latest python versions as well?

Hi,

when making use of a .yaml file the _snapshot_tags_to_be_set and _volume_tags_to_be_set parameters are mandatory.
I only want to propagate existing tags from instances to volumes to snapshots and not add default tags to everything.

A config like:
`---
_instance_tags_to_propagate:

• 'Name'

_volume_tags_to_propagate:

• 'Name'
• 'instance_id'
• 'device'
  `

Fails with:
$ graffiti-monkey --region ap-southeast-1 --dryrun --config config.yaml --verbose Traceback (most recent call last): File "/usr/local/bin/graffiti-monkey", line 9, in <module> load_entry_point('graffiti-monkey==1.0.0', 'console_scripts', 'graffiti-monkey')() File "/usr/local/lib/python2.7/dist-packages/graffiti_monkey-1.0.0-py2.7.egg/graffiti_monkey/cli.py", line 211, in run cli.run() File "/usr/local/lib/python2.7/dist-packages/graffiti_monkey-1.0.0-py2.7.egg/graffiti_monkey/cli.py", line 200, in run self.initialize_monkey() File "/usr/local/lib/python2.7/dist-packages/graffiti_monkey-1.0.0-py2.7.egg/graffiti_monkey/cli.py", line 166, in initialize_monkey self.config["_volume_tags_to_be_set"], KeyError: '_volume_tags_to_be_set'

It also seems that the dryrun has a problem.
In the dryrun (new) tags are only propagated from instances to volumes but not from volumes to snapshots.
I guess in the 2nd run with the snapshots the volumes are still the same (because its a dryrun) and the tags are not there.

Cheers,
Felix

Hi Team,

How do we propagate all the tags that are there for an ec2 instance to its corresponding EBS and Snapshots?

Thanks,
Raghav

I just ran graffiti-monkey for the first time, and while all the volumes were tagged appropriately, graffiti-monkey crashed when it came to tagging snapshots with the same tags.

Here's the output from the error:

2016-11-04 00:29:36 [INFO] Completed processing all volumes
2016-11-04 00:29:36 [INFO] Getting list of all snapshots
2016-11-04 00:29:41 [INFO] Found 1415 snapshots
2016-11-04 00:29:41 [INFO] Processing snapshot 1 of 1415 total snapshots
Traceback (most recent call last):
  File "/usr/local/bin/graffiti-monkey", line 11, in <module>
    sys.exit(run())
  File "/usr/local/lib/python2.7/site-packages/graffiti_monkey/cli.py", line 168, in run
    cli.run()
  File "/usr/local/lib/python2.7/site-packages/graffiti_monkey/cli.py", line 158, in run
    self.start_tags_propagation()
  File "/usr/local/lib/python2.7/site-packages/graffiti_monkey/cli.py", line 138, in start_tags_propagation
    self.monkey.propagate_tags()
  File "/usr/local/lib/python2.7/site-packages/graffiti_monkey/core.py", line 71, in propagate_tags
    self.tag_snapshots()
  File "/usr/local/lib/python2.7/site-packages/graffiti_monkey/core.py", line 153, in tag_snapshots
    self.tag_snapshot(snapshot)
  File "/usr/local/lib/python2.7/site-packages/graffiti_monkey/core.py", line 180, in tag_snapshot
    for tag_name in self._volume_tags_to_propagate:
TypeError: 'NoneType' object is not iterable

Replacing boto with boto3 will make it easy to run graffiti monkey as a lambda.

An attempt to make this change is here: https://github.com/joehack3r/graffiti-monkey/tree/migrate-to-boto3

There should be a nice message, rather than a Boto exception

The ability to be able to use a LB, as the root for the tagging would be handy.

Hi,

I have installed and set-up the graffiti monkey. However, AWS CLI is running perfect and I'm a admin user but still getting the unauthorized errors. Please check the below logs and help me:

$ graffiti-monkey -v --region us-west-2
2017-07-07 15:37:11 [INFO] [graffiti_monkey.core(51):init] Connecting to region us-west-2 using profile default
2017-07-07 15:37:11 [INFO] [graffiti_monkey.core(78):tag_volumes] Getting list of all volumes
Traceback (most recent call last):
File "/usr/bin/graffiti-monkey", line 11, in
sys.exit(run())
File "/usr/lib/python2.7/site-packages/graffiti_monkey/cli.py", line 168, in run
cli.run()
File "/usr/lib/python2.7/site-packages/graffiti_monkey/cli.py", line 158, in run
self.start_tags_propagation()
File "/usr/lib/python2.7/site-packages/graffiti_monkey/cli.py", line 138, in start_tags_propagation
self.monkey.propagate_tags()
File "/usr/lib/python2.7/site-packages/graffiti_monkey/core.py", line 70, in propagate_tags
self.tag_volumes()
File "/usr/lib/python2.7/site-packages/graffiti_monkey/core.py", line 79, in tag_volumes
volumes = self._conn.get_all_volumes()
File "/usr/lib/python2.7/site-packages/boto/ec2/connection.py", line 2157, in get_all_volumes
[('item', Volume)], verb='POST')
File "/usr/lib/python2.7/site-packages/boto/connection.py", line 1186, in get_list
raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 403 Forbidden

UnauthorizedOperationYou are not authorized to perform this operation.2ac05d20-ee08-4a73-9b41-0e32aba32c44

Thanks.

I've made the following changes:

• commented out the following in core.py:

tags_to_set['instance_id'] = instance_id
tags_to_set['device'] = device

• using this config:

---
region: us-east-1
_instance_tags_to_propagate:
  - 'Environment'

_volume_tags_to_propagate:
  - 'Environment'

_volume_tags_to_be_set:
#  -  key:   'role'
#     value: 'ebs'

_snapshot_tags_to_be_set:
#  -  key:   'role'
#     value: 'ebs_snapshot'

_instance_filter:
# Filter values with the same key (eg, tag:app) are an OR
# Different filter keys are an AND operation (eg, tag:app and tag:team)
# See the Filter section in http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html#API_DescribeInstances_RequestParameters
# Example entries:
#    'tag:app':   ['app1', 'app2']
#    'tag:team': 'team1'

_volumes_to_tag:
# An empty list means tag all volumes
# Example entries:
#  - 'vol-1ab2c345'
  - 'vol-0784cc294e20cbb39'

_snapshots_to_tag:
# An empty list means tag all snapshots
# Example entries:
#  - 'snap-12ab3c45'
#  - 'snap-6de7f890'

• Running ./graffiti-monkey --config ../conf/example_config.yml --append --nosnapshots --dryrun tells me it is going to make the following changes, adding the Environment tag:

2018-06-13 20:24:08 [INFO] Starting Graffiti Monkey
2018-06-13 20:24:08 [INFO] Options: dryrun True, append True, novolumes False, nosnapshots True
2018-06-13 20:24:08 [INFO] Connecting to region us-east-1 using profile default
2018-06-13 20:24:08 [INFO] Using volume list from cli/config file
2018-06-13 20:24:09 [INFO] Found 1 volume(s)
2018-06-13 20:24:09 [INFO] Processing volume 1 of 1 total volumes
2018-06-13 20:24:09 [INFO] DRYRUN: Volume vol-0784cc294e20cbb39 would have been tagged {u'Name': u'kubernetes.metismachine.io-dynamic-pvc-46cc64b6-6a94-11e8-a763-02c1f252ba9e', u'kubernetes.io/created-for/pvc/namespace': u'spark-cluster', u'kubernetes.io/cluster/kubernetes.metismachine.io': u'owned', u'kubernetes.io/created-for/pvc/name': u'cassandra-data-cassandra-3', 'Environment': u'production', u'kubernetes.io/created-for/pv/name': u'pvc-46cc64b6-6a94-11e8-a763-02c1f252ba9e', u'KubernetesCluster': u'kubernetes.metismachine.io'}
2018-06-13 20:24:09 [INFO] Processed a total of 300 GB of AWS Volumes
2018-06-13 20:24:09 [INFO] Completed processing all volumes
2018-06-13 20:24:09 [INFO] Graffiti Monkey completed successfully!

• However, when I run it without the --dryrun, it tells me it ran successfully but the EBS volume, the one in my config file, vol-0784cc294e20cbb39, the one in my AWS account, does not have the new Environment tag.

[20:24] jmorganwalker@ip-10-0-1-11:~DEV_ROOT/graffiti-monkey/bin# ./graffiti-monkey --config ../conf/example_config.yml --append --nosnapshots                                            master ✱ ◼
2018-06-13 20:24:29 [INFO] Starting Graffiti Monkey
2018-06-13 20:24:29 [INFO] Options: dryrun False, append True, novolumes False, nosnapshots True
2018-06-13 20:24:29 [INFO] Connecting to region us-east-1 using profile default
2018-06-13 20:24:29 [INFO] Using volume list from cli/config file
2018-06-13 20:24:30 [INFO] Found 1 volume(s)
2018-06-13 20:24:30 [INFO] Processing volume 1 of 1 total volumes
2018-06-13 20:24:30 [INFO] Processed a total of 300 GB of AWS Volumes
2018-06-13 20:24:30 [INFO] Completed processing all volumes
2018-06-13 20:24:30 [INFO] Graffiti Monkey completed successfully!

• I tried running with -vvvvv and I can see it go through what seems to be a successful boto responses but still nothing in AWS

Any help would be greatly appreciated.

Currently if options are specified in both the config file and as command line arguments, the config file 'wins'. I think this is the wrong way around and am proposing reversing this order.

By reversing the logic, my standard options would be in the config file, and if I want to override those options (my defaults) I use a command line argument.

This has become more of an issue now I've added support for items such as the boto profile and volume ids as command line arguments. e.g. I want my 'test' profile to be the default so I put that in the config file, but if I want to target my production account I have to edit the config file - it would be easier if I could use the command line --profile argument to override.

Thoughts on changing the primary option source to be the command line argument?

Thanks,

Paul

Hi,

Anyone run this as a Lambda job? How easy would this be to get working in Lambda?

Thanks,
Andrew.

if it can only take instance ids as arguments and then perform its magic ont he tag copying that will be great , currently it just takes the whole region and start doing every thing