jcjorel / ec2-spot-converter Goto Github PK

View Code? Open in Web Editor NEW

177.0 11.0 20.0 464 KB

A tool to convert AWS EC2 instances back and forth between On-Demand and Spot billing models.

License: MIT License

Python 99.95% Shell 0.05%

ec2 spot ec2-spot ec2-spot-instances converter tool cost-optimization ec2-on-demand conversion billing-model

ec2-spot-converter's People

Contributors

Stargazers

Watchers

ec2-spot-converter's Issues

A Way to change VPC/Network/Zone

Just a Suggestion, options to change vpc / network-id . We faced a problem where was not availability of a instance type in the current zone and we had to change network id changing zone

ELB target group retrieval issue

I'm using the the script for an on-demand to spot conversion with the invocation

./ec2-spot-converter --stop-instance --review-conversion-result --instance-id <id> --update-cw-alarms --check-targetgroups * --wait-for-tg-states

Output:

[INFO] 2021-02-28 06:07:19,059 ec2-spot-converter - [STEP 1/26] Read DynamoDB state table...
[INFO] 2021-02-28 06:07:19,116 ec2-spot-converter -   => SUCCESS. Record '<id>' read succesfully.
[INFO] 2021-02-28 06:07:19,142 ec2-spot-converter - [STEP 2/26] Discover instance state...
[ERROR] 2021-02-28 06:07:19,631 ec2-spot-converter - Failed to list target groups: An error occurred (ValidationError) when calling the DescribeTargetGroups operation: 'ec2-spot-converter' is not a valid target group ARN.
[ERROR] 2021-02-28 06:07:19,631 ec2-spot-converter - Failed to perform step 'DiscoverInstanceState'! Reason=Failed to retrieve ELB target groups!

I've used the latest policy.json attached as an IAM role.
Not sure if it needs the * to be in quotes?

Update: The load balancer in use is an application load balancer

ec2 target group registration is lost after transition

I have load balancer target groups that have 2c2 instances registered as their targets. After the transition to spot, those registrations are lost.
Suggested fix: save the registration information of the instances and reattach them after the new instance is up

Failure while creating instance

At step 16/25 the script fails with perhaps some permission issue. I used the policy.json attached in repo

[INFO] 2021-02-09 18:25:45,871 ec2-spot-converter - [STEP 14/25] Terminate instance...
[INFO] 2021-02-09 18:25:45,978 ec2-spot-converter -   => SUCCESS. Successfully terminated instance i-0g6j3gaeda57a3d.
[INFO] 2021-02-09 18:25:45,998 ec2-spot-converter - [STEP 15/25] Wait resource release...
[INFO] 2021-02-09 18:25:46,089 ec2-spot-converter - Waiting for detached ENIs to become 'available'...
[INFO] 2021-02-09 18:25:53,259 ec2-spot-converter -   => SUCCESS. All resources released : ['eni-018kf47ke05c84'].
[INFO] 2021-02-09 18:25:53,277 ec2-spot-converter - [STEP 16/25] Create new instance...
Traceback (most recent call last):
  File "./ec2-spot-converter", line 1471, in <module>
    sys.exit(main(sys.argv))
  File "./ec2-spot-converter", line 1437, in main
    return_code, reason, keys = step["Function"]()
  File "./ec2-spot-converter", line 939, in create_new_instance
    response = ec2_client.run_instances(**launch_specifications)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (UnauthorizedOperation) when calling the RunInstances operation: You are not authorized to perform this operation.

The script gave an encoded message. I'll try to figure out decoding it

Usage as a Lambda Function?

As title says, do you see any roadblocks to adopting this code for a lambda function? I'm worried about potential timeouts and refactoring the code to be able to be called from a lambda event.

What are your thoughts?

Tag Keys starting with aws: error with reserved word

When instances are created with CloudFormation or Service Catalog, the services will create their own tags on instances. These tags all start with "aws:". When trying to convert, the error comes up that these are reserved tags.....

The tag keys maybe just renamed to something like changing "aws:cloudformation" to "AWScloudformation".

botocore.exceptions.ClientError: An error occurred (InvalidParameterValue) when calling the RunInstances operation: Tag keys starting with 'aws:' are reserved for internal use

error deregistering image (--delete-ami)

Getting this error when using --delete-ami

[INFO] 2023-06-05 12:31:49,365 ec2-spot-converter - [STEP 22/26] Reboot new instance (if needed and requested)... 
[INFO] 2023-06-05 12:31:49,365 ec2-spot-converter -   => SUCCESS. No reason to reboot instance 'i-0113bca587b7f5c14'... Skipping...
[INFO] 2023-06-05 12:31:49,568 ec2-spot-converter - [STEP 23/26] Update CloudWatch alarms... => SKIPPED! Need '--update-cw-alarms' argument.
[INFO] 2023-06-05 12:31:49,568 ec2-spot-converter - [STEP 24/26] Untag resources... 
[INFO] 2023-06-05 12:31:49,870 ec2-spot-converter -   => SUCCESS. Successfully untagged ['i-0113bca587b7f5c14', 'eni-0e2f07f1dbb86ca46'].
[INFO] 2023-06-05 12:31:50,102 ec2-spot-converter - [STEP 25/26] Waiting for instance to be at expected states in target groups... => SKIPPED! Need '--wait-for-tg-states' argument.
[INFO] 2023-06-05 12:31:50,102 ec2-spot-converter - [STEP 26/26] Deregister image... 
Traceback (most recent call last):
  File "/Users/rsoares/dev/github/rafaeltuelho/sno-for-100/.run/ec2-spot-converter", line 1621, in <module>
    sys.exit(main(sys.argv))
             ^^^^^^^^^^^^^^
  File "/Users/rsoares/dev/github/rafaeltuelho/sno-for-100/.run/ec2-spot-converter", line 1587, in main
    return_code, reason, keys = step["Function"]()
                                ^^^^^^^^^^^^^^^^^^
  File "/Users/rsoares/dev/github/rafaeltuelho/sno-for-100/.run/ec2-spot-converter", line 1212, in deregister_image
    snap_ids = [blk["Ebs"]["SnapshotId"] for blk in img["BlockDeviceMappings"]]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rsoares/dev/github/rafaeltuelho/sno-for-100/.run/ec2-spot-converter", line 1212, in <listcomp>
    snap_ids = [blk["Ebs"]["SnapshotId"] for blk in img["BlockDeviceMappings"]]
                ~~~^^^^^^^
KeyError: 'Ebs'

Invalid option error for r6g.medium spot conversion

@jcjorel Perhaps it is due to the new instance type but when I tried converting an r6g.medium from on demand to spot it gave me this error

[INFO] 2021-08-29 08:01:06,276 ec2-spot-converter - [STEP 14/26] Terminate instance...
[INFO] 2021-08-29 08:01:06,424 ec2-spot-converter -   => SUCCESS. Successfully terminated instance i-08da23413cc50235a.
[INFO] 2021-08-29 08:01:06,442 ec2-spot-converter - [STEP 15/26] Wait resource release...
[INFO] 2021-08-29 08:01:06,635 ec2-spot-converter - Waiting for instance 'terminated' state...
[INFO] 2021-08-29 08:01:11,686 ec2-spot-converter - Waiting for instance 'terminated' state...
[INFO] 2021-08-29 08:01:18,748 ec2-spot-converter -   => SUCCESS. All resources released : ['eni-0315f33b44b37eade'].
[INFO] 2021-08-29 08:01:18,766 ec2-spot-converter - [STEP 16/26] Create new instance...
Traceback (most recent call last):
  File "./ec2-spot-converter", line 1570, in <module>
    sys.exit(main(sys.argv))
  File "./ec2-spot-converter", line 1536, in main
    return_code, reason, keys = step["Function"]()
  File "./ec2-spot-converter", line 953, in create_new_instance
    response = ec2_client.run_instances(**launch_specifications)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (UnsupportedOperation) when calling the RunInstances operation: The r6g.medium instance type does not support specifying CpuOptions.

Since the original instance is terminated is there a quick way to roll back? Or I will need to re-create the machine from backed up AMI

region not specified error

When trying to run the tool, I get an exception:

> ./ec2-spot-converter --generate-dynamodb-table
Traceback (most recent call last):
  File "./ec2-spot-converter", line 58, in <module>
    ec2_client               = boto3.client("ec2",               config=config)
  File "/usr/lib/python3.6/site-packages/boto3/__init__.py", line 93, in client
    return _get_default_session().client(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/boto3/session.py", line 263, in client
    aws_session_token=aws_session_token, config=config)
  File "/usr/lib/python3.6/site-packages/botocore/session.py", line 838, in create_client
    client_config=config, api_version=api_version)
  File "/usr/lib/python3.6/site-packages/botocore/client.py", line 87, in create_client
    verify, credentials, scoped_config, client_config, endpoint_bridge)
  File "/usr/lib/python3.6/site-packages/botocore/client.py", line 328, in _get_client_args
    verify, credentials, scoped_config, client_config, endpoint_bridge)
  File "/usr/lib/python3.6/site-packages/botocore/args.py", line 73, in get_client_args
    endpoint_url, is_secure, scoped_config)
  File "/usr/lib/python3.6/site-packages/botocore/args.py", line 154, in compute_client_args
    s3_config=s3_config,
  File "/usr/lib/python3.6/site-packages/botocore/args.py", line 220, in _compute_endpoint_config
    return self._resolve_endpoint(**resolve_endpoint_kwargs)
  File "/usr/lib/python3.6/site-packages/botocore/args.py", line 303, in _resolve_endpoint
    service_name, region_name, endpoint_url, is_secure)
  File "/usr/lib/python3.6/site-packages/botocore/client.py", line 402, in resolve
    service_name, region_name)
  File "/usr/lib/python3.6/site-packages/botocore/regions.py", line 134, in construct_endpoint
    partition, service_name, region_name)
  File "/usr/lib/python3.6/site-packages/botocore/regions.py", line 148, in _endpoint_for_partition
    raise NoRegionError()
botocore.exceptions.NoRegionError: You must specify a region.

Need to add to readme an explanation on how to configure it (you can see the options here)

Unable to stop instance

When I run the final command to convert an on demand instance to spot instance, the process is not going beyond step 6.

[INFO] 2021-07-03 01:05:59,943 ec2-spot-converter - [STEP 1/26] Read DynamoDB state table...
[INFO] 2021-07-03 01:05:59,980 ec2-spot-converter -   => SUCCESS. Record 'i-00df331228ea82943' read succesfully.
[INFO] 2021-07-03 01:06:00,104 ec2-spot-converter - [STEP 2/26] Discover instance state... : RECOVERED STATE. SKIPPED!
[INFO] 2021-07-03 01:06:00,104 ec2-spot-converter -   => SUCCESS. Instance is in state running...
[INFO] 2021-07-03 01:06:00,105 ec2-spot-converter - [STEP 3/26] Deregister from ELB target groups... => SKIPPED! Need '--check-targetgroups' argument.
[INFO] 2021-07-03 01:06:00,105 ec2-spot-converter - [STEP 4/26] Wait for drainage of ELB target groups... => SKIPPED! Need '--check-targetgroups' argument.
[INFO] 2021-07-03 01:06:00,105 ec2-spot-converter - [STEP 5/26] Stop the instance... : RECOVERED STATE. SKIPPED!
[INFO] 2021-07-03 01:06:00,105 ec2-spot-converter -   => SUCCESS. Stopping 'i-00df331228ea82943'...
[INFO] 2021-07-03 01:06:00,105 ec2-spot-converter - [STEP 6/26] Wait for expected instance state...
[INFO] 2021-07-03 01:06:00,168 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:06:15,220 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:06:30,276 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:06:45,325 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:07:00,374 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:07:15,438 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:07:30,480 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:07:45,536 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:08:00,587 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:08:15,645 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:08:30,698 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:08:45,756 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:09:00,838 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:09:15,886 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:09:30,951 ec2-spot-converter - Waiting for instance to stop... (current state=running)
[INFO] 2021-07-03 01:09:46,010 ec2-spot-converter - Waiting for instance to stop... (current state=running)

However, the first time I ran this, the instance stopped at step 6, but no more steps were executed (I waited for more than 10 minutes). Now when I am trying it (tried it multiple times), it's getting stuck at step 6 as above.

Any idea why this might be happening? I am new to this, so bear with me if my question sounds noob.

Copying tags for volumes when DeleteOnTermination is false

While using the script for an on-demand to spot conversion I noticed the tags on original volume doesn't get copied to the new volume created.
In my case the DeleteOnTermination was originally false so there was an extra diff for volume,

Impact:
I generally keep a backup schedule based on tags. Since the new volume won't have tag there's a requirement to manually add it, else the backup schedule will use the old volume.

No biggie. But wanted to know your view

jcjorel / ec2-spot-converter Goto Github PK

ec2-spot-converter's People

Contributors

Stargazers

Watchers

Forkers

ec2-spot-converter's Issues

A Way to change VPC/Network/Zone

ELB target group retrieval issue

ec2 target group registration is lost after transition

Failure while creating instance

Usage as a Lambda Function?

Tag Keys starting with aws: error with reserved word

error deregistering image (--delete-ami)

Invalid option error for r6g.medium spot conversion

region not specified error

Unable to stop instance

Copying tags for volumes when DeleteOnTermination is false

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent