• https://gist.github.com/rehborn/11699aa2e29abedd6f7fa6197346f22f

Looks like the cidr ranges have grown and once again reaches the limit of security group. This is what I get now running the example code:

Response:
{
  "errorMessage": "An error occurred (RulesPerSecurityGroupLimitExceeded) when calling the AuthorizeSecurityGroupIngress operation: The maximum number of rules per security group has been reached.",
  "errorType": "ClientError",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 42, in lambda_handler\n    result = update_security_groups(ip_ranges)\n",
    "  File \"/var/task/lambda_function.py\", line 94, in update_security_groups\n    if update_security_group(client, group, new_ranges[\"GLOBAL\"], INGRESS_PORTS['Https']):\n",
    "  File \"/var/task/lambda_function.py\", line 136, in update_security_group\n    added += add_permissions(client, group, permission, to_add)\n",
    "  File \"/var/task/lambda_function.py\", line 170, in add_permissions\n    client.authorize_security_group_ingress(GroupId=group['GroupId'], IpPermissions=[add_params])\n",
    "  File \"/var/runtime/botocore/client.py\", line 316, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n",
    "  File \"/var/runtime/botocore/client.py\", line 626, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"
  ]
}

Request ID:
"4d2b63d4-4f60-4ea7-93aa-bf835a7ef1d4"

Function Logs:
nge: 52.66.194.128/26
Found CLOUDFRONT region: ap-southeast-1 range: 13.228.69.0/24
Found CLOUDFRONT region: us-east-2 range: 18.216.170.128/25
Found CLOUDFRONT region: us-east-1 range: 3.231.2.0/25
Found CLOUDFRONT region: ap-southeast-1 range: 52.220.191.0/26
Found CLOUDFRONT region: us-east-1 range: 34.232.163.208/29
Found CLOUDFRONT region: us-west-2 range: 35.162.63.192/26
Found CLOUDFRONT region: us-west-2 range: 34.223.80.192/26
Found CLOUDFRONT region: us-east-1 range: 34.226.14.0/24
Found CLOUDFRONT region: ap-northeast-1 range: 13.113.203.0/24
Found CLOUDFRONT region: ca-central-1 range: 99.79.168.0/23
Found CLOUDFRONT region: us-east-1 range: 34.195.252.0/24
Found CLOUDFRONT region: us-west-1 range: 52.52.191.128/26
Found CLOUDFRONT region: eu-west-2 range: 52.56.127.0/25
Found CLOUDFRONT region: us-west-2 range: 34.216.51.0/25
Found CLOUDFRONT region: ap-northeast-1 range: 52.199.127.192/26
Found CLOUDFRONT region: eu-west-1 range: 52.212.248.0/26
Found CLOUDFRONT region: ap-southeast-2 range: 13.210.67.128/26
Found CLOUDFRONT region: eu-central-1 range: 35.158.136.0/24
Found CLOUDFRONT region: eu-central-1 range: 52.57.254.0/24
Found CLOUDFRONT region: ap-northeast-2 range: 52.78.247.128/26
Found CLOUDFRONT region: eu-west-3 range: 52.47.139.0/24
Found 0 CloudFront_g HttpSecurityGroups to update
Found 1 CloudFront_g HttpsSecurityGroups to update
Found 0 CloudFront_r HttpSecurityGroups to update
Found 1 CloudFront_r HttpsSecurityGroups to update
sg-08c92bbebac0b0caf: Adding 120.52.22.96/27:443
sg-08c92bbebac0b0caf: Adding 180.163.57.128/26:443
sg-08c92bbebac0b0caf: Adding 120.253.240.192/26:443
sg-08c92bbebac0b0caf: Adding 116.129.226.128/26:443
sg-08c92bbebac0b0caf: Adding 223.71.71.128/25:443
sg-08c92bbebac0b0caf: Adding 120.253.245.128/26:443
sg-08c92bbebac0b0caf: Adding 210.51.40.0/24:443
sg-08c92bbebac0b0caf: Adding 58.254.138.0/25:443
sg-08c92bbebac0b0caf: Adding 116.129.226.0/25:443
sg-08c92bbebac0b0caf: Adding 120.52.39.128/27:443
sg-08c92bbebac0b0caf: Adding 118.193.97.64/26:443
sg-08c92bbebac0b0caf: Adding 223.71.71.96/27:443
sg-08c92bbebac0b0caf: Adding 180.163.57.0/25:443
sg-08c92bbebac0b0caf: Adding 223.71.11.0/27:443
sg-08c92bbebac0b0caf: Adding 36.103.232.128/26:443
sg-08c92bbebac0b0caf: Adding 111.51.66.0/24:443
sg-08c92bbebac0b0caf: Adding 120.52.153.192/26:443
sg-08c92bbebac0b0caf: Adding 119.147.182.0/25:443
sg-08c92bbebac0b0caf: Adding 120.232.236.0/25:443
sg-08c92bbebac0b0caf: Adding 58.254.138.128/26:443
sg-08c92bbebac0b0caf: Adding 120.253.245.192/27:443
sg-08c92bbebac0b0caf: Adding 120.52.12.64/26:443
sg-08c92bbebac0b0caf: Adding 36.103.232.0/25:443
sg-08c92bbebac0b0caf: Adding 119.147.182.128/26:443
sg-08c92bbebac0b0caf: Adding 118.193.97.128/25:443
sg-08c92bbebac0b0caf: Adding 120.232.236.128/26:443
sg-08c92bbebac0b0caf: Adding 120.253.241.160/27:443
[ERROR] ClientError: An error occurred (RulesPerSecurityGroupLimitExceeded) when calling the AuthorizeSecurityGroupIngress operation: The maximum number of rules per security group has been reached.
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 42, in lambda_handler
    result = update_security_groups(ip_ranges)
  File "/var/task/lambda_function.py", line 94, in update_security_groups
    if update_security_group(client, group, new_ranges["GLOBAL"], INGRESS_PORTS['Https']):
  File "/var/task/lambda_function.py", line 136, in update_security_group
    added += add_permissions(client, group, permission, to_add)
  File "/var/task/lambda_function.py", line 170, in add_permissions
    client.authorize_security_group_ingress(GroupId=group['GroupId'], IpPermissions=[add_params])
  File "/var/runtime/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/var/runtime/botocore/client.py", line 626, in _make_api_call
    raise error_class(parsed_response, operation_name)
END RequestId: 4d2b63d4-4f60-4ea7-93aa-bf835a7ef1d4
REPORT RequestId: 4d2b63d4-4f60-4ea7-93aa-bf835a7ef1d4	Duration: 909.64 ms	Billed Duration: 1000 ms	Memory Size: 128 MB	Max Memory Used: 83 MB	

This is truly an edge case bug, if it can be called that, in the function update_security_group.

If there are are IpPermissions present in the security group the code will not update the rules if INGRESS_PORTS changes or someone has manually deleted all rules for one value in INGRESS_PORTS.

To reproduce:

1. Set INGRESS_PORTS = [ 80 ]
2. Run the code
3. Set INGRESS_PORTS = [ 80, 443 ]
4. Run the code

Expected behaviour:
IpPermissions has been updated to include port 80 and 443

Actual behaviour:
Only rules for port 80 are present

I recognize that this is an edge case, and that this might not be fixed. In that case this issue will just serve as documentation of this behaviour.

from Line 118 - 121 you reference the rangeToUpdate by index instead of looping through. It's possible to have security groups setup in multiple VPCs

Hi there. Ive used this code in the past and it has worked great. ive tried again more recently and have come across and issue with the urllib modules. when running the test event, i get the following error message:

Response:
{
"errorMessage": "Unable to import module 'lambda_function'"
}
Request ID:
"xx"
Function Logs:
START RequestId: xx Version: $LATEST
Unable to import module 'lambda_function': No module named request
END RequestId: xx
REPORT RequestId: xx Duration: 0.47 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 60 MB Init Duration: 223.79 ms

"No module named request" suggests that attribute of module urllib no longer exists. if you import the module without the attribute you then receive and error saying "attribute 'request' doesnt exist".. i have tried urllib2.

Am i doing something wrong here?

The Python language governing body – Python Software Foundation (PSF) – will end support for Python 2.7 on January 1, 2020. Amazon is encouraging users to upgrade to Python 3 as a result of this.

Can this sample be upgraded to use Python 3 soon please?

First, thank you for this very helpful lambda!

Second, just wanted to share this as just a warning bell for the future. At the time of this posting, https://ip-ranges.amazonaws.com/ip-ranges.json includes 34 39 distinct "CLOUDFRONT" i.p. ranges.
However, AWS only permits up to 40 i.p. ranges to be added to a single security group.

If a time comes when AWS publishes > 40 i.p. ranges for CloudFront, then this lambda will fail with something like this:

sg-abcd1234: Adding 1.2.255.128/26:80

sg-abcd1234: Adding 1.3.255.128/26:80

An error occurred (RulesPerSecurityGroupLimitExceeded) when calling the AuthorizeSecurityGroupIngress operation: The maximum number of rules per security group has been reached.: ClientError

If that happens, we may need to not just create 2 security groups for http and https, but a few more, in order to distribute the growing list.

I don't know the rate at which the ip ranges tend to be added to CloudFront, so it's possible this may not be an eminent threat whatsoever.

https://aws.amazon.com/blogs/security/how-to-automatically-update-your-security-groups-for-amazon-cloudfront-and-aws-waf-by-using-aws-lambda/

The example (located in this repo) is based off of python2, although python2 will be EOF soon.

Pull request #18 provides a python3 example, by simply running 2to3 on the file (https://docs.python.org/2/library/2to3.html#to3-reference)

Tested to run in a Lambda Python 3.7 runtime env.

This solution no longer works. I've reached the max rules per security group and now I'm getting weird issues.

As seen in https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html#subscribe-notifications the SNS subscription should always be done in the us-east-1 zone, thus the --region us-east-1 parameter specification should be added to the subscription command in README

The script to update security groups with CloudFront IPs is returning

"errorType": "ClientError",
"errorMessage": "An error occurred (RulesPerSecurityGroupLimitExceeded) when calling the AuthorizeSecurityGroupIngress operation: The maximum number of rules per security group has been reached."

Presumably because the number of IP addresses from CloudFront now exceeds the the default AWS limit on RulesPerSecurityGroup, which is 60.

I think the README should mention this issue with instructions on how to increase VPC RulesPerSecurityGroup.

START RequestId: dbb585bb-3a54-4d1a-a690-54e5085c55f8 Version: $LATEST
[INFO] 2020-02-14T18:19:39.441Z dbb585bb-3a54-4d1a-a690-54e5085c55f8 Received SNS event: 95df01b4-ee98-5cb9-9903-4c221d41eb5e
[INFO] 2020-02-14T18:19:39.441Z dbb585bb-3a54-4d1a-a690-54e5085c55f8 Updating from https://ip-ranges.amazonaws.com/ip-ranges.json
END RequestId: dbb585bb-3a54-4d1a-a690-54e5085c55f8
REPORT RequestId: dbb585bb-3a54-4d1a-a690-54e5085c55f8 Duration: 300099.78 ms Billed Duration: 300000 ms Memory Size: 128 MB Max Memory Used: 28 MB Init Duration: 170.57 ms
2020-02-14T18:24:39.539Z dbb585bb-3a54-4d1a-a690-54e5085c55f8 Task timed out after 300.10 seconds

{
"errorMessage": "Unable to import module 'lambda_function'"
}

I updated as below

#import urllib.request, urllib.error, urllib.parse
import urllib3

line 68 as
response = urllib3.request.urlopen(url)

but still error out as below.

{
"stackTrace": [
[
"/var/task/lambda_function.py",
51,
"lambda_handler",
"ip_ranges = json.loads(get_ip_groups_json(message['url'], message['md5']))"
],
[
"/var/task/lambda_function.py",
68,
"get_ip_groups_json",
"response = urllib3.request.urlopen(url)"
]
],
"errorType": "AttributeError",
"errorMessage": "'module' object has no attribute 'urlopen'"
}

Requiring your customers to do things in Lambda instead of providing a descent way to configure such things in tools such as terraform is a HUGE design bug.
Sorry, this repository should not exist.

From latest ip-ranges.json, CloudFront staring to have IPv6 address appear under ipv6_prefixes section as below

{
  "ipv6_prefix": "2600:9000::/28",
  "region": "GLOBAL",
  "service": "CLOUDFRONT"
}

it's time to add ipv6_prefixes range also

Hi,

regarding this blog post here "https://aws.amazon.com/about-aws/whats-new/2022/02/amazon-cloudfront-managed-prefix-list/?nc1=h_ls" it is possible get the "same result", but as "AWS-managed prefix lists".

But I do not understand the difference between the services "CLOUDFRONT_ORIGIN_FACING" and "CLOUDFRONT". Because they use the IP ranges with service "CLOUDFRONT_ORIGIN_FACING" and your solution is using service "CLOUDFRONT".

Doesn't that matter?

Just tested this and it looks to me like line 28:
'AutoUpdate': os.environ.get('TagAutoUpdate', 'AutoUpdateTag'),

Needs changing to:
'AutoUpdate': os.environ.get('TagAutoUpdate', 'AutoUpdate'),

As then it works as per:
https://aws.amazon.com/blogs/security/how-to-automatically-update-your-security-groups-for-amazon-cloudfront-and-aws-waf-by-using-aws-lambda/

....apart from the fact that the article above mentions Python 2.7, and not 3.7, which is required.

The example showing how to get a list of services is incorrect.

I'm putting together a repository of open source Lambda functions, and since I use this function, I was hoping to include it. I'm a little confused about the licensing on it through, because the LICENSE.txt says Apache 2.0, the NOTICE.txt says All Rights Reserved, and the update_security_groups_lambda/update_security_groups.py says both.

jq -r '.prefixes[] | select(.service=="CLOUDFRONT") | .ip_prefix' < ip-ranges.json |wc -l

returns 67 IP ranges

The rule limit per security group is 50. This sample needs updated to make use of multiple security groups.

I will work on a pull request, just wanted to make sure you knew.