cloudposse / terraform-aws-security-group Goto Github PK

Terraform module to create AWS Security Group and rules.

Tip

👽 Use Atmos with Terraform

Cloud Posse uses atmos to easily orchestrate multiple environments using Terraform.
Works with Github Actions, Atlantis, or Spacelift.

Watch demo of using Atmos with Terraform

Example of running atmos to manage infrastructure from our Quick Start tutorial.

This module is primarily for setting security group rules on a security group. You can provide the ID of an existing security group to modify, or, by default, this module will create a new security group and apply the given rules to it.

This module can be used very simply, but it is actually quite complex because it is attempting to handle numerous interrelationships, restrictions, and a few bugs in ways that offer a choice between zero service interruption for updates to a security group not referenced by other security groups (by replacing the security group with a new one) versus brief service interruptions for security groups that must be preserved.

It is desirable to avoid having service interruptions when updating a security group. This is not always possible due to the way Terraform organizes its activities and the fact that AWS will reject an attempt to create a duplicate of an existing security group rule. There is also the issue that while most AWS resources can be associated with and disassociated from security groups at any time, there remain some that may not have their security group association changed, and an attempt to change their security group will cause Terraform to delete and recreate the resource.

Changes to a security group can cause service interruptions in 2 ways:

1. Changing rules may be implemented as deleting existing rules and creating new ones. During the period between deleting the old rules and creating the new rules, the security group will block traffic intended to be allowed by the new rules.
2. Changing rules may alternately be implemented as creating a new security group with the new rules and replacing the existing security group with the new one (then deleting the old one). This usually works with no service interruption in the case where all resources that reference the security group are part of the same Terraform plan. However, if, for example, the security group ID is referenced in a security group rule in a security group that is not part of the same Terraform plan, then AWS will not allow the existing (referenced) security group to be deleted, and even if it did, Terraform would not know to update the rule to reference the new security group.

The key question you need to answer to decide which configuration to use is "will anything break if the security group ID changes". If not, then use the defaults create_before_destroy = true and preserve_security_group_id = false and do not worry about providing "keys" for security group rules. This is the default because it is the easiest and safest solution when the way the security group is being used allows it.

If things will break when the security group ID changes, then set preserve_security_group_id to true. Also read and follow the guidance below about keys and limiting Terraform security group rules to a single AWS security group rule if you want to mitigate against service interruptions caused by rule changes. Note that even in this case, you probably want to keep create_before_destroy = true because otherwise, if some change requires the security group to be replaced, Terraform will likely succeed in deleting all the security group rules but fail to delete the security group itself, leaving the associated resources completely inaccessible. At least with create_before_destroy = true, the new security group will be created and used where Terraform can make the changes, even though the old security group will still fail to be deleted.

The most important option is create_before_destroy which, when set to true (the default), ensures that a new replacement security group is created before an existing one is destroyed. This is particularly important because a security group cannot be destroyed while it is associated with a resource (e.g. a load balancer), but "destroy before create" behavior causes Terraform to try to destroy the security group before disassociating it from associated resources, so plans fail to apply with the error

Error deleting security group: DependencyViolation: resource sg-XXX has a dependent object

With "create before destroy" and any resources dependent on the security group as part of the same Terraform plan, replacement happens successfully:

1. New security group is created
2. Resource is associated with the new security group and disassociated from the old one
3. Old security group is deleted successfully because there is no longer anything associated with it

(If there is a resource dependent on the security group that is also outside the scope of the Terraform plan, the old security group will fail to be deleted and you will have to address the dependency manually.)

Note that the module's default configuration of create_before_destroy = true and preserve_security_group_id = false will force "create before destroy" behavior on the target security group, even if the module did not create it and instead you provided a target_security_group_id.

Unfortunately, just creating the new security group first is not enough to prevent a service interruption. Keep reading.

A security group by itself is just a container for rules. It only functions as desired when all the rules are in place. If using the Terraform default "destroy before create" behavior for rules, even when using create_before_destroy for the security group itself, an outage occurs when updating the rules or security group, because the order of operations is:

1. Delete existing security group rules (triggering a service interruption)
2. Create the new security group
3. Associate the new security group with resources and disassociate the old one (which can take a substantial amount of time for a resource like a NAT Gateway)
4. Create the new security group rules (restoring service)
5. Delete the old security group

To resolve this issue, the module's default configuration of create_before_destroy = true and preserve_security_group_id = false causes any change in the security group rules to trigger the creation of a new security group. With that, a rule change causes operations to occur in this order:

1. Create the new security group
2. Create the new security group rules
3. Associate the new security group with resources and disassociate the old one
4. Delete the old security group rules
5. Delete the old security group

There can be a downside to creating a new security group with every rule change. If you want to prevent the security group ID from changing unless absolutely necessary, perhaps because the associated resource does not allow the security group to be changed or because the ID is referenced somewhere (like in another security group's rules) outside of this Terraform plan, then you need to set preserve_security_group_id to true.

The main drawback of this configuration is that there will normally be a service outage during an update, because existing rules will be deleted before replacement rules are created. Using keys to identify rules can help limit the impact, but even with keys, simply adding a CIDR to the list of allowed CIDRs will cause that entire rule to be deleted and recreated, causing a temporary access denial for all of the CIDRs in the rule. (For more on this and how to mitigate against it, see The Importance of Keys below.)

Also note that setting preserve_security_group_id to true does not prevent Terraform from replacing the security group when modifying it is not an option, such as when its name or description changes. However, if you can control the configuration adequately, you can maintain the security group ID and eliminate impact on other security groups by setting preserve_security_group_id to true. We still recommend leaving create_before_destroy set to true for the times when the security group must be replaced, to avoid the DependencyViolation described above.

We provide a number of different ways to define rules for the security group for a few reasons:

• Terraform type constraints make it difficult to create collections of objects with optional members
• Terraform resource addressing can cause resources that did not actually change to nevertheless be replaced (deleted and recreated), which, in the case of security group rules, then causes a brief service interruption
• Terraform resource addresses must be known at plan time, making it challenging to create rules that depend on resources being created during apply and at the same time are not replaced needlessly when something else changes
• When Terraform rules can be successfully created before being destroyed, there is no service interruption for the resources associated with that security group (unless the security group ID is used in other security group rules outside of the scope of the Terraform plan)

If you are using "create before destroy" behavior for the security group and security group rules, then you can skip this section and much of the discussion about keys in the later sections, because keys do not matter in this configuration. However, if you are using "destroy before create" behavior, then a full understanding of keys as applied to security group rules will help you minimize service interruptions due to changing rules.

When creating a collection of resources, Terraform requires each resource to be identified by a key, so that each resource has a unique "address", and changes to resources are tracked by that key. Every security group rule input to this module accepts optional identifying keys (arbitrary strings) for each rule. If you do not supply keys, then the rules are treated as a list, and the index of the rule in the list will be used as its key. This has the unwelcome behavior that removing a rule from the list will cause all the rules later in the list to be destroyed and recreated. For example, changing [A, B, C, D] to [A, C, D] causes rules 1(B), 2(C), and 3(D) to be deleted and new rules 1(C) and 2(D) to be created.

To mitigate against this problem, we allow you to specify keys (arbitrary strings) for each rule. (Exactly how you specify the key is explained in the next sections.) Going back to our example, if the initial set of rules were specified with keys, e.g. [{A: A}, {B: B}, {C: C}, {D: D}], then removing B from the list would only cause B to be deleted, leaving C and D intact.

Note, however, two cautions. First, the keys must be known at terraform plan time and therefore cannot depend on resources that will be created during apply. Second, in order to be helpful, the keys must remain consistently attached to the same rules. For example, if you did

rule_map = { for i, v in rule_list : i => v }

then you will have merely recreated the initial problem with using a plain list. If you cannot attach meaningful keys to the rules, there is no advantage to specifying keys at all.

A single security group rule input can actually specify multiple AWS security group rules. For example, ipv6_cidr_blocks takes a list of CIDRs. However, AWS security group rules do not allow for a list of CIDRs, so the AWS Terraform provider converts that list of CIDRs into a list of AWS security group rules, one for each CIDR. (This is the underlying cause of several AWS Terraform provider bugs, such as #25173.) As of this writing, any change to any element of such a rule will cause all the AWS rules specified by the Terraform rule to be deleted and recreated, causing the same kind of service interruption we sought to avoid by providing keys for the rules, or, when create_before_destroy = true, causing a complete failure as Terraform tries to create duplicate rules which AWS rejects. To guard against this issue, when not using the default behavior, you should avoid the convenience of specifying multiple AWS rules in a single Terraform rule and instead create a separate Terraform rule for each source or destination specification.

This module provides 3 ways to set security group rules. You can use any or all of them at the same time.

The easy way to specify rules is via the rules input. It takes a list of rules. (We will define a rule a bit later.) The problem is that a Terraform list must be composed of elements that are all the exact same type, and rules can be any of several different Terraform types. So to get around this restriction, the second way to specify rules is via the rules_map input, which is more complex.

Error: Inconsistent conditional result types
The true and false result expressions must have consistent types. The given
expressions are object and object, respectively.

cidr_rule = {
  type        = "ingress"
  cidr_blocks = ["0.0.0.0/0"]
}

self_rule = {
  type        = "ingress"
  self        = true
}

rules = tolist([local.cidr_rule, local.self_rule])

Invalid value for "v" parameter: cannot convert tuple to list of any single type.

rules = tolist([{
  type        = "ingress"
  cidr_blocks = ["0.0.0.0/0"]
  self        = null
},
{
  type        = "ingress"
  cidr_blocks = []
  self        = true
}])

{ mine = local.my_rules, theirs = var.their_rules }

The rules_map input takes an object.

• The attribute names (keys) of the object can be anything you want, but need to be known during terraform plan, which means they cannot depend on any resources created or changed by Terraform.
• The values of the attributes are lists of rule objects, each object representing one Security Group Rule. As explained above in "Why the input is so complex", each object in the list must be exactly the same type. To use multiple types, you must put them in separate lists and put the lists in a map with distinct keys.

Example:

rules_map = {
  ingress = [{
    key         = "ingress"
    type        = "ingress"
    from_port   = 0
    to_port     = 2222
    protocol    = "tcp"
    cidr_blocks = module.subnets.nat_gateway_public_ips
    self        = null
    description = "2222"
  }],
  egress = [{
    key         = "egress"
    type        = "egress"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
    self        = null
    description = "All output traffic"
  }]
}

For this module, a rule is defined as an object.

• The attributes and values of the rule objects are fully compatible (have the same keys and accept the same values) as the Terraform aws_security_group_rule resource, except:
  • The security_group_id will be ignored, if present
  • You can include an optional key attribute. If present, its value must be unique among all security group rules in the security group, and it must be known in the Terraform "plan" phase, meaning it cannot depend on anything being generated or created by Terraform.

The key attribute value, if provided, will be used to identify the Security Group Rule to Terraform in order to prevent Terraform from modifying it unnecessarily. If the key is not provided, Terraform will assign an identifier based on the rule's position in its list, which can cause a ripple effect of rules being deleted and recreated if a rule gets deleted from start of a list, causing all the other rules to shift position. See "Unexpected changes..." below for more details.

The other way to set rules is via the rule_matrix input. This splits the attributes of the aws_security_group_rule resource into two sets: one set defines the rule and description, the other set defines the subjects of the rule. Again, optional "key" values can provide stability, but cannot contain derived values. This input is an attempt at convenience, and should not be used unless you are using the default settings of create_before_destroy = true and preserve_security_group_id = false, or else a number of failure modes or service interruptions are possible: use rules_map instead.

As with rules and explained above in "Why the input is so complex", all elements of the list must be the exact same type. This also holds for all the elements of the rules_matrix.rules list. Because rule_matrix is already so complex, we do not provide the ability to mix types by packing object within more objects. All of the elements of the rule_matrix list must be exactly the same type. You can make them all the same type by following a few rules:

• Every object in a list must have the exact same set of attributes. Most attributes are optional and can be omitted, but any attribute appearing in one object must appear in all the objects.
• Any attribute that takes a list value in any object must contain a list in all objects. Use an empty list rather than null to indicate "no value". Passing in null instead of a list may cause Terraform to crash or emit confusing error messages (e.g. "number is required").
• Any attribute that takes a value of type other than list can be set to null in objects where no value is needed.

The schema for rule_matrix is:

{
  # these top level lists define all the subjects to which rule_matrix rules will be applied
  key                       = an optional unique key to keep these rules from being affected when other rules change
  source_security_group_ids = list of source security group IDs to apply all rules to
  cidr_blocks               = list of ipv4 CIDR blocks to apply all rules to
  ipv6_cidr_blocks          = list of ipv6 CIDR blocks to apply all rules to
  prefix_list_ids           = list of prefix list IDs to apply all rules to

  self = boolean value; set it to "true" to apply the rules to the created or existing security group, null otherwise

  # each rule in the rules list will be applied to every subject defined above
  rules = [{
    key       = an optional unique key to keep this rule from being affected when other rules change
    type      = type of rule, either "ingress" or "egress"
    from_port = start range of protocol port
    to_port   = end range of protocol port, max is 65535
    protocol  = IP protocol name or number, or "-1" for all protocols and ports

    description = free form text description of the rule
  }]
}

When configuring this module for "create before destroy" behavior, any change to a security group rule will cause an entire new security group to be created with all new rules. This can make a small change look like a big one, but is intentional and should not cause concern.

As explained above under The Importance of Keys, when using "destroy before create" behavior, security group rules without keys are identified by their indices in the input lists. If a rule is deleted and the other rules therefore move closer to the start of the list, those rules will be deleted and recreated. This can make a small change look like a big one when viewing the output of Terraform plan, and will likely cause a brief (seconds) service interruption.

You can avoid this for the most part by providing the optional keys, and limiting each rule to a single source or destination. Rules with keys will not be changed if their keys do not change and the rules themselves do not change, except in the case of rule_matrix, where the rules are still dependent on the order of the security groups in source_security_group_ids. You can avoid this by using rules or rules_map instead of rule_matrix when you have more than one security group in the list. You cannot avoid this by sorting the source_security_group_ids, because that leads to the "Invalid for_each argument" error because of terraform#31035.

You can supply a number of rules as inputs to this module, and they (usually) get transformed into aws_security_group_rule resources. However, Terraform works in 2 steps: a plan step where it calculates the changes to be made, and an apply step where it makes the changes. This is so you can review and approve the plan before changing anything. One big limitation of this approach is that it requires that Terraform be able to count the number of resources to create without the benefit of any data generated during the apply phase. So if you try to generate a rule based on something you are creating at the same time, you can get an error like

Error: Invalid for_each argument
The "for_each" value depends on resource attributes that cannot be determined until apply,
so Terraform cannot predict how many instances will be created.

This module uses lists to minimize the chance of that happening, as all it needs to know is the length of the list, not the values in it, but this error still can happen for subtle reasons. Most commonly, using a function like compact on a list will cause the length to become unknown (since the values have to be checked and nulls removed). In the case of source_security_group_ids, just sorting the list using sort will cause this error. (See terraform#31035.) If you run into this error, check for functions like compact somewhere in the chain that produces the list and remove them if you find them.

Setting inline_rules_enabled is not recommended and NOT SUPPORTED: Any issues arising from setting inlne_rules_enabled = true (including issues about setting it to false after setting it to true) will not be addressed, because they flow from fundamental problems with the underlying aws_security_group resource. The setting is provided for people who know and accept the limitations and trade-offs and want to use it anyway. The main advantage is that when using inline rules, Terraform will perform "drift detection" and attempt to remove any rules it finds in place but not specified inline. See this post for a discussion of the difference between inline and resource rules, and some of the reasons inline rules are not satisfactory.

KNOWN ISSUE (#20046): If you set inline_rules_enabled = true, you cannot later set it to false. If you try, Terraform will complain and fail. You will either have to delete and recreate the security group or manually delete all the security group rules via the AWS console or CLI before applying inline_rules_enabled = false.

Objects not of the same type: Any time you provide a list of objects, Terraform requires that all objects in the list must be the exact same type. This means that all objects in the list have exactly the same set of attributes and that each attribute has the same type of value in every object. So while some attributes are optional for this module, if you include an attribute in any one of the objects in a list, then you have to include that same attribute in all of them. In rules where the key would othewise be omitted, include the key with value of null, unless the value is a list type, in which case set the value to [] (an empty list), due to #28137.

See examples/complete/main.tf for even more examples.

module "label" {
  source = "cloudposse/label/null"
  # Cloud Posse recommends pinning every module to a specific version
  # version = "x.x.x"
  namespace  = "eg"
  stage      = "prod"
  name       = "bastion"
  attributes = ["public"]
  delimiter  = "-"

  tags = {
    "BusinessUnit" = "XYZ",
    "Snapshot"     = "true"
  }
}

module "vpc" {
  source = "cloudposse/vpc/aws"
  # Cloud Posse recommends pinning every module to a specific version
  # version = "x.x.x"
  cidr_block = "10.0.0.0/16"

  context = module.label.context
}

module "sg" {
  source = "cloudposse/security-group/aws"
  # Cloud Posse recommends pinning every module to a specific version
  # version = "x.x.x"

  # Security Group names must be unique within a VPC.
  # This module follows Cloud Posse naming conventions and generates the name
  # based on the inputs to the null-label module, which means you cannot
  # reuse the label as-is for more than one security group in the VPC.
  #
  # Here we add an attribute to give the security group a unique name.
  attributes = ["primary"]

  # Allow unlimited egress
  allow_all_egress = true

  rules = [
    {
      key         = "ssh"
      type        = "ingress"
      from_port   = 22
      to_port     = 22
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      self        = null  # preferable to self = false
      description = "Allow SSH from anywhere"
    },
    {
      key         = "HTTP"
      type        = "ingress"
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = []
      self        = true
      description = "Allow HTTP from inside the security group"
    }
  ]

  vpc_id  = module.vpc.vpc_id

  context = module.label.context
}

module "sg_mysql" {
  source = "cloudposse/security-group/aws"
  # Cloud Posse recommends pinning every module to a specific version
  # version = "x.x.x"

  # Add an attribute to give the Security Group a unique name
  attributes = ["mysql"]

  # Allow unlimited egress
  allow_all_egress = true

  rule_matrix =[
    # Allow any of these security groups or the specified prefixes to access MySQL
    {
      source_security_group_ids = [var.dev_sg, var.uat_sg, var.staging_sg]
      prefix_list_ids = [var.mysql_client_prefix_list_id]
      rules = [
        {
          key         = "mysql"
          type        = "ingress"
          from_port   = 3306
          to_port     = 3306
          protocol    = "tcp"
          description = "Allow MySQL access from trusted security groups"
        }
      ]
    }
  ]

  vpc_id  = module.vpc.vpc_id

  context = module.label.context
}

Available targets:

  help                                Help screen
  help/all                            Display help for all targets
  help/short                          This help short screen
  lint                                Lint terraform code

{
  "additional_tag_map": {},
  "attributes": [],
  "delimiter": null,
  "descriptor_formats": {},
  "enabled": true,
  "environment": null,
  "id_length_limit": null,
  "label_key_case": null,
  "label_order": [],
  "label_value_case": null,
  "labels_as_tags": [
    "unset"
  ],
  "name": null,
  "namespace": null,
  "regex_replace_chars": null,
  "stage": null,
  "tags": {},
  "tenant": null
}

[
  "default"
]

Check out these related projects.

• terraform-null-label - Terraform module designed to generate consistent names and tags for resources. Use terraform-null-label to implement a strict naming convention.

For additional context, refer to some of these links.

• terraform-provider-aws - Terraform AWS provider

This project is under active development, and we encourage contributions from our community.

Many thanks to our outstanding contributors:

For 🐛 bug reports & feature requests, please use the issue tracker.

In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow.

1. Review our Code of Conduct and Contributor Guidelines.
2. Fork the repo on GitHub
3. Clone the project to your own machine
4. Commit changes to your own branch
5. Push your work back up to your fork
6. Submit a Pull Request so that we can review your changes

NOTE: Be sure to merge the latest changes from "upstream" before making a pull request!

Join our Open Source Community on Slack. It's FREE for everyone! Our "SweetOps" community is where you get to talk with others who share a similar vision for how to rollout and manage infrastructure. This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build totally sweet infrastructure.

Sign up for our newsletter and join 3,000+ DevOps engineers, CTOs, and founders who get insider access to the latest DevOps trends, so you can always stay in the know. Dropped straight into your Inbox every week — and usually a 5-minute read.

Join us every Wednesday via Zoom for your weekly dose of insider DevOps trends, AWS news and Terraform insights, all sourced from our SweetOps community, plus a live Q&A that you can’t find anywhere else. It's FREE for everyone!

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.

All other trademarks referenced herein are the property of their respective owners.

Copyright © 2021-2024 Cloud Posse, LLC