Follow up for #23

I think that remote state functionality should be supported as well. That would enable the usage in more complex environments (e.g. thinking of centrally managed VPCs)

What should be the default for handling multiple stacks within an app?

1. All in the same output directory (well, why have multiple stacks then?)
2. A dedicated folder per stack, totally separated in terms of state
3. A dedicated folder per stack, integrated as modules - monolithic state

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

For target languages other than nodejs, a Homebrew package would be desirable - see https://formulae.brew.sh/formula/cdk8s for an example.

As a follow up for #31, there are a handful of unhandled types, e.g.

      The attribute {"storageName":"_autoscalingGroups","_name":"autoscalingGroups","type":{"_type":"ComputedComplexNestedResourcesAutoscalingGroups","isList":true,"isMap":false,"isComputed":true,"isOptional":false,"level":3,"struct":{"name":"ComputedComplexNestedResourcesAutoscalingGroups","attributes":[{"storageName":"_name","_name":"name","type":{"_type":"string","isList":false,"isMap":false,"isComputed":true,"isOptional":false,"level":4},"optional":false,"computed":true,"terraformName":"name","terraformFullName":"[object Object]_[object Object]_[object Object]_name"}],"isClass":true,"isAnonymous":true}},"optional":false,"computed":true,"terraformName":"autoscaling_groups","terraformFullName":"[object Object]_[object Object]_autoscaling_groups"} isn't implemented yet

These are:

• deeply nested computed complex types in the config interface
• nested computed complex type in computed complex types

The sscaff is performing a mkdir(targetDir, { recursive: true}) in order to make sure the targetDir exists. The recursive option was introduced in node 10.12. When using an older node version, the init command fails.

We should make sure, the user is using at least node 10.12 and raise an error if the user runs something older.

Specifying it in the package.json could be one option

New package will be called cdktf.

The process of writing the providers.tf.json file is a bit brittle.

• The Terraform version constraint is written on cdktf get
• The actual provider config is merged on cdktf synth

This breaks, when a user removes the synthesize target folder (e.g. dist). On the next synth, the provider config is still written, but the version constraints are lost.

On the other hand, when doing another cdktf get, the providers.tf.json is overwritten and leaves the dist folder in an invalid state. In order to run a cdktf get successfully, the dist folder has to be deleted.

Example

{
  "//": "generated from `cdktf get`",
  "terraform": {
    "required_providers": {
      "aws": "~> 2.0"
    }
  },
  "//": "generated from `cdktf synth`",
  "provider": {
    "aws": [
      {
        "region": "eu-central-1",
        "ignore_tags": [
          {
            "keys": [
              "foo"
            ]
          }
        ]
      }
    ]
  }
}

Possible Solution 1:

• Perform cdktf get in a temporary folder
• Write some kind of lock file for exact provider and module versions in project root directory
• Generate version constraints on synth from that lock file.

This would make sure, that the provider / module classes are always matching the source they were generated from. An upgrade would be an explicit run of cdktf get, which would regenerated the lock file.

Example

Given the following cdktf.json file:

{
  "language": "python",
  "app": "pipenv run ./main.py",
  "terraformProviders": ["aws@~> 2.0"],
  "output": "imports"
}

This could generated the following cdktf.lock file

{
  "providers": {
    "aws": {
      "version": "2.59.0"
    }
  },
  "modules": {}
}

Possible Solution 2:

Go with the approach described in #43 and make each resource (including providers) aware of the version they were generated from. Use this version information to generate the correct Terraform provider version constraints.

References

This is loosely related to #43

The resource Kinesis Analytics Application has nested computed attributes for block types which aren't listed on the documentation. I think it makes sense to ignore those attributes. Thoughts?

      "inputs": {
        "nesting_mode": "list",
        "block": {
          "attributes": {
            "id": {
              "type": "string",
              "computed": true
            },
            "name_prefix": {
              "type": "string",
              "required": true
            },
            "starting_position_configuration": {
              "type": [
                "list",
                [
                  "object",
                  {
                    "starting_position": "string"
                  }
                ]
              ],
              "computed": true
            },
            "stream_names": {
              "type": [
                "set",
                "string"
              ],
              "computed": true
            }
          }

The Go Schema defines it as expected:

						"starting_position_configuration": {
							Type:     schema.TypeList,
							Computed: true,
							Elem: &schema.Resource{
								Schema: map[string]*schema.Schema{
									"starting_position": {
										Type:     schema.TypeString,
										Computed: true,
									},
								},
							},
						},
						"stream_names": {
							Type:     schema.TypeSet,
							Computed: true,
							Elem:     &schema.Schema{Type: schema.TypeString},
						},

To ensure that our examples are always in a valid, deployable state, it'd be very helpful to have full end2end tests for our examples.

Each example is:

• Build
• Synthesized
• Planned / Applied
• Destroyed

For this VPC module, there's a discrepancy for the actual implementation of the inputs on Github and the API response of the Terraform registry.

Terraform Registry response:

{
  "id": "terraform-aws-modules/vpc/aws/2.33.0",
  "owner": "antonbabenko",
  "namespace": "terraform-aws-modules",
  "name": "vpc",
  "version": "2.33.0",
  "provider": "aws",
  "...": "...",
  "inputs": [
    {
      "...": "..."
    },
    {
      "name": "redshift_subnet_assign_ipv6_address_on_creation",
      "type": "bool",
      "description": "Assign IPv6 address on redshift subnet, must be disabled to change IPv6 CIDRs. This is the IPv6 equivalent of map_public_ip_on_launch",
      "default": "",
      "required": true
    },
    {
      "...": "..."
    }
  ]
}

Actual implementation

variable "redshift_subnet_assign_ipv6_address_on_creation" {
  description = "Assign IPv6 address on redshift subnet, must be disabled to change IPv6 CIDRs. This is the IPv6 equivalent of map_public_ip_on_launch"
  type        = bool
  default     = null
}

According to this blog post, the input should be optional when the default is set to null. Likely a bug in the Terraform Registry?

This affects all inputs where the default is null (12 in this module). The effect is, that those variables are rendered as required attributes, and the Typescript compiler expects them to be set.

I wonder if it's safe to assume that a default of "" (empty string) makes a variable always optional and we could ignore the required flag in those cases. Thoughts?

At the moment, the module generator generates accessors and interfaces for inputs. We should extend it to provide getters for outputs as well.

We could make use of the comment property to encode metadata into the generated Terraform JSON.

On Stack level:

• Stack Name
• cdktf Version

On Resource level:

• cdk path

With the current implementation of computed Lists and Maps we can access single elements (see #31 / #12). What we can't do right now:

• Iterate programatically
• Pass a computed List / Map as input to another Resource / Module

I found a bug where if an application is setting the output directory using new App({ outdir: 'cdk.out' }); it is not being respected by the synth command as it uses the output directory set as the default in config.ts. I am seeing an error:

ERROR: synthesis failed, app expected to create "dist"

Since we're restructuring this repo into a monorepo, we'll have to add tooling to support this. In particular:

• Lerna
• Yarn Workspaces

Besides from that, we should look into the custom tooling of cdk8s

There are optional computed complex types in the provider schema, e.g. in the aws_security_group the attributes egress / ingress. Here an excerpt from the schema:

             "egress": {
                "type": [
                  "set",
                  [
                    "object",
                    {
                      // ... attributes 
                    }
                  ]
                ],
                "optional": true,
                "computed": true
              },

As I understand optional && computed, this means there could be egress or ingress rules created by AWS. Which, at least for egress is true. However, apparently the AWS Terraform provider removes this computed object (see comment)

At the moment the attribute is rendered without taking into account the computed option, which looks like this:

  // egress
  private _egress?: SecurityGroupEgress[];
  public get egress() {
    return this._egress; // Getting the computed value is not yet implemented
  }
  public set egress(value: SecurityGroupEgress[]) {
    this._egress = value;
  }

We should probably enable access to the those computed rules as well. Something along the lines of this:

  // egress
  private _egress?: SecurityGroupEgress[];
  public get egress(index?: string) {
    return this._egress || return new SecurityGroupEgressList(this, 'egress', index);
  }
  public set egress(value: SecurityGroupEgress[] | undefined) {
    this._egress = value;
  }

We'd have to generate:

• an interface (e.g. SecurityGroupEgress) for the optional type
• a class (e.g. SecurityGroupEgressList) which implements the interface to wrap the computed value
• Make sure the return type of the getter function is intuitively usable

In particular the last point seems to be difficult.

For now it's probably fine as is, but when heading towards beta we should come back to this.

We're using the jsii/superchain to run our builds via github actions. In order to make those builds work, we need Terraform (and jq to install it) as dependencies. They're installed during the build run at the moment. It would be better to have a prebuilt Docker image.

Follow up for #31

Version 3 of the constructs package got released two days ago. One of the breaking changes was the removal of the Token class, which is used in this project (within TerraformResource).

We'll have to come up with a solution, if we don't want to be stuck with v2 forever. Most pragmatic would be, to just inline the removed parts into this project directly.

@eladb could you provide a bit more context about the topic?

In #8, I discovered naming collisions between a few modules (essentially, two different resources are generating identical names for interfaces / classes). That's fine, as along it's scoped to the resource's module. However, when generating an index.ts file (e.g. for the AWS Provider), those duplicated names are conflicting.

Why an index.ts?

In contrast to cdk8s, we're dealing with many files representing a package (e.g. the AWS provider). It'd be good to have a main entrypoint into the package from the user point of view. Plus, when we're going to build our packages with jsii, we'll have to have this main entrypoint anyway.

Proposal

I'd suggest to go with namespaced exports, in a similar way as it's described here. This should solve the naming conflicts, and yield a better user experience when importing resource classes / interfaces.

I'd love to see cross references between Terraform Stacks work seamlessly and in an unsurprising way for the user.

The AWS CDK has something similar in place

import { Construct } from 'constructs';
import { App, TerraformStack } from 'cdktf';
import { S3Bucket } from './.gen/providers/aws'

class MyStack extends TerraformStack {
  public readonly bucketName: string | undefined;

  constructor(scope: Construct, name: string) {
    super(scope, name);

    const s3Bucket = new S3Bucket(this, "assets", {
      bucketPrefix: "my-assets"
    })

    this.bucketName = s3Bucket.bucket
  }
}

interface StackProps {
  bucketName: string | undefined
}

class MyOtherStack extends TerraformStack {
  constructor(scope: Construct, name: string, props: StackProps) {
    super(scope, name);

    console.log(props.bucketName)

    // do some fancy stuff with the bucketName reference
  }
}


const app = new App();
const stack = new MyStack(app, 'my-stack');
new MyOtherStack(app, 'my-other-stack', { bucketName: stack.bucketName});

app.synth();

Preconditions are #32 and depending on the implementation of multiple stacks #33

We do have the following working computed types:

function determineGetAttCall(type: string, terraformAttributeName: string) {
      if (type === 'string') { return `this.getStringAttribute('${terraformAttributeName}')`; }
      if (type === 'number') { return `this.getNumberAttribute('${terraformAttributeName}')`; }
      if (type === 'string[]') { return `this.getListAttribute('${terraformAttributeName}')`; }
      return undefined;
}

We're missing the following types for computed attributes:

       types: Set(36) {
          '{ [key: string]: string }',
          'AcmCertificateDomainValidationOptions[]',
          'boolean',
          'ApiGatewayAccountThrottleSettings[]',
          '{ [key: string]: boolean }',
          '{ [key: string]: string }[]',
          'number[]',
          'CloudhsmV2ClusterClusterCertificates[]',
          'DaxClusterNodes[]',
          'DefaultRouteTableRoute[]',
          'DefaultSecurityGroupEgress[]',
          'DefaultSecurityGroupIngress[]',
          'EksClusterCertificateAuthority[]',
          'EksClusterIdentity[]',
          'EksNodeGroupResources[]',
          'ElasticBeanstalkEnvironmentAllSettings[]',
          'ElasticacheClusterCacheNodes[]',
          'EmrClusterStep[]',
          'GlobalacceleratorAcceleratorIpSets[]',
          'KinesisAnalyticsApplicationInputsStartingPositionConfiguration[]',
          '{ [key: string]: number }',
          'MediaPackageChannelHlsIngest[]',
          'MqBrokerInstances[]',
          'NetworkAclEgress[]',
          'NetworkAclIngress[]',
          'OrganizationsOrganizationAccounts[]',
          'OrganizationsOrganizationNonMasterAccounts[]',
          'OrganizationsOrganizationRoots[]',
          'OrganizationsOrganizationalUnitAccounts[]',
          'RouteTableRoute[]',
          'SecurityGroupEgress[]',
          'SecurityGroupIngress[]',
          'SsmDocumentParameter[]',
          'VpcEndpointDnsEntry[]',
          'VpnConnectionRoutes[]',
          'VpnConnectionVgwTelemetry[]'
        }
      }

While the interfaces for complex computed attributes are generated, they're not implemented the corresponding attribute getter.

export interface AcmCertificateDomainValidationOptions {
  readonly domainName?: string;
  readonly resourceRecordName?: string;
  readonly resourceRecordType?: string;
  readonly resourceRecordValue?: string;
}

// ...

// domain_validation_options
  public get domainValidationOptions() {
    throw new Error('computed attribute "$acm_certificate.domain_validation_options" has unsupported type "AcmCertificateDomainValidationOptions[]"');
  }

I'd expect, that I can access those attributes as any other attribute as well.

edit: updated with list of types we have / miss

The provider version should be locked to a specific version for testing. Otherwise, there will be failed tests after each provider release.

The Typescript examples currently have the following as the app key in the cdktf.json file:

{
  "language": "typescript",
  "app": "node main.js",
.....
}

This means the users have to run two commands to generate Terraform configuration.

First they compile the Typescript application by running

tsc

Then, they run the synthesize command

cdktf synth

I think ideally we should have users just run cdktf synth and it should do the Typescript compilation and the Terraform JSON generation.

For generating a help page for the CLI, this might be handy - https://github.com/EliseevNP/cli-docs-generator

For some reason the semantics of the combination of computed and optional differs for attributes in the Terraform provider schema.

One example in the s3 bucket resource:

The arn attribute is computed and optional, but it wouldn't make sense to provide a value for this.

              "arn": {
                "type": "string",
                "optional": true,
                "computed": true
              },

For the bucket attribute, it makes a lot of sense to provide value.

              "bucket": {
                "type": "string",
                "optional": true,
                "computed": true
              },

Just by looking at the JSON schema, it's impossible to differentiate between the two types. Even by looking at the Go schema it's not possible to differentiate these two types.

I'm wondering if this is just inconsistent by accident, or if it's intentional.

Just came across this edge case:

For the aws_acm_certificate resource, it depends on the use case which of the attributes are required.

There are three use cases:

• Create AWS
• Import
• Create Private

Let's take the attribute certificate_authority_arn as an example. It's only required when using it for the Create Private case. This is only reflected in the documentation, but not in the provider schema

For now, I'll just follow the provider schema, so it'll be marked as optional.

It'd be pretty cool to publish the provider schema JSON somehow (e.g. github pages or s3 / cloudfront). It'd simplify and speed up the resource generation significantly.

We need to publish cdktf and cdktf-cli packages using the HashiCorp official account. #54 uses @anubhavmishra's personal account to publish the packages.

It'd be good when users could define outputs for their stacks, so that they can easily access their created resource data.

Something like:

class MyStack extends TerraformStack {
  constructor(scope: Construct, name: string) {
    new Output(this, "name", { value: "foo" })
  }
}

They type could probably inferred.

In Terrastack (~ a month ago) the provider alias attribute was part of the Schema. Now it's missing from the AWS provider schema. Wondering if that change was intentional.

We can't tell at runtime from which provider version the resources were generated. I think it would be good for compatibility checking and version locking to have this knowledge in the generated resource classes. Something like this perhaps:

export class TestResource extends TerraformResource {
  constructor(scope: Construct, id: string, config: TestResourceConfig) {
    super(scope, id, {
      terraformResourceType: 'test_resource',
      terraformGeneratorMetadata: {
        providerName: 'aws',
        providerVersion: '2.59.0'
      }
    });
  }
}

The providerName is straightforward. The providerVersion is not easily available at the moment. The only idea I have right now: Parse the provider file names in .terraform. There are likely better approaches - anyone?

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

The CDK for Terraform only supports generation of publicly available modules hosted in registry.terraform.io. In order to support private Terraform registries and other module sources, we will need to extend the CDK for Terraform to obtain module metadata to private or version-controlled sources.

References

N/A

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Verify that JavaScript Language Binding works. Include documentation/getting started. Since TypeScript is supported, JavaScript should work similarly.

References

AWS CDK Getting Started with JavaScript reference
https://docs.aws.amazon.com/cdk/latest/guide/work-with-cdk-javascript.html

Parsing and downloading the providers and modules

We need to document how to use TerraformOutput in Typescript.

The project examples should use the new versions of the cdktf and cdktf-cli packages after they are published to the NPM registry. Depends on #57