Provides a template for creating a Cloud Foundation Toolkit Terraform module

Home Page: https://registry.terraform.io/modules/terraform-google-modules/module-template/google

License: Apache License 2.0

Shell 12.12% Makefile 18.42% HCL 57.34% Go 12.11%

cft-terraform

terraform-google-module-template's Introduction

CFT Module Template

A template for starting a Cloud Foundation Toolkit Terraform module.

Usage

The template must be rendered using Cookiecutter.

Generate a new module by running the following command:

cookiecutter https://github.com/terraform-google-modules/terraform-google-module-template.git

Generated Module

A newly generated module includes logic to create a Google Cloud Storage bucket, a functional example module, and Kitchen-Terraform integration tests. All of this content should be modified to suit the purpose of the new module.

Testing

Changes to this template must be tested to ensure that generated modules remain functional.

Refer to the [README][./terraform-google-{{cookiecutter.module_name}}/README.md] and [CONTRIBUTING][./terraform-google-{{cookiecutter.module_name}}/CONTRIBUTING.md] documents of the template to understand the requirements for testing the generated module.

Export a Service Account key and env variables:

export SERVICE_ACCOUNT_JSON=$(< credentials.json)
export TF_VAR_org_id="your_org_id"
export TF_VAR_folder_id="your_folder_id"
export TF_VAR_billing_account="your_billing_account_id"

Generate a module and execute its tests by running the following command:

make test

The module will be generated at ./staging/terraform-google-module-test. If the tests pass, the module will be removed; otherwise, it will be left in place for inspection.

In order to execute the test following tools need to be installed:

make
docker

terraform-google-module-template's People

Contributors

Stargazers

Watchers

terraform-google-module-template's Issues

Add pre-commit-terraform

terraform-docs does not currently support Terraform 0.12.

pre-commit-terraform has implemented a workaround for using terraform-docs with Terraform 0.12. This seems like a good excuse to integrate the pre-commit framework in to our modules.

Add pre-commit-terraform to the module template and replace the existing usage of terraform-docs with the terraform_docs hook. This should include removing the logic within test/make.sh, the action in Makefile, and the package in the Dockerfile.

Add quotes around `$(CURDIR)` in Makefile

As an engineer working on the Cloud Foundation Toolkit
I want to not encounter issues when I execute tests from within a directory path containing a space or other special character
So that I can run make targets without issue.

$(CURDIR) might be a value containing a space (e.g. "/home/users/John Doe/src/..."), and will not be automatically escaped. Wrapping it in quotes in make targets resolves this issue.

Use `$_DOCKER_IMAGE_DEVELOPER_TOOLS` substitution in docker image name

terraform-google-module-template/terraform-google-{{cookiecutter.module_name}}/build/lint.cloudbuild.yaml

Line 16 in f116ac1

 - name: 'gcr.io/cloud-foundation-cicd/cft/developer-tools:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS' 

Import behavior changes from IAM test suite

Minor behavior changes to integration tests were made in terraform-google-modules/terraform-google-iam#34.

Fix false positive failures in trap handler
Display kitchen full log in trap handler.
Changes to find_files to get lint tests passing
Fix make check_terraform with 0.12, requires terraform init now

Update to use new testing framework

TL;DR

The module template should be updated to use our new testing framework.

Terraform Resources

No response

Detailed design

No response

Additional information

No response

Use `credentials.json` as the single source of credentials

To simplify the development and testing experience, any dependency on SERVICE_ACCOUNT_JSON or other credentials environment variables being set by the user should be replaced with a solution which automatically populates the variables based on credentials.json.

Build out INT Tests

TL;DR

The current int.cloudbuild.yaml is a placeholder. It could be built out similar to the LINT test, however if using the test/test.sh not only will script -qfec be required to wrap the make commands, the template Makefile might need --network=cloudbuild added to the Docker calls which might be undesirable for standard usage.

Terraform Resources

No response

Detailed design

No response

Additional information

No response

Update docker_test_integration target to use CFT CLI

TL;DR

Currently docker_test_integration calls which https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit/blob/d5809031847e01656ec0538ce66a59bd7dc68aee/infra/build/developer-tools/build/scripts/test_integration.sh which calls task helpers. All of this is handled by the CLI and can be swapped with cft test run all --verbose

Expected behavior

No response

Observed behavior

No response

Terraform Configuration

N/A

Terraform Version

N/A

Additional information

No response

Fix version constraint conflict which currently brakes integration tests

[terraform-google-module-template]$ make test
./test/test.sh
---
driver:
  variables:
    project_id: ####
make[1]: Entering directory /####/terraform-google-module-template/staging/terraform-google-module-test'
docker run --rm -it \
        -e PROJECT_ID \
        -e SERVICE_ACCOUNT_JSON \
        -e GOOGLE_APPLICATION_CREDENTIALS=/cft/workdir/credentials.json \
        -v "/####/terraform-google-module-template/staging/terraform-google-module-test":/cft/workdir \
        gcr.io/cloud-foundation-cicd/cft/kitchen-terraform:2.0.0 \
        make test_integration
Activated service account credentials for: [###@###.iam.gserviceaccount.com]
test/ci_integration.sh
+ kitchen create ''
-----> Starting Kitchen (v1.24.0)
-----> Creating <simple-example-default>...
       Terraform v0.12.0
       
       Your version of Terraform is out of date! The latest version
       is 0.12.2. You can update by downloading from www.terraform.io/downloads.html
$$$$$$ Running command `terraform init -input=false -lock=true -lock-timeout=0s  -upgrade -force-copy -backend=true  -get=true -get-plugins=true -verify-plugins=true` in directory /cft/workdir/test/fixtures/simple_example
       Upgrading modules...
       - example in ../../../examples/simple_example
       - example.module_test in ../../..
       
       Error: Unsupported Terraform Core version
       
       Module example.module_test (from "../..") does not support Terraform version
       0.12.0. To proceed, either choose another supported Terraform version or
       update the module's version constraint. Version constraints are normally set
       for good reason, so updating the constraint may lead to other errors or
       unexpected behavior.
       
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Create failed on instance <simple-example-default>.  Please see .kitchen/logs/simple-example-default.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

+ finish
+ echo 'BEGIN: finish() trap handler'
BEGIN: finish() trap handler
+ kitchen destroy ''
-----> Starting Kitchen (v1.24.0)
-----> Destroying <simple-example-default>...
       Terraform v0.12.0
       
       Your version of Terraform is out of date! The latest version
       is 0.12.2. You can update by downloading from www.terraform.io/downloads.html
$$$$$$ Running command `terraform init -input=false -lock=true -lock-timeout=0s  -force-copy -backend=true  -get=true -get-plugins=true -verify-plugins=true` in directory /cft/workdir/test/fixtures/simple_example
       Initializing modules...
       
       Error: Unsupported Terraform Core version
       
       Module example.module_test (from "../..") does not support Terraform version
       0.12.0. To proceed, either choose another supported Terraform version or
       update the module's version constraint. Version constraints are normally set
       for good reason, so updating the constraint may lead to other errors or
       unexpected behavior.
       
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Destroy failed on instance <simple-example-default>.  Please see .kitchen/logs/simple-example-default.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

make: *** [Makefile:78: test_integration] Error 20
make[1]: *** [Makefile:142: test_integration_docker] Error 2
make[1]: Leaving directory '/####/terraform-google-module-template/staging/terraform-google-module-test'
make[1]: Entering directory '/####/terraform-google-module-template/staging/terraform-google-module-test'
docker run --rm -it \
        -e PROJECT_ID \
        -e SERVICE_ACCOUNT_JSON \
        -e GOOGLE_APPLICATION_CREDENTIALS=/cft/workdir/credentials.json \
        -v "/####/terraform-google-module-template/staging/terraform-google-module-test":/cft/workdir \
        gcr.io/cloud-foundation-cicd/cft/kitchen-terraform:2.0.0 \
        /bin/bash -c "source test/ci_integration.sh && setup_environment && kitchen destroy"
Activated service account credentials for: [####@####.iam.gserviceaccount.com]
-----> Starting Kitchen (v1.24.0)
-----> Destroying <simple-example-default>...
       Terraform v0.12.0
       
       Your version of Terraform is out of date! The latest version
       is 0.12.2. You can update by downloading from www.terraform.io/downloads.html
$$$$$$ Running command `terraform init -input=false -lock=true -lock-timeout=0s  -force-copy -backend=true  -get=true -get-plugins=true -verify-plugins=true` in directory /cft/workdir/test/fixtures/simple_example
       Initializing modules...
       
       Error: Unsupported Terraform Core version
       
       Module example.module_test (from "../..") does not support Terraform version
       0.12.0. To proceed, either choose another supported Terraform version or
       update the module's version constraint. Version constraints are normally set
       for good reason, so updating the constraint may lead to other errors or
       unexpected behavior.
       
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Destroy failed on instance <simple-example-default>.  Please see .kitchen/logs/simple-example-default.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

make[1]: *** [Makefile:132: docker_destroy] Error 20
make[1]: Leaving directory '/####/terraform-google-module-template/staging/terraform-google-module-test'
make: *** [Makefile:3: test] Error 2

Add `.terraform.lock.hcl` files to the initial `.gitignore` file

a new section with

# Terraform lock file
**/.terraform.lock.hcl

Should be added to terraform section of the initial .gitignore file

terraform-google-module-template/terraform-google-{{cookiecutter.module_name}}/.gitignore

Lines 24 to 30 in 3886075

 # Local .terraform directories 

 **/.terraform/* 

 # .tfstate files 

 *.tfstate 

 *.tfstate.*

The file is created automatically since TF 0.14

Github token reference interpreted as a cookiecutter variable.

Looks like cfcb92c broke the cookiecutter template because the Github token reference ${{ secrets.GITHUB_TOKEN }} is interpreted as a cookiecutter variable.

Fix Test Suite

Running make tests yields errors on master:

Error: "ci-module_test" name must be 4 to 30 characters with lowercase and uppercase letters, numbers, hyphen, single-quote, double-quote, space, and exclamation point.

  on .terraform/modules/project/terraform-google-modules-terraform-google-project-factory-f93d3cd/modules/core_project_factory/main.tf line 126, in resource "google_project" "main":
 126: resource "google_project" "main" {


make[1]: *** [docker_test_prepare] Error 1

It looks like https://github.com/terraform-google-modules/terraform-google-module-template/blob/master/terraform-google-%7B%7Bcookiecutter.module_name%7D%7D/test/setup/main.tf#L21 is trying to replace hyphens with underscores, when it should be the other way around.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

fix(deps): update go modules and/or dev-tools (github.com/GoogleCloudPlatform/cloud-foundation-toolkit/infra/blueprint-test, github.com/stretchr/testify)

Detected dependencies

gomod

terraform-google-{{cookiecutter.module_name}}/test/integration/go.mod

go 1.21

github.com/GoogleCloudPlatform/cloud-foundation-toolkit/infra/blueprint-test v0.11.1

github.com/stretchr/testify v1.8.4

regex

terraform-google-{{cookiecutter.module_name}}/Makefile

cft/developer-tools 1

build/int.cloudbuild.yaml

cft/developer-tools 1

build/lint.cloudbuild.yaml

cft/developer-tools 1

terraform-google-{{cookiecutter.module_name}}/build/int.cloudbuild.yaml

cft/developer-tools 1

terraform

terraform-google-{{cookiecutter.module_name}}/examples/simple_example/main.tf

terraform-google-{{cookiecutter.module_name}}/examples/simple_example/versions.tf

hashicorp/terraform >= 0.13

terraform-google-{{cookiecutter.module_name}}/test/setup/main.tf

terraform-google-modules/project-factory/google ~> 14.0

terraform-google-{{cookiecutter.module_name}}/test/setup/versions.tf

google >= 3.25.0

google-beta >= 3.25.0

hashicorp/terraform >= 0.13

terraform-google-{{cookiecutter.module_name}}/versions.tf

google >= 3.53, < 6

hashicorp/terraform >= 0.13

Check this box to trigger a request for Renovate to run again on this repository

developer-tools 1.2 does not run with rootless docker

TL;DR

There seems to be a change between 1.1 and 1.2 which prevents the developer-tools image from running under podman (userspace mode) or docker (inside sysbox, similar to rootless docker).

If this is the wrong repo to file under, I'd appreciate a pointer to the correct one.

Expected behavior

I'd expect the image to run normally.

Observed behavior

Running with docker in sysbox, I get the following error:

$ docker run --rm -it gcr.io/cloud-foundation-cicd/cft/developer-tools:1.2
docker: failed to register layer: ApplyLayer exit status 1 stdout:  stderr: lchown /build/terraform-validator: invalid argument.

Running version 1.1 works fine:

$ docker run --rm -it gcr.io/cloud-foundation-cicd/cft/developer-tools:1.1
Loading /usr/local/bin/task_helper_functions.sh from ~/.bashrc
Invoking init_credentials from ~/.bashrc
Disable this behavior by setting CFT_DISABLE_INIT_CREDENTIALS=yes
Error: SERVICE_ACCOUNT_JSON must contain the JSON string (not the
file path) of the service account credentials.  For example:
export SERVICE_ACCOUNT_JSON=$(< ~/.credentials/my-sa-key.json)
[root@bdd688f8bb2a workspace]#

I get a similar error message when running on podman on my laptop, so this may just be an issue caused by running in rootless mode:

$ podman run --rm -it gcr.io/cloud-foundation-cicd/cft/developer-tools:1.2
Error: writing blob: adding layer with blob "sha256:cd161d4c1a089eaebfd0f869672c4d18d849997b8f1ce20887250ad61820844e": Error processing tar file(exit status 1): potentially insufficient UIDs or GIDs available in user namespace (requested 806984:89939 for /build/terraform-validator): Check /etc/subuid and /etc/subgid: lchown /build/terraform-validator: invalid argument

I'm not sure if this is a bug in sysbox, docker/runc, or something to do with this image.

I filed a bug with more info in the sysbox repository: nestybox/sysbox#443 -- there's also a similar-looking docker bug here: moby/moby#41821

Terraform Configuration

n/a

Terraform Version

n/a

Additional information

No response

combine_docfiles.py fails if the license comment is missing in Terraform configuration files

terraform-google-modules/terraform-google-vpc-service-controls#1 (comment)

	# Local .terraform directories
	*/.terraform/

	# .tfstate files
	*.tfstate
	.tfstate.

terraform-google-modules / terraform-google-module-template Goto Github PK

terraform-google-module-template's Introduction

CFT Module Template

Usage

Generated Module

Testing

terraform-google-module-template's People

Contributors

Stargazers

Watchers

Forkers

terraform-google-module-template's Issues

TL;DR

Terraform Resources

Detailed design

Additional information

TL;DR

Terraform Resources

Detailed design

Additional information

TL;DR

Expected behavior

Observed behavior

Terraform Configuration

Terraform Version

Additional information

Open

Detected dependencies

TL;DR

Expected behavior

Observed behavior

Terraform Configuration

Terraform Version

Additional information

Recommend Projects

Recommend Topics

Recommend Org