terraform-google-modules / terraform-google-project-factory Goto Github PK

When the service account helper script is invoked in some environments (such as the WSL), variables like $RANDOM may not be set. The script doesn't catch errors before setting up credentials and can fail in an unexpected way. We should be prefer to fail early rather than running bad commands.

Deletion of Default Compute Service Account causes issues where you cannot use services:

• AppEngine Flex
• Cloud Run

Perhaps - Use the Default Compute Account: ([PROJECT_NUMBER][email protected]) and Reduce the IAM Permission on it, instead of deletion of the Default and Recreation.

Similar to how we have functionality to delete the default GCE service account, can we add an option to delete default VPC gateway route on the project VPC?

The customer is asking for this feature as they wish to intercept all egress traffic with a security device. They will create their own route for this interception and routing of traffic from the security device to the gateway.

Thanks!

The link contained in the "Verifying Setup" section of the readme.md file is wrong. https://github.com/terraform-google-modules/terraform-google-project-factory#verifying-setup

A preconditions checker script is included to verify that all preconditions are met before the Project Factory runs. The script will run automatically if the script dependencies (Python, "google-auth", and "google-api-python-client") are available at runtime. If the dependencies are not met, the precondition checking step will be skipped.

The precondition checker script can be directly invoked before running the project factory:

./scripts/preconditions/preconditions.py
--credentials_path "./credentials.json"
--billing_account 000000-000000-000000
--org_id 000000000000
--folder_id 000000000000
--shared_vpc 'shared-vpc-host-ed64'

The link in the readme should point to the actual file location as such: https://github.com/terraform-google-modules/terraform-google-project-factory/blob/master/test/scripts/preconditions/test_preconditions.py

But instead it now shows: https://github.com/terraform-google-modules/terraform-google-project-factory/blob/master/scripts/preconditions/preconditions.py

I will create a pull request to fix the readme file.

In #72, we messed up the logic for granting networkUser permission.

There are meant to be 3 different conditions.

1. No shared_vpc and no subnets => no grants
2. shared_vpc and no subnets => grant to the project (all networks/subnets)
3. shared_vpc and subnets => grant to the subnets (and not to the project)

Hi,
the module creates the bucket in the US (bucket_name and bucket_project options), it would be cool to have the ability to choose the region.

Thanks

I'm trying to use this module to create a gcp project but it fails when it tries to delete the default service account. I'm using google cloud build as a ci tool.

Finished Step #3 Step #3: Step #3: Step #3: above and apply again to incrementally change your infrastructure. Step #3: any resources that successfully completed. Please address the error Step #3: Instead, your Terraform state file has been partially updated with Step #3: Terraform does not automatically rollback in the face of errors. Step #3: Step #3: Step #3: * module.project.module.project-factory.null_resource.delete_default_compute_service_account: Error running command '/workspace/environments/dev/.terraform/modules/112ba820419105134cc3b145beda68a6/scripts/delete-service-account.sh veamly-platform-dev [email protected] ': exit status 127. Output: /bin/sh: /workspace/environments/dev/.terraform/modules/112ba820419105134cc3b145beda68a6/scripts/delete-service-account.sh: not found Step #3: Step #3: 1 error(s) occurred:

If you try to use the Project Factory without a Shared VPC, it fails with this error:

* module.project-demo-without-shared-vpc.google_project_iam_member.controlling_group_vpc_membership: google_project_iam_member.controlling_group_vpc_membership: value of 'count' cannot be computed

module "host-project" {
  source            = "terraform-google-modules/project-factory/google"
  random_project_id = true
  name              = "${local.host_project_name}"
  billing_account   = "${var.billing_account}"
  auto_create_network = false
  group_name = "${var.sysadmingroup}"
  labels = "${var.labels}"
  folder_id = "${var.folder_id}"
  # org_id = "${var.org_id}
}

When both org_id and folder_id are specified:

• google_project.main: 'org_id' and 'folder_id' cannot be both set.

When org_id is not specified:
module "host-project": missing required argument "org_id"

How can I create a project in a folder?

Initializing modules...

• module.project-factory
  Found version 0.2.1 of terraform-google-modules/project-factory/google on registry.terraform.io
  Getting source "terraform-google-modules/project-factory/google"

Initializing provider plugins...

• Checking for available provider plugins on https://releases.hashicorp.com...

Provider "gsuite" not available for installation.

A provider named "gsuite" could not be found in the official repository.

This may result from mistyping the provider name, or the given provider may
be a third-party provider that cannot be installed automatically.

In the latter case, the plugin must be installed manually by locating and
downloading a suitable distribution package and placing the plugin's executable
file in the following directory:
terraform.d/plugins/darwin_amd64

Terraform detects necessary plugins by inspecting the configuration and state.
To view the provider versions requested by each module, run
"terraform providers".

Hi,
If you try to destroy a project with "complex" APIs activated, you have errors because the option disable_dependent_services is not set:

* google_project_service.project_services.16: Error disabling service: Error disabling service "replicapoolupdater.googleapis.com" for project "test-project-factory-22bd": googleapi: Error 400: The service replicapoolupdater.googleapis.com is depended on by the following active services: resourceviews.googleapis.com,deploymentmanager.googleapis.com,replicapool.googleapis.com; Please specify disable_dependent_services=true if you want to proceed with disabling all services., failedPrecondition
* module.test.module.project-factory.google_project_service.project_services[15] (destroy): 1 error(s) occurred:

* google_project_service.project_services.15: Error disabling service: Error disabling service "replicapool.googleapis.com" for project "test-project-factory-22bd": googleapi: Error 400: The service replicapool.googleapis.com is depended on by the following active services: resourceviews.googleapis.com,replicapoolupdater.googleapis.com,deploymentmanager.googleapis.com; Please specify disable_dependent_services=true if you want to proceed with disabling all services., failedPrecondition
* module.test.module.project-factory.google_project_service.project_services[2] (destroy): 1 error(s) occurred:

* google_project_service.project_services.2: Error disabling service: Error disabling service "containerregistry.googleapis.com" for project "test-project-factory-22bd": googleapi: Error 400: The service containerregistry.googleapis.com is depended on by the following active services: resourceviews.googleapis.com,replicapoolupdater.googleapis.com,container.googleapis.com,deploymentmanager.googleapis.com,replicapool.googleapis.com; Please specify disable_dependent_services=true if you want to proceed with disabling all services., failedPrecondition
* module.test.module.project-factory.google_project_service.project_services[7] (destroy): 1 error(s) occurred:

* google_project_service.project_services.7: Error disabling service: Error disabling service "monitoring.googleapis.com" for project "test-project-factory-22bd": googleapi: Error 400: The service monitoring.googleapis.com is depended on by the following active service: stackdriver.googleapis.com; Please specify disable_dependent_services=true if you want to proceed with disabling all services., failedPrecondition

The variable disable_services_on_destroy use to be added to the core module call in the /main.tf file. However, at some point it seems it was removed (commit 3738754 )

Is this intended? If so, it is possible to regain that funcionality somehow?

Hello,
my intention is to create a script script.sh that uses one Terraform config main.tf with a single project factory module in order to create an arbitrary number of GCP projects.
The idea of the usage is:

./script.sh project-x
./script.sh project-y
./script.sh project-z

where projects x,y,z are GCP projects that i'd like to create and script.sh does roughly
terraform apply -auto-approve -var "credentials_path=credentials.json" -var "project_name=${1}"
and the main.tf is only composed of a single module which obtains a different project name from script.sh each time.

The problem is that if the first call to create project-x succeeds, a Terraform state file is created, and subsequent script call to create project-y would react to the existing Terraform state file by deleting project-x.
What I'd like to achieve is to somehow ensure that the sequence of the script.sh calls leads to creation and peaceful co-existence of all projects.

What would you recommend, please ?

IDEAS:

• I am experimenting with script.sh doing the terraform apply calls in separate directories specific to the project name, thus getting Terraform state files only related to specific projects. E.g. ./script.sh project-y would create directory project-y and do the rest of the work inside of it. This seems to kinda work, but I am not sure whether I like this solution.

• Another idea I have is to maintain a dynamic Terraform config file main.tf where ./script.sh project-ywould add a module for project project-y into main.tf, then do terraform apply and hopefully let Terraform maintain a single state file for all the projects without problems.

We should be able to create a project without having roles/organization.viewer.

Currently the org_id is used in 2 places:

• google_project resource (not needed unless the project is straight under the org)
• google_organization (this resource is used to get the org domain in order to construct the default service account email string. This can be optional by letting the user pass 'domain' input directly

Using v0.21 of this module and v1.19.1 of terraform-provider-google, we see the following warning:

Warning: module.priv-env.module.vault-project.google_project.project: "app_engine": [DEPRECATED] Use the google_app_engine_application resource instead.

The Kitchen-Terraform Dockerfile
installs the gem bundle under /opt/kitchen, which is different from the WORKDIR.

Additionally, the Makefile does not use bundler to execute Kitchen.

Since Bundler is context sensitive, the combination of these two points means that the gems provided by the Gemfile are not necessarily used when executing the Kitchen-Terraform tasks in the Makefile.

I discovered this while attempting to test using an alternative source for the Kitchen-Terraform gem.

I notice that the docs say that 1.10 is supported. Is 1.11?

This module expects to be run on an OS that has been configured for a google cloud developer. If you follow the installation instructions on a fresh, minimal linux install you will encounter unsatisfied dependencies. I have had to add these to my build server:

1. gcloud
2. jq
   These ought to be mentioned as requirements

Based on the default 3 conditions

1. No shared_vpc and no subnets => no grants
2. shared_vpc and no subnets => grant to the project (all networks/subnets)
3. shared_vpc and subnets => grant to the subnets (and not to the project)

If the subnet id get messed up ( fat finger) while attempting case 3, project factory ends up sharing the entire set of networks in the host project VPC to the service project. Which is a serious security issue. Ideally it should follow step 1 and should not grant any thing and error out.

I have a user that does not have the correct permissions to run the setup-sa.sh script. The documentation does not tell me what permissions I need in order to run this scripts. Here is my error:

 $ ./helpers/setup-sa.sh my-org my-project
Verifying organization...
Verifying project...
Skipping billing account verification... (parameter not passed)
Creating Seed Service Account...
Created service account [project-factory-6999].
Downloading key to credentials.json...
Applying permissions for org org-number and project chlove-baseproject...
ERROR: (gcloud.organizations.add-iam-policy-binding) User [[email protected]] does not have permission to access organization [org-number:getIamPolicy] (or it may not exist): The caller does not have permission

I removed org number and project id for security sake.

What permissions does the user need to run the setup-sa.sh script?

The CI tests merged still have outstanding issues:

• group_email is invalid: https://concourse.infra.cft.tips/teams/cft/pipelines/terraform-google-project-factory/jobs/integration-tests/builds/137#L5c267bbe:392

The tests currently don't make it easy to test G Suite-related functionality in the module.

We should add fixtures to consistently exercise the create_group and API/SA group functionalities.

Error: module.host-project.module.project-factory.google_project_service.project_services: : invalid or unknown key: disable_dependent_services

cft.tfvars
organization_id = "12345678790"
billing_account = "AAAAA-BBBBB-CCCCC"
credentials_path = "demo_credentials_20190430131031.json"

terraform.tfvars
host_project_name = "shared-vpc-host-090519"
network_name = "shared-vpc-net-090509"

terraform -v
Terraform v0.11.13

• provider.google v1.20.0
• provider.google-beta v1.20.0
• provider.null v2.1.2
• provider.random v2.1.2

This line should be non-authoritative: e16b554#diff-7a370d8342e7203b805911c92454f0f4R187

Running make docker_create after make docker_build_terraform and make docker_build_kitchen_terraform results in the following error:

   Initializing provider plugins...
   - Checking for available provider plugins on https://releases.hashicorp.com...
   
   Provider "gsuite" not available for installation.
   
   A provider named "gsuite" could not be found in the official repository.

Debug output reveals the cause of the issue:

   2018/12/12 18:22:11 [DEBUG] checking for provider in "."
   2018/12/12 18:22:11 [DEBUG] checking for provider in "/cftk/bin"
   2018/12/12 18:22:11 [DEBUG] checking for provider in ".terraform/plugins/linux_amd64"
   2018/12/12 18:22:11 [DEBUG] found provider "terraform-provider-google_v1.19.1_x4"
   2018/12/12 18:22:11 [DEBUG] found provider "terraform-provider-null_v1.0.0_x4"
   2018/12/12 18:22:11 [DEBUG] found provider "terraform-provider-random_v2.0.0_x4"
   2018/12/12 18:22:11 [DEBUG] checking for provider in "/cftk/home/.terraform.d/plugins"
   2018/12/12 18:22:11 [WARN] found legacy provider "terraform-provider-google"
   2018/12/12 18:22:11 [WARN] found legacy provider "terraform-provider-gsuite"
   2018/12/12 18:22:11 [DEBUG] found valid plugin: "google", "0.0.0", "/cftk/home/.terraform.d/plugins/terraform-provider-google"
   2018/12/12 18:22:11 [DEBUG] found valid plugin: "gsuite", "0.0.0", "/cftk/home/.terraform.d/plugins/terraform-provider-gsuite"
   2018/12/12 18:22:11 [DEBUG] found valid plugin: "google", "1.19.1", "/cftk/workdir/test/fixtures/minimal/.terraform/plugins/linux_amd64/terraform-provider-google_v1.19.1_x4"                                                                                                                                        
   2018/12/12 18:22:11 [DEBUG] found valid plugin: "null", "1.0.0", "/cftk/workdir/test/fixtures/minimal/.terraform/plugins/linux_amd64/terraform-provider-null_v1.0.0_x4"                                                                                                                                              
   2018/12/12 18:22:11 [DEBUG] found valid plugin: "random", "2.0.0", "/cftk/workdir/test/fixtures/minimal/.terraform/plugins/linux_amd64/terraform-provider-random_v2.0.0_x4"

Because the plugin version is removed from the filename, Terraform considers the plugin to be v0.0.0 and thus not satisfactory for the ~> 0.1.9 requirement of the test fixture module.

We were ironing out some permission issues so needed to re-run a TF pipeline a few times. It appears possible to create a project, fail to activate the right APIs, and then cause other parts of the pipeline to fail. (Note: the module name is our internal fork of project factory with no changes).

To reproduce, try to create a new project using a Service Account lacking this permission:

module.project-factory-project.module.sfdc_project.data.google_organization.org: data.google_organization.org: Error reading Organization Not Found : organizations/849937818733: googleapi: Error 403: The caller does not have permission, forbidden

If you fix that, then you will get this error:

Error: Error refreshing state: 1 error(s) occurred:
 * module.project-factory-project.module.sfdc_project.data.google_compute_default_service_account.default: 1 error(s) occurred:
 * module.project-factory-project.module.sfdc_project.data.google_compute_default_service_account.default: data.google_compute_default_service_account.default: Error reading GCE service account not found: googleapi: Error 403: Project 722914074885 is not found and cannot be used for API calls. If it is recently created, enable Compute Engine API by visiting https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project=722914074885 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry., accessNotConfigured

All subsequent plans will fail because this call happens early in the plan, preventing the rest.

GCP project names can not be reused, attempting to do so results in a 409 error. However, this module won't accept an imported project, e.g. terraform import module.X.google_project.main my-project, because id and project_id can not be supplied as inputs. The imported project results in

-/+ module.my-project.google_project.main (new resource required)
id: "my-project-name" => (forces new resource)
auto_create_network: "true" => "false"
billing_account: "XXXXX-XXXXX-XXXXX" => "XXXXX-XXXXX-XXXXX"
folder_id: "" =>
name: "my-project-name" => "my-project-name"
number: "808999117972" =>
org_id: "1234567890" => "1234567890"
policy_data: "" =>
policy_etag: "" =>
project_id: "my-project-name" => "${local.temp_project_id}" (forces new resource)
skip_delete: "" =>

Currently, any changes to the project name updates the project ID, requiring a full recreation of the project. This is causing us significant trouble as we roll out new governance and standards changes across the organization. In local testing, adding ignore_changes = ["project_id"] to the google_project's lifecycle fixes this.

I'm happy to submit a PR if this is an acceptable direction.

credentials_path variable is cumbersome to pass when automating things with CICD.

E.g with Jenkins we still need to:

• upload the service account key file
• store it in some place that's doesn't get destroyed (so no in the build workspace)
• refer to the service account keyfile path in our terraform.tfvars, which might be seen by many users

In most cases we could use Application Default Credentials for this.
AFAIK, credentials_path is used only to delete the default service account on a new project.

This is just a tracking issue while we wait for a flag to be added to the API to disable the creation of the default service account.

The tests currently only work if your SA key has an absolute path.

Context: 50a7b7b

During preconditions check, if the seed project has a large number of APIs enabled, the python precondition check cannot page and preconditions fail. Suggest increasing the pageSize option as below.

        parent = "projects/" + self.project_id
        request = service.services().list(
            parent=parent,
            filter='state:ENABLED',
            pageSize='200'
        )

Error observed that led to the discovery.
': exit status 1. Output: [
{
"type": "Required APIs on service account project",
"name": "projects/project-abc123",
"satisfied": [
"cloudresourcemanager.googleapis.com",
"appengine.googleapis.com",
"cloudbilling.googleapis.com",
"admin.googleapis.com"
],
"unsatisfied": [
"iam.googleapis.com"
]
},

With the merge of #64, the gcloud Alpha commands component must be installed in order to test for the existence of liens. The lien tests in the project-factory and minimal controls are currently failing due to the missing component.

https://cloud.google.com/vpc/docs/deprovisioning-shared-vpc#deleting_shared_vpc_service_project

When deleting a GCP project that is a service project associated with a Host Shared VPC project, terraform should attempt to remove the association before deleting the project.

Currently, the G Suite functionality causes a lot of pain and complexity in set up. We should make it a totally optional component (meaning you can use the project factory without ever installing the G Suite provider).

New architecture:

• outer wrapper module references inner module
• gsuite add on handles group creation and service account group management, passes through to inner module
• inner module which holds all the non-gsuite components from current module

Tested on v0.2.1 and master

Possibly related to #47; we find that passing an interpolated value to org_id that references a data source or resource fails with this error during refresh:

module.project.null_resource.args_missing: : invalid or unknown key: ERROR: Variable `group_name` was passed. Please provide either `org_id` or `domain` variables

Our use case: we store shared state, like the organization and folders, in a separate backend which we reference using terraform_remote_state. This bug makes it impossible to reference shared state in this way.

Is there a workaround we can pursue in the near term?

After #34, a few issues remain.

• deduplicate terraform.tfvars file between full and minimal fixtures (#34 (comment))
• Test failures for gsuite group email
• Test failures for gsuite membership on roles/iam.serviceAccountUser
• Document optional test coverage: project-factory-shared-vpc
• Test project-factory-sa-role by default
• Document optional test coverage: project-factory-usage

Additionally, I noticed a number of unexpected skips and failures even with a full tfvars:

Profile: full
Version: (not specified)
Target:  local://

  ↺  project-factory-gsuite: Project Factory G Suite integration (2 failed) (1 skipped)
     ↺  includes the gsuite group in the given role
     ✔  Command: `gcloud projects get-iam-policy pf-test-int-full-e736 --format=json` exit_status should eq 0
     ✔  Command: `gcloud projects get-iam-policy pf-test-int-full-e736 --format=json` stderr should eq ""
     ×  Command: `gcloud iam service-accounts get-iam-policy project-service-account@pf-test-int-full-e736.iam.gserviceaccount.com --format=json` includes the group email 
     expected: not nil
          got: nil
     ×  Command: `gcloud iam service-accounts get-iam-policy project-service-account@pf-test-int-full-e736.iam.gserviceaccount.com --format=json` does not overwrite the membership of role roles/iam.serviceAccountUser
     expected: not nil
          got: nil
     ✔  Command: `gcloud iam service-accounts get-iam-policy project-service-account@pf-test-int-full-e736.iam.gserviceaccount.com --format=json` exit_status should eq 0
     ✔  Command: `gcloud iam service-accounts get-iam-policy project-service-account@pf-test-int-full-e736.iam.gserviceaccount.com --format=json` stderr should eq ""
  ✔  project-factory-app-engine: Project Factory App Engine configuration
     ✔  Command: `gcloud app describe --project pf-test-int-full-e736 --format=json` should include {:authDomain => "clearify.co"}
     ✔  Command: `gcloud app describe --project pf-test-int-full-e736 --format=json` should include {:featureSettings => {}}
     ✔  Command: `gcloud app describe --project pf-test-int-full-e736 --format=json` should include {:id => "pf-test-int-full-e736"}
     ✔  Command: `gcloud app describe --project pf-test-int-full-e736 --format=json` should include {:name => "apps/pf-test-int-full-e736"}
     ✔  Command: `gcloud app describe --project pf-test-int-full-e736 --format=json` should include {:locationId => "us-east4"}
     ✔  Command: `gcloud app describe --project pf-test-int-full-e736 --format=json` should include {:servingStatus => "SERVING"}
     ✔  Command: `gcloud app describe --project pf-test-int-full-e736 --format=json` exit_status should equal 0
     ✔  Command: `gcloud app describe --project pf-test-int-full-e736 --format=json` stderr should eq ""
  ↺  project-factory-shared-vpc: Project Factory shared VPC
     ↺  Skipped control due to only_if condition.
  ✔  project-factory: Project Factory
     ✔  Command: `gcloud projects describe pf-test-int-full-e736` exit_status should equal 0
     ✔  Command: `gcloud projects describe pf-test-int-full-e736` stderr should eq ""
     ✔  Command: `gcloud services list --project pf-test-int-full-e736` exit_status should equal 0
     ✔  Command: `gcloud services list --project pf-test-int-full-e736` stderr should eq ""
     ✔  Command: `gcloud services list --project pf-test-int-full-e736` stdout should match /compute\.googleapis\.com/
     ✔  Command: `gcloud services list --project pf-test-int-full-e736` stdout should match /container\.googleapis\.com/
     ✔  Command: `gcloud iam service-accounts list --project pf-test-int-full-e736 --format='json(email)'` includes the Google App Engine API service account user
     ✔  Command: `gcloud iam service-accounts list --project pf-test-int-full-e736 --format='json(email)'` includes the service account generated by the project factory
     ✔  Command: `gcloud iam service-accounts list --project pf-test-int-full-e736 --format='json(email)'` includes the service account created outside of the project factory
     ✔  Command: `gcloud iam service-accounts list --project pf-test-int-full-e736 --format='json(email)'` exit_status should equal 0
     ✔  Command: `gcloud iam service-accounts list --project pf-test-int-full-e736 --format='json(email)'` stderr should eq ""
  ↺  project-factory-sa-role: Project factory service account role
     ↺  Skipped control due to only_if condition.
  ↺  project-factory-usage: Project factory usage bucket
     ↺  Skipped control due to only_if condition.

In the main Readme file, it states that Google App Engine Admin API - appengine.googleapis.com is required. I was able to run the module successfully without it, and I don't know why it would be required in the first place.

Maybe just update the Readme file?

My input:

module "project-factory" {
source = "github.com/terraform-google-modules/terraform-google-project-factory"
random_project_id = "true"
name = "${var.team_project_name}"
folder_id = "${var.team_folder_id}"
org_id = "${var.organization_id}"
billing_account = "${var.billing_account}"
credentials_path = "${local.credentials_file_path}"
}

Hello -

Curious about best practice for deploying multiple projects and keeping tfstate files unique to each. Would you manage those with workspaces or using modules or?

Thanks

The README does not mention that you need to install the prereqs for precondions.py to run. Maybe we should mention it?

modules/core_project_factory/scripts/preconditions/preconditions.py \
  --credentials_path "/Users/chlove/.config/gcloud/application_default_credentials.json" \
  --billing_account  my-account \
  --org_id my-id
Traceback (most recent call last):
  File "modules/core_project_factory/scripts/preconditions/preconditions.py", line 34, in <module>
    raise e
  File "modules/core_project_factory/scripts/preconditions/preconditions.py", line 25, in <module>
    import google.auth
ModuleNotFoundError: No module named 'google'

I know that I need to run pip3 with the requirements files, but not everyone will know that :)

The second converge is necessary for app engine settings to converge.

Several of the tests have variables with blank defaults when they are actually required (and tests fail to run if they're not provided). For example: https://github.com/terraform-google-modules/terraform-google-project-factory/blob/master/test/fixtures/full/variables.tf#L55

In general, blank defaults are an anti-pattern which we should strongly avoid. It's against our standards. Please check through all variables again and confirm that everything with a default value can be left at the default.

When using project factory to create the projects (during today's training - was online), I couldn't get the project created...

My billing account definitely exists and is correct in the shape 'XXXXXX-XXXXXX-XXXXXX'.

The error I get:

Error: Error applying plan:

1 error(s) occurred:

* module.project-factory.google_project.project: 1 error(s) occurred:

* google_project.project: Error setting billing account "XXXXXX-XXXXXX-XXXXXX" for project "projects/blah-blah-blah-d50d": googleapi: Error 400: Precondition check failed., failedPrecondition

When I try to reproduce with TF_LOG=trace, I get this respose for the request to /v1/projects/alexg-proj-factory-d50d/billingInfo:

2018-09-17T17:19:25.091-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: ---[ RESPONSE ]--------------------------------------
2018-09-17T17:19:25.091-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: HTTP/2.0 400 Bad Request
2018-09-17T17:19:25.091-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: Cache-Control: private
2018-09-17T17:19:25.091-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: Content-Type: application/json; charset=UTF-8
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: Date: Mon, 17 Sep 2018 20:19:25 GMT
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: Server: ESF
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: Vary: Origin
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: Vary: X-Origin
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: Vary: Referer
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: X-Content-Type-Options: nosniff
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: X-Frame-Options: SAMEORIGIN
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: X-Xss-Protection: 1; mode=block
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: {
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:   "error": {
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:     "code": 400,
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:     "message": "Precondition check failed.",
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:     "errors": [
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:       {
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:         "message": "Precondition check failed.",
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:         "domain": "global",
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:         "reason": "failedPrecondition"
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:       }
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:     ],
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:     "status": "FAILED_PRECONDITION"
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:   }
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: }
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4:
2018-09-17T17:19:25.092-0300 [DEBUG] plugin.terraform-provider-google_v1.17.1_x4: -----------------------------------------------------```

This leaved the resources in a strange state and clearning up to restart basically involves manual drop of folders and "half-created" project. `terraform refresh` doesn't work because Compute Engine API is not enabled there (which cannot be done without linking a billing account, of course). But that's another problem - the issue is with 

#34 (comment)

I need to be able to tell terraform-google-modules/project-factory/google to use terraform's google_project_services resource (instead of google_project_service).

I propose a new attribute apis_authority for use in modules/core_project_factory/main.tf:

resource "google_project_service" "project_services" {
  count = "${ apis_authority == "true" ? 0 : length(var.activate_apis)}"
  ...

resource "google_project_services" "project_services_authority" {
  count = "${ apis_authority == "true" ? 1 : 0}"
  ...

I can implement a workaround in our code by not passing activate_apis to the module but I see that the module is treating container.googleapis.com as a special case and I'm concerned about future special cases.

Typical usage:

module "my-project" {
  source = "terraform-google-modules/project-factory/google"
  apis_authority = true
  activate_apis = [ ... ]
  ...

If I specify shared_vpc in a service project I receive following error during plan or apply. If I remove shared_vpc it works but creates a standalone project instead of a service project.

* module.service_project01.google_compute_subnetwork_iam_member.group_role_to_vpc_subnets:
google_compute_subnetwork_iam_member.group_role_to_vpc_subnets: value of 'count' cannot be computed
* module.service_project01.google_compute_shared_vpc_service_project.shared_vpc_attachment: google_compute_shared_vpc_service_project.shared_vpc_attachment: value of 'count' cannot be computed
* module.service_project01.google_project_iam_member.controlling_group_vpc_membership: google_project_iam_member.controlling_group_vpc_membership: value of 'count' cannot be computed
* module.service_project01.google_compute_subnetwork_iam_member.gke_shared_vpc_subnets: google_compute_subnetwork_iam_member.gke_shared_vpc_subnets: value of 'count' cannot be computed
* module.service_project01.google_compute_subnetwork_iam_member.service_account_role_to_vpc_subnets: google_compute_subnetwork_iam_member.service_account_role_to_vpc_subnets: value of 'count' cannot be computed
* module.service_project01.google_compute_subnetwork_iam_member.apis_service_account_role_to_vpc_subnets: google_compute_subnetwork_iam_member.apis_service_account_role_to_vpc_subnets: value of 'count' cannot be computed
* module.service_project01.google_project_iam_member.gke_host_agent: google_project_iam_member.gke_host_agent: value of 'count' cannot be computed

Example resources:

module "host_project01" {
  source            = "/modules/project-factory"
  random_project_id = "true"
  name              = "host-project-01"
  org_id            = "${var.organization_id}"
  billing_account   = "${var.billing_account}"
  activate_apis     = ["compute.googleapis.com"]
  credentials_path  = "${local.credentials_file_path}"
}

module "service_project01" {
  source            = "/modules/project-factory"
  random_project_id = "true"
  name              = "service-project01"
  org_id            = "${var.organization_id}"
  billing_account   = "${var.billing_account}"
  folder_id         = "${google_folder.prod.id}"
  shared_vpc        = "${module.host_project01.project_id}" # If I remove this it works but creates a standalone project.
  activate_apis     = ["compute.googleapis.com"]
  credentials_path  = "${local.credentials_file_path}"
  group_name        = "testgroup"
  group_role        = "roles/owner"
}

Summary

The project-factory can use a service account with a misconfigured host project, which leads to a deadlocked Terraform configuration with confusing errors.

Description

The project factory requires that the host project associated with an instantiating service account has certain APIs enabled, but the project factory does not verify that these APIs are enabled before generating a new project. This can cause the project factory to partially generate a new project and then fail, and subsequent calls to terraform apply will fail with a number of related but unhelpful errors.

When terraform apply is run with the misconfigured environment, it will create a new project, attempt to associate a billing account with that project, and fail since cloudbilling.googleapis.com is disabled. Following runs of terraform apply will see that the google_project was created and then attempt to look up the default compute service account with the google_service_account data source. This will fail because the compute.googleapis.com API is not enabled, but this hides the more interesting errors - that the service project doesn't have a billing account enabled, and that the host project doesn't have the Cloud Billing API enabled and thus can't remedy the issue through Terraform.

Expected behavior

When the project-factory runs with a service account whose host project doesn't have the right APIs enabled, it terminates before generating resources.

Actual behavior

First invocation of terraform apply:

module.project-factory.google_project.project: Still creating... (10s elapsed)

Error: Error applying plan:

1 error(s) occurred:

* module.project-factory.google_project.project: 1 error(s) occurred:

* google_project.project: Error setting billing account "<BILLING ACCOUNT>" for project "projects/tftest-1-4901": googleapi: Error 403: Cloud Billing API has not been used in project <HOST NUMBER> before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudbilling.googleapis.com/overview?project=<HOST NUMBER> then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry., accessNotConfigured

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

Note: that the error generated references the host project number on the first failed run.

Running terraform plan or terraform apply

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

random_id.random_project_id_suffix: Refreshing state... (ID: SQE)
data.google_organization.org: Refreshing state...
google_project.project: Refreshing state... (ID: tftest-1-4901)
data.null_data_source.data_given_group_email: Refreshing state...
data.null_data_source.data_final_group_email: Refreshing state...
data.null_data_source.data_group_email_format: Refreshing state...
data.google_compute_default_service_account.default: Refreshing state...

Error: Error refreshing state: 1 error(s) occurred:

* module.project-factory.data.google_compute_default_service_account.default: 1 error(s) occurred:

* module.project-factory.data.google_compute_default_service_account.default: data.google_compute_default_service_account.default: Error reading GCE service account not found: googleapi: Error 403: Project <SERVICE NUMBER> is not found and cannot be used for API calls. If it is recently created, enable Compute Engine API by visiting https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project=<SERVICE NUMBER> then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry., accessNotConfigured

Note: in subsequent runs the error references the service project.

Steps to reproduce

note: these steps are approximate and should be checked to make sure that this reproduces correctly.

1. Create a new host project, but don't enable any APIs.
2. Create a service account within that project with helpers/setup-sa.sh.

• This creates a service account with the correct rights to create projects, but without the APIs enabled that it needs to work

1. Create a terraform configuration with the project factory.
2. Run terraform apply; the command should fail.
3. Verify that terraform can no longer generate a plan with terraform plan

Workarounds

If a terraform configuration is deadlocked in the describe manner, the correct APIs can be enabled retroactively and terraform will be able to run.

1. Enable cloudbilling.googleapis.com on the host project through the Google Cloud console.
2. Enable compute.googleapis.com on the service project through the Google Cloud console.
   • Using the console to enable this API will automatically prompt the user to add a billing account.
3. (Maybe) enable serviceusage.googleapis.com on the host project?

So i would like to replace an existing secondary ip range but this fails to work.

Terraform change

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ google_compute_subnetwork.network-with-private-secondary-ip-ranges
      secondary_ip_range.4.ip_cidr_range: "192.168.50.0/24" => "192.168.60.0/24"
      secondary_ip_range.4.range_name:    "secondary-range-50" => "secondary-range-60"


Plan: 0 to add, 1 to change, 0 to destroy.

------------------------------------------------------------------------

Error produced

googleapi: Error 400: Invalid value for field 'resource.secondaryIpRanges': ''. Cannot add and remove secondary IP ranges in the same request. Added ranges: [192.168.60.0/24]. 
Removed ranges: [192.168.50.0/24]., invalid

As this error is from the Google Api, can you please change the way terraform applies this change by deleting first and then adding the change.

Getting the following on a number of example projects in the proj factory:

Error: module.project-factory.module.project-factory.null_resource.preconditions: : invalid or unknown key: environment