We should add validation that either org_id is set or folder_id is set, and fail early otherwise.

This issue is based on #14 (comment).

Systematically adding the email_violation block to each notifier makes Forseti's forseti notifiers run fail if sendgrid_api_key is not set to a valid key.

Email notifications should stay optional and only enabled if sendgrid_api_key is not empty.

#14 (comment)

Hi, I have found firewall difference between this terraform installation and original forseti source.
Original source have firewall rules which are named "forset-client-allow-ssh-external" and "forseti-client-deny-all". But this terraform plugin doesn't have.

https://github.com/terraform-google-modules/terraform-google-forseti/blob/master/modules/client/main.tf

https://github.com/GoogleCloudPlatform/forseti-security/blob/dev/install/gcp/installer/forseti_client_installer.py

It this intentional?

depends_on on module.server_rules for the server VM means everytime a user changes a rule file (frequent when starting with Forseti), the whole VM gets re-created.

There is no need for this: ideally what should happen is that we run gsutil cp -R <BUCKET>/rules <FORSETI_HOME>/rules and then forseti server configuration reload. Simplier, we can just restart the server VM so that the startup script is re-executed when changing configs.

Organization Administrator is a role that can potentially be very destructive.

We need to identify a subset of permissions that are really needed to create Forseti, and possibly help create a custom role that might have only those permissions (in case users are not comfortable with granting the wider role).

So far, we've identified that the Organization Administrator role can be replaced with a custom role with the following permissions:

• resourcemanager.projects.getIamPolicy
• resourcemanager.organizations.getIamPolicy
• resourcemanager.organizations.setIamPolicy
• resourcemanager.folders.getIamPolicy
• resourcemanager.folders.setIamPolicy

#159 ignores errors during kitchen destroy to work around #148, but this means that some resources are not being cleaned up by the pipeline. We will need to explicitly clean those resources separately until #148 is fixed.

When passing folder_id without org_id, we should be able to create a Forseti that:

• Scans only the folder identified by folder_id (already done with root_resource_id)
• Only grants permissions on the folder level

Currently, when passing both flags, all the organization-level roles are still granted (most are not needed since Forseti will be bound to a folder). This can be a blocker for companies where separation of business units at the top-level is important.

I'm wondering if Forseti can work without any organization permissions, but it should be (at minimum the service accounts should have limited org-level roles if folder_id is passed).

Is Debian the only supported distribution? The default image being used appears to be Ubuntu 18.04 rather than Debian, but the description of the variable seems to suggest otherwise - https://github.com/terraform-google-modules/terraform-google-forseti/blob/master/variables.tf#L73

Did a fresh install using v1.4.1 and inventory complained about not having groupssettings.googleapis.com enabled.

Currently, only specific notifiers like IAM and KMS have Slack as a configuration option for notifications.

The DM dependency looks weird to me knowing that DM is just another provisioning tool like Terraform.

Should we move provisioning of Forseti resources from Deployment Manager to native Terraform resources (e.g: CloudSQL instance, database, VMs and service accounts) instead of relying on the existing deployment manager scripts ?

Add in functionality of composite root found here: forseti-security/forseti-security#2596

Deployment Manager Forseti Installer will install Forseti using composite root with extra flag
--composite-root-resources="projects/project_id"

Need to manage new IAM roles and change the server configuration:
If a user has project owner role for project A, they should be able to install forseti directly on project A if the root resource is project A. The installer / terraform will go in and grant the server account all the project level viewer roles so that it can inventory the project, and not on an org level.

Similar to the README of the root module, the READMEs of the example modules should define the required permissions and APIs.

When modifying something on the server VM (such as the server region) - this triggers delete / create of the server VM but doesn't change the server_ip in the client config. This makes the client time out when running Forseti command as it's trying to contact the old server IP.

forseti-security/forseti-security#2395

Sendgrid should be made truly optional.

Currently, the module requires passing a path to a Service Account key as an argument. This can make it harder to run as part of a pipeline where (a) an environment variable is used to track the file location (b) the application default credentials on a GCE VM should be used.

@adrienthebo this commit introduced a bug which makes the root_resource_id set to "" instead of organizations/<ORG_ID> because the default value is [""] and thus has 1 element.

We need to change the default value to [] instead.

Users should be able to enable and configure the GCS bucket for the GCP Cloud Security Command Center (CSCC) violations.

        enabled: ${enable_cscc_bucket}
        gcs_path: gs://${storage_bucket_name}/scanner_violations_cscc

cf:

• #14 (comment)
• #14 (comment 2)

The Forseti server TCP rule currently allows any VM in the network to make gRPC calls to the server (allowed range is the ip range for the subnetwork where Forseti is deployed): we can limit that firewall rule to have the forseti-client service account as the source; since only the client VM will make gRPC calls to the server VM.

When attempting to configure Forseti as follows:

module "forseti" {
	source = "github.com/terraform-google-modules/terraform-google-forseti"
	download_forseti = "true"
	gcs_location = "us-central1"
	cloud_sql_region = "us-central1"
	sendgrid_api_key = "<REDACTED>"
	notification_recipient_email = "[email protected]"
	gsuite_admin_email = "[email protected]"
	project_id = "${module.forseti-project.project_id}"
	credentials_file_path = "../credentials/terraform.json"
}

I receive an error message as follows:

Error: Error applying plan:

1 error(s) occurred:

* module.forseti.null_resource.execute_forseti: Error running command 'cd forseti-security; python install/gcp_installer.py --no-cloudshell --service-account-key-file ../credentials/terraform.json --gsuite-superadmin-email [email protected] --sendgrid-api-key <REDACTED> --cloudsql-region us-central1 --notification-recipient-email [email protected] --gcs-location us-central1': exit status 1. Output: Read gcloud info: Success
VPC Host Project barefoot-forseti

################################################################################
#  Installing Forseti Server                                                   #
################################################################################


+-------------------------------------------------------------------------------
|  Pre-installation checks
+-------------------------------------------------------------------------------

Dry run: False
Advanced mode: False
Read gcloud info: Success
Current gcloud version: 216.0.0
Gcloud alpha components: True
You are: [email protected]
Project id: barefoot-forseti
Organization id: 571656855759
Service account activated
Billing: Enabled

+-------------------------------------------------------------------------------
|  Configuring GSuite Admin Information
+-------------------------------------------------------------------------------

To read G Suite Groups and Users data, please provide a G Suite super admin email address. This step is optional.
See https://forsetisecurity.org/docs/latest/setup/install.html to know what will not work without G Suite integration.

Email: Traceback (most recent call last):
  File "install/gcp_installer.py", line 23, in <module>
    gcp_installer.run()
  File "/Users/jberlinsky/src/clients/BarefootCoders/google-infrastructure/forseti-security/install/gcp/gcp_installer.py", line 125, in run
    instructions = forseti_server.run_setup(final_setup=False)
  File "/Users/jberlinsky/src/clients/BarefootCoders/google-infrastructure/forseti-security/install/gcp/installer/forseti_installer.py", line 125, in run_setup
    self.preflight_checks()
  File "/Users/jberlinsky/src/clients/BarefootCoders/google-infrastructure/forseti-security/install/gcp/installer/forseti_server_installer.py", line 60, in preflight_checks
    self.get_email_settings()
  File "/Users/jberlinsky/src/clients/BarefootCoders/google-infrastructure/forseti-security/install/gcp/installer/forseti_server_installer.py", line 360, in get_email_settings
    constants.QUESTION_GSUITE_SUPERADMIN_EMAIL).strip()
EOFError: EOF when reading a line


Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

At a very quick glance, it looks like this line is trying to collect data from STDIN, which null_resource.execute_forseti is passing as a CLI flag. It looks like this changed on or about 8/23/2018, in this commit.

Temporary Resolution: If I go into the cloned forseti-security/ repository and check out the prior commit, forseti-security/forseti-security@87e42da, I'm able to proceed with deploying Forseti.

Suggested Fix: This seems to be an issue with Forseti itself, but it also seems unnecessarily restrictive to limit Forseti checkouts to just branch names. Perhaps an appropriate first step here would be allowing all git refs (including SHAs and named branches) to be used in the forseti_repo_branch variable. From there, some GH Issues should probably be filed with the Forseti project.

e.g. v.2.13.0 -> v2.13.1

patches are mainly for bug fixes so auto pickup would be more user friendly

The setup helper (/helpers/setup.sh), in line 86 has a remove iam policy binding for a newly created service account, which didn't have that policy in the first place, This causes the script to complain that the service account doesn't have the relevant permissions. I believe it should be replaced by an add like all the other bindings in the file:

86 gcloud projects remove-iam-policy-binding ${PROJECT_ID} \
87    --member="serviceAccount:${SERVICE_ACCOUNT_ID}" \
89    --role="roles/iam.serviceAccountUser" \
90    --user-output-enabled false

Some deployments might require forseti server access to be only through private IP. It would be good if the server can be created without access_config parameter for these cases. This can be controlled via private_server or private_client variable

Currently we have only this firewall rule which opens the port of 50051, but Config Validator server use the port 50052. Please add firewall rule to open 50052 as well.

resource "google_compute_firewall" "forseti-server-allow-grpc" {
name = "forseti-server-allow-grpc-${local.random_hash}"
project = "${local.network_project}"
network = "${var.network}"
target_service_accounts = ["${google_service_account.forseti_server.email}"]
source_ranges = "${var.server_grpc_allow_ranges}"
priority = "100"

allow {
protocol = "tcp"
ports = ["50051"]
}

depends_on = ["null_resource.services-dependency"]
}

Hi guys,

I used the input variable
cloudsql_db_name = "my-custom-db-name"
to change the db name that gets used. This gets configured correctly in the settings and startup script for the server, especially here:

But it is not respected for the definition of the cloudsql db here:

This led to an non working setup after running the module. I would suggest to fix that, if you want I can do a PR.

Regards,
Simon

Right now it requires users to input the ORG_ID manually, would be nice to grab it from PROJECT_ID.

See a discussion on this here: https://github.com/terraform-google-modules/terraform-google-forseti/pull/2/files/c6a9ac706fd60fc0179dfdd4870f3b54cb0aa046#r205215428

How should I handle creation of custom rules? For example, the Service Account Key rule currently is set to 100 days in the rules module, but what if I want that to be 90?

Should I send a pull request to update this module with another variable or is there some way I can add custom rules? Fork this repo and start my own custom module?

My ideal process would be that when a config rule changes in source, it would start a chain of events which would update that rule in the bucket and have forseti re-run the scanners.

The cloudsql instance which is created in module.forseti.module.server.google_sql_database_instance.master has disk_autoresize: "true". This is all well and good. However this means that future terraform applys attempt to delete and recreate the database in order to return the disk size to the original 25GB. This is both i) quite destructive and ii) doomed to failure because of GCP's soft deletion rules.

Would it be possible for a suitable ignore_changes entry to be added to this resource's lifecycle?

#14 (comment)

I ran into an error while running tf destroy on a simple Forseti deployment:

Error: Error applying plan:

1 error(s) occurred:

* module.forseti-install-simple.module.server.google_sql_user.root (destroy): 1 error(s) occurred:

* google_sql_user.root: Error, failed to deleteuser root in instance forseti-server-db-ce89442d: googleapi: Error 400: Missing parameter: host., required

Another tf destroy afterwards also gave me some weird errors:

11 error(s) occurred:

* module.forseti-install-simple.module.server.output.forseti-server-storage-bucket: variable "server_config" is nil, but no error was reported
* module.forseti-install-simple.module.client.output.forseti-client-vm-public-ip: Resource 'google_compute_instance.forseti-client' does not have attribute 'network_interface.0
.access_config.0.nat_ip' for variable 'google_compute_instance.forseti-client.network_interface.0.access_config.0.nat_ip'
* module.forseti-install-simple.local.random_hash: local.random_hash: variable "random_hash_suffix" is nil, but no error was reported
* module.forseti-install-simple.module.server.output.forseti-server-vm-ip: variable "forseti-server" is nil, but no error was reported
* module.forseti-install-simple.module.client.output.forseti-client-storage-bucket: Resource 'google_storage_bucket.client_config' does not have attribute 'id' for variable 'go
ogle_storage_bucket.client_config.id'
* module.forseti-install-simple.module.server.output.forseti-server-service-account: variable "forseti_server" is nil, but no error was reported

* module.forseti-install-simple.module.client.output.forseti-client-vm-name: Resource 'google_compute_instance.forseti-client' does not have attribute 'name' for variable 'goog
le_compute_instance.forseti-client.name'
* module.forseti-install-simple.module.server.output.forseti-server-vm-name: variable "forseti-server" is nil, but no error was reported
* module.forseti-install-simple.module.server.output.forseti-server-vm-public-ip: variable "forseti-server" is nil, but no error was reported
* module.forseti-install-simple.module.client.output.forseti-client-service-account: Resource 'google_service_account.forseti_client' does not have attribute 'email' for variab
le 'google_service_account.forseti_client.email'
* module.forseti-install-simple.module.client.output.forseti-client-vm-ip: Resource 'google_compute_instance.forseti-client' does not have attribute 'network_interface.0.networ
k_ip' for variable 'google_compute_instance.forseti-client.network_interface.0.network_ip'

The accScript.sh uses the GNU version of sed. If running this on a Mac, the apply will fail due to the default sed is a different (BSD) flavor.

A quick workaround is to install gsed using 'brew install gnu-sed' and updating the accScript to use the gsed command instead of sed

The README.md should be updated to reflect this

See forseti-security/forseti-security#2577

Startup scripts failed to get executed on a fresh deployment

Created a default terraform plan and all went well, 71 resources created, and no errors but when I login to the server/client, there's no forseti service nor forseti_security directory under /home/ubuntu or anywhere. Is this related to this bug? https://stackoverflow.com/questions/50422205/gce-startup-script-fails-to-run-on-ubuntu-18-04

module "forseti" {
  source  = "terraform-google-modules/forseti/google"
  version = "1.0.0"

  gsuite_admin_email = "[email protected]"
  domain             = "mydomain.com"
  project_id         = "myproject"
  org_id             = "7777777777777"
}

terraform output

...
module.forseti.client.google_compute_instance.forseti-client: Still creating... (10s elapsed)
module.forseti.client.google_compute_instance.forseti-client: Still creating... (20s elapsed)
module.forseti.module.client.google_compute_instance.forseti-client: Creation complete after 22s (ID: forseti-client-vm-xxxxx)

Apply complete! Resources: 71 added, 0 changed, 0 destroyed.

We should add test coverage on the forseti client VM to verify that Forseti can produce scans:

• forseti config show works and points to the correct server address and port
• forseti inventory create can generate an inventory
• forseti scanner run can run successfully

This issue applies to #14.

The main.tf file is expecting the accScript.sh file in the scripts folder:

37 launch_first = "sh scripts/accScript.sh ${var.gsuite_admin_email}; "

And at the moment, the accScript.sh file is in the root folder:

drwxr-xr-x 3 user user  4096 Nov  6 11:46 examples
drwxr-xr-x 2 user user  4096 Nov  7 10:06 helpers
-rw-r--r-- 1 user user 11357 Nov  6 11:46 LICENSE
-rw-r--r-- 1 user user  3975 Nov  6 11:46 main.tf
-rw-r--r-- 1 user user  2011 Nov  6 11:46 Makefile
-rw-r--r-- 1 user user   781 Nov  6 11:46 outputs.tf
-rw-r--r-- 1 user user  6704 Nov  6 11:46 README.md
drwxr-xr-x 2 user user  4096 Nov  6 11:46 scripts
drwxr-xr-x 3 user user  4096 Nov  6 11:46 test
-rw-r--r-- 1 user user  1831 Nov  6 11:46 variables.tf```

The output variable for the random-hash suffix does not appear to resolve correctly.

I see the suffix as an output in outputs.tf:

output "suffix" {
  description = "The random suffix appended to Forseti resources"
  value       = "${local.random_hash}"
}

However, when I try to reference it from my forseti-install module via ${module.forseti-install.suffix}I get:

"suffix" is not a valid output for module "forseti-install"

If cscc_violations_enabled is set to "true", then maybe we should:

• Enable the Cloud Command Security Center API (securitycenter.googleapis.com) in the Forseti project
• Grant the roles/securityCenter.editor role to the Forseti server service account

Those are two manual steps that are still required when setting up the integration.

There is a Proof of Concept hosting Forseti on GKE (targeted for v2.13 release)

Ref. forseti-security/forseti-security#2162 (comment)

Anyone with terraform experience willing to PoC a deployment to GKE?

It would be a matter of replacing the client and server vm deployment sections with the GKE deployment
and possibly tweaking firewall rules to ensure its secure.

A rough draft of a bash script deployment as example:
https://github.com/GoogleCloudPlatform/forseti-security/blob/docker-poc/install/scripts/k8s_setup_forseti.sh

Related to #165 and #148.

The current CI pipeline experiences consistent failures unrelated to changes being tested. Stale resources leftover in the test projects are a large part of this problem.

The CI pipeline should be updated to create a new project for each build. This will involve migrating the existing test fixture code from https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit to this repository. An added bonus of this change will be providing a mechanism for creating consistent test environments for developers.

to track the initialization of the VM (startup script output)

The client and server VM have the goog-dm labels, which appear to be an artifact from the Deployment Manager based approach used in the v1 release of this module. We should remove them.

See #14 (comment)

We need to:

• make the USER=ubuntu configurable in startup scripts.
• make Forseti's path configurable (atm. hardcoded to /home/ubuntu/forseti-security).
• See #14 comment

Users should be able to choose their own user on their VM, and change the path where Forseti should be installed.

In order to support deploying forseti-security on GKE, this module should deploy a GKE cluster. It should have the following features.

1. Deploy forseti-security via Helm
2. If the user already has a GKE cluster they want to use, don't deploy a new GKE cluster, and don't deploy a server VM
3. If the user doesn't have a GKE cluster, deploy a GKE cluster with hardening best practices, and don't deploy a server VM
4. If the user doesn't want to deploy on GKE, deploy the server VM

• Terra-form sub-module - For Helm to function on GKE (BETA)
• Terra-form changes for optional deployment of Server VM (GA)
• Terra-form changes for optional deployment of Client VM (GA)

The tf helper doesn't grant the appropriate roles in the following sections:

https://github.com/terraform-google-modules/terraform-google-forseti/blob/b7f70a6b63a894ea60ab73834e8b02ed58b260c0/helpers/forseti-setup.tf#L48-L52

https://github.com/terraform-google-modules/terraform-google-forseti/blob/b7f70a6b63a894ea60ab73834e8b02ed58b260c0/helpers/forseti-setup.tf#L54-L58

It looks like the new version of Terraform made some breaking changes to the syntax. Indeed when I tried to use this module with the new version, I quickly ran into errors I wasn't sure how to resolve.

I ran the included migration tool (I committed the results here), and am still receiving errors:

Error: Incorrect condition type

  on terraform-google-forseti/modules/server/main.tf line 252, in data "template_file" "forseti_server_config":
 252:     GROUPS_SETTINGS_DISABLE_POLLING                     = var.groups_settings_disable_polling ? "true" : "false"
    |----------------
    | var.groups_settings_disable_polling is "False"

The condition expression must be of type bool.

Are there plans to add support for the new version?

If CSCC is configured, then the Forseti server service account needs the org role binding roles/securitycenter.findingsEditor.

Currently, users will have to create this IAM binding outside of the module with:

resource "google_organization_iam_member" "cscc" {
  org_id = "${var.org_id}"
  role   = "roles/securitycenter.findingsEditor"
  member = "serviceAccount:${module.forseti.forseti-server-service-account}"
}