Describe the bug
The rover login command fails when trying to login after following the instructions at Getting started with Azure Cloud Adoption Framework landing zones for Terraform

vscode@f912543adaf1:/tf/caf$ rover login

  /$$$$$$   /$$$$$$  /$$$$$$$$       /$$$$$$$
 /$$__  $$ /$$__  $$| $$_____/      | $$__  $$
| $$  \__/| $$  \ $$| $$            | $$  \ $$  /$$$$$$  /$$    /$$/$$$$$$   /$$$$$$
| $$      | $$$$$$$$| $$$$$         | $$$$$$$/ /$$__  $$|  $$  /$$/$$__  $$ /$$__  $$
| $$      | $$__  $$| $$__/         | $$__  $$| $$  \ $$ \  $$/$$/ $$$$$$$$| $$  \__/
| $$    $$| $$  | $$| $$            | $$  \ $$| $$  | $$  \  $$$/| $$_____/| $$
|  $$$$$$/| $$  | $$| $$            | $$  | $$|  $$$$$$/   \  $/ |  $$$$$$$| $$
 \______/ |__/  |__/|__/            |__/  |__/ \______/     \_/   \_______/|__/


              version: aztfmod/rover:1.0.1-2106.3012

@calling verify_azure_session

Checking existing Azure session
ERROR: Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /common/oauth2/devicecode?api-version=1.0 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fb6d23e4e50>: Failed to establish a new connection: [Errno -5] No address associated with hostname'))
Error on or near line 206; exiting with status 1
Error on or near line 206; exiting with status 1

@calling clean_up_variables
cleanup variables
clean_up backend_files

Even az login command fails, ore better, I can successfully login via browser, but the command fails with the following error:

vscode@f912543adaf1:/tf/caf$ az login 
The default web browser has been opened at https://login.microsoftonline.com/common/oauth2/authorize. Please continue the login in the web browser. If no web browser is available or if the web browser fails to open, use device code flow with `az login --use-device-code`.
Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /common/oauth2/token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fc94e5c0130>: Failed to establish a new connection: [Errno -5] No address associated with hostname'))

Also, note that the folder name shown close to the green button in VS Code lower bar is not Azure CAF rover, but CAF Starter Terraform landing zones.

To Reproduce
1.I cloned the repo as shown in the instructions using git clone https://github.com/Azure/caf-terraform-landingzones-starter.git
2.Opened the folder in VS Code
3.Clicked the green button and selected Open folder in container...
4.Run rover login in the prompt command

Expected behavior
I would expect to successfully login.

Screenshots
See the text above.

Configuration (please complete the following information):

• OS and version: Windows 10
• Version of the rover: version: aztfmod/rover:1.0.1-2106.3012
• Version of the landing zone: lastest

Hi,

Thanks for the awesome work, can we have a config example of level 1 deployment?

Thanks

Describe the bug
I have an error when I try to instantiate version pipelines in sandpit folder

Terraform returned errors:

Error: authorization.RoleDefinitionsClient#CreateOrUpdate: Failure responding to request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=409 Code="RoleDefinitionWithSameNameExists" Message="A role definition cannot be updated with a name that already exists."

on /home/vscode/.terraform.cache/modules/launchpad/modules/roles/custom_roles/module.tf line 13, in resource "azurerm_role_definition" "custom_role":
13: resource "azurerm_role_definition" "custom_role" {

Error: authorization.RoleDefinitionsClient#CreateOrUpdate: Failure responding to request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=409 Code="RoleDefinitionWithSameNameExists" Message="A role definition cannot be updated with a name that already exists."

on /home/vscode/.terraform.cache/modules/launchpad/modules/roles/custom_roles/module.tf line 13, in resource "azurerm_role_definition" "custom_role":
13: resource "azurerm_role_definition" "custom_role" {

Error on or near line 473: Error running terraform apply; exiting with status 2001

configuration/sandpit/pipelines/README-pipelines.md

step 2.1

When

To Reproduce
environment=sandpit

rover -lz /tf/caf/public/landingzones/caf_launchpad
-var-folder /tf/caf/configuration/${environment}/level0/launchpad
-parallelism 30
-level level0
-env ${environment}
-launchpad
-a apply

Expected behavior
Create the launchpad

Screenshots

Terraform returned errors:

Develop Terraform based solution for the Constoso reference architecture :

https://github.com/Azure/Enterprise-Scale/tree/main/docs/reference/contoso

The current Api version is:
rbac.authorization.k8s.io/v1beta1

The following Warning appears due to this:
ClusterRole is deprecated in v1.17+, unavailable in v1.22+

The Api version needs to be updated to:
rbac.authorization.k8s.io/v1

Hello,
Any ETA about migration on terraform 1 ?

Thanks

Either add a Deny All to existing AKS NSG rules, Flow logs and test
Or remove all and use default NSG rules for all AKS Subnets

Describe the bug
Walking through the https://github.com/Azure/caf-terraform-landingzones-starter/blob/starter/configuration/sandpit/pipelines/README-pipelines.md instructions, there is not a clear step by step guide that works from downloading the starter project to running the pipelines within a DevOps environment. The instructions feels like you should already have knowledge how to do some steps.

To Reproduce
Simple things are missing such as

• Step to pull the public landingzone project in.
• Authenticate to your Azure environment
• Changes to make in configuration files. (e.g, regions)
• Steps to create Admin and Agent PAT
• Steps to actually run pipelines in DevOps.

Expected behavior
A read me document that:

1. Requires little to no previous experience to follow and run.
2. Simple to follow, with correct url's, folder links.

When executing the AKS deployment on the step eval terraform apply ${parameter_files} (or plan) this results in the following:

│ Error: Output refers to sensitive values
│
│   on .terraform/modules/caf/modules/compute/aks/output.tf line 39:
│   39: output kube_admin_config_raw {
│
│ Expressions used in outputs can only refer to sensitive values if the sensitive attribute is true.

Process cannot continue.

Using latest GA terraform cli. @mosabami is receiving the same thing.

Hi team,

Can you please add -var-folder /tf/caf/configuration/demo/level1/foundations to demo Enterprise scale?

Regards,
Jorge Arteiro

Describe the bug
There is no policy for ACR integration in the RI.

Expected behavior
For ACR integration and following our best best practices we will need to Create policy within our RI for AKS to only allow images to be pulled from ACR

Add documentation and sample code for sandpit setup:

• Mono subscription
• Adding diagnostics
• Adding Azure AD groups for team collaboration
• Add Virtual WAN topology setup
• Simple pipelines (optional)

Current AKS baseline example deploys cluster baseline configuration with Flux V1. Flux V1 is on path for deprecation. New Flux v2 address gaps from V1 along with useful new features . The Flux V2 operator can be used to deploy workloads using GitOps.

Terraform provider flux can be used to deploy Flux on Kubernetes cluster.

The adminGroupObjectIDs is not being set in our walkthrough. No admin groups associated, had to do it by hand after. If the instructions should have included setting that group name, in aks.tfvars before we deploy, we probably should add that. Or at least call out what the group must be named.

This is because the aks.tfvars is trying to look up by name, but we don't have permissions to query AD groups, it works if we set it to 7304e4e7-b148-4ada-a135-6049c702d21e (no query needed)

When deploying Sandpit configuration, in current configuration, you need to have AAD privileges, this is used to create AAD group that are needed when working as a DevOps team, but might not be required for first experience.

We want to comment this configuration so you are able to spin up sandpit samples and pipelines even if you don't have AAD advanced permission.

Organize into platform and app config folders.
Standardize folder names and *.tfstate file names to match.
Ensure deployments are simple and can be deployed in a reasonable timeframe.

When using command provided in https://github.com/Azure/caf-terraform-landingzones-starter/blob/starter/configuration/demo/README.md for deploying the launchpad (1. Launchpad-level0 landing zones -> Deploy the launchpad), I get the following:

$ export environment=demo
$ rover -lz /tf/caf/public/landingzones/caf_launchpad \
  -var-folder /tf/caf/configuration/${environment}/level0 \
  -parallelism 30 \
  -level level0 \
  -env ${environment} \
  -launchpad \
  -a plan
...
var.dynamic_keyvault_secrets
  Enter a value:

In order to make it correctly I changed the value of -var-folder in the above to /tf/caf/configuration/${environment}/level0/launchpad so it could correctly find the vars.

Seems like an easy fix but I'm new to rover and the structure of this project so opted to submit this issue instead for anyone else that may come across this.

Describe the bug
A clear and concise description of what the bug is.

Terraform plan return code: 0
Terraform returned errors:

Error: Unsupported block type

  on /home/vscode/.terraform.cache/modules/launchpad/modules/security/keyvault/keyvault.tf line 50, in resource "azurerm_key_vault" "keyvault":
  50:   dynamic "contact" {

Blocks of type "contact" are not expected here.

Error on or near line 446: Error running terraform plan; exiting with status 2000

To Reproduce
Steps to reproduce the behavior:

1. git clone https://github.com/Azure/caf-terraform-landingzones-starter.git
2. Reopen in container
3. git clone --branch 2010.0.0 https://github.com/Azure/caf-terraform-landingzones.git /tf/caf/public
4. rover login
5. rover -lz /tf/caf/public/landingzones/caf_launchpad -launchpad -var-folder /tf/caf/configuration/demo/level0/launchpad -a apply
6. See error

Expected behavior
Expecting the launchpad to deploy.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS: Windows 10 with WSL2 Ubuntu 20.04
• Rover Version: aztfmod/rover:2010.2808

Additional context
Following along with the getting started video here: https://www.youtube.com/watch?v=M5BXm30IpdY

Initializing provider plugins...

• Finding hashicorp/random versions matching "~> 2.2.1"...
• Finding hashicorp/external versions matching "~> 1.2.0"...
• Finding hashicorp/null versions matching "~> 2.1.0"...
• Finding hashicorp/tls versions matching "~> 2.2.0"...
• Finding aztfmod/azurecaf versions matching "~> 1.2.0"...
• Finding latest version of hashicorp/time...
• Finding hashicorp/azurerm versions matching "~> 2.55.0"...
• Finding hashicorp/azuread versions matching "~> 1.4.0"...
• Finding latest version of hashicorp/template...
  ╷
  │ Error: Failed to install provider
  │
  │ Error while installing hashicorp/azuread v1.4.0: could not query provider registry for registry.terraform.io/hashicorp/azuread: failed to retrieve
  │ authentication checksums for provider: the request failed after 2 attempts, please try again later: Get
  │ "https://releases.hashicorp.com/terraform-provider-azuread/1.4.0/terraform-provider-azuread_1.4.0_SHA256SUMS": x509: certificate signed by unknown
  │ authority
  ╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/template v2.2.0: could not query provider registry for registry.terraform.io/hashicorp/template: failed to retrieve
│ authentication checksums for provider: the request failed after 2 attempts, please try again later: Get
│ "https://releases.hashicorp.com/terraform-provider-template/2.2.0/terraform-provider-template_2.2.0_SHA256SUMS": x509: certificate signed by unknown
│ authority
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/random v2.2.1: could not query provider registry for registry.terraform.io/hashicorp/random: failed to retrieve
│ authentication checksums for provider: the request failed after 2 attempts, please try again later: Get
│ "https://releases.hashicorp.com/terraform-provider-random/2.2.1/terraform-provider-random_2.2.1_SHA256SUMS": x509: certificate signed by unknown
│ authority
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/external v1.2.0: could not query provider registry for registry.terraform.io/hashicorp/external: failed to retrieve
│ authentication checksums for provider: the request failed after 2 attempts, please try again later: Get
│ "https://releases.hashicorp.com/terraform-provider-external/1.2.0/terraform-provider-external_1.2.0_SHA256SUMS": x509: certificate signed by unknown
│ authority
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/null v2.1.2: could not query provider registry for registry.terraform.io/hashicorp/null: failed to retrieve
│ authentication checksums for provider: the request failed after 2 attempts, please try again later: Get
│ "https://releases.hashicorp.com/terraform-provider-null/2.1.2/terraform-provider-null_2.1.2_SHA256SUMS": x509: certificate signed by unknown
│ authority
╵

╷
│ Error: Failed to install provider
│
│ Error while installing aztfmod/azurecaf v1.2.3: could not query provider registry for registry.terraform.io/aztfmod/azurecaf: failed to retrieve
│ authentication checksums for provider: the request failed after 2 attempts, please try again later: Get
│ "https://github.com/aztfmod/terraform-provider-azurecaf/releases/download/v1.2.3/terraform-provider-azurecaf_1.2.3_SHA256SUMS": x509: certificate
│ signed by unknown authority
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/azurerm v2.55.0: could not query provider registry for registry.terraform.io/hashicorp/azurerm: failed to retrieve
│ authentication checksums for provider: the request failed after 2 attempts, please try again later: Get
│ "https://releases.hashicorp.com/terraform-provider-azurerm/2.55.0/terraform-provider-azurerm_2.55.0_SHA256SUMS": x509: certificate signed by unknown
│ authority
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/tls v2.2.0: could not query provider registry for registry.terraform.io/hashicorp/tls: failed to retrieve
│ authentication checksums for provider: the request failed after 2 attempts, please try again later: Get
│ "https://releases.hashicorp.com/terraform-provider-tls/2.2.0/terraform-provider-tls_2.2.0_SHA256SUMS": x509: certificate signed by unknown authority
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/time v0.7.1: could not query provider registry for registry.terraform.io/hashicorp/time: failed to retrieve
│ authentication checksums for provider: the request failed after 2 attempts, please try again later: Get
│ "https://releases.hashicorp.com/terraform-provider-time/0.7.1/terraform-provider-time_0.7.1_SHA256SUMS": x509: certificate signed by unknown
│ authority
╵

Error on or near line 23; exiting with status 1

@calling clean_up_variables
cleanup variables
clean_up backend_files

I ran into an error when trying to deploy infrastructure with just default parameters:

_module.caf.module.application_gateways["agw1_az1"].azurerm_application_gateway.agw: Creation complete after 15m2s [id=/subscriptions/203633e9-0e19-48c0-b142-64922c37d994/resourceGroups/rjhi-rg-agw-re1/providers/Microsoft.Network/applicationGateways/rjhi-agw-app_gateway]

**Error: creating Managed Kubernetes Cluster "rjhi-aks-akscluster-re1-001" (Resource Group "rjhi-rg-aks-re1"): containerservice.ManagedClustersClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="RouteTableMissingDefaultRouteError" Message="Default route 0.0.0.0/0 missing from route table /subscriptions/203633e9-0e19-48c0-b142-64922c37d994/resourceGroups/rjhi-rg-aks_spoke_re1/providers/Microsoft.Network/routeTables/rjhi-route-default_to_firewall_re1."

on .terraform/modules/caf/modules/compute/aks/aks.tf line 40, in resource "azurerm_kubernetes_cluster" "aks":
40: resource "azurerm_kubernetes_cluster" "aks" {**_

I'm following instructions on this page after I run "eval terraform apply ${parameter_files}"
https://github.com/Azure/caf-terraform-landingzones-starter/blob/starter/enterprise_scale/construction_sets/aks/online/aks_secure_baseline/01-terraform.md

Add CI for caf LZ and solutions.

add azurerm_ip_group to only allow AKS-system & AKS_nodepool subnet to egress.
May be jumpbox too for kubectl.

** Describe the bug **
have a Private subnet in our RI even though it is not in use by any required resource

Expected behavior
Remove the Private subnet and its NSG from our RI

Describe the bug

│ Error: Error loading state error
│ 
│   with data.terraform_remote_state.remote["foundations"],
│   on locals.remote_tfstates.tf line 19, in data "terraform_remote_state" "remote":
│   19:   backend = var.landingzone.backend_type
│ 
│ error loading the remote state: failed to lock azure state: 2 errors
│ occurred:
│       * blobs.Client#AcquireLease: Failure sending request: StatusCode=409 --
│ Original Error: Error occurred unmarshalling JSON - Error = 'invalid
│ character '<' looking for beginning of value' JSON = '<?xml version="1.0"
│ encoding="utf-8"?><Error><Code>LeaseAlreadyPresent</Code><Message>There is
│ already a lease present.
│ RequestId:37995ab0-501e-00e9-4eeb-87af64000000
│ Time:2021-08-02T22:10:36.1165552Z</Message></Error>'
│       * blob metadata "terraformlockid" was empty

To Reproduce
Sync repo and deploy level2 as instructed with rover:

rover -lz /tf/caf/walkthrough/landingzones/caf_solution \
-var-folder /tf/caf/walkthrough/configuration/sandpit/level2/networking/hub \
-tfstate hub.tfstate \
-level level2 \
-env sandpit \
-a apply

Describe the bug
after running the following, receive an error, seemingly from Terraform itself, that suggests rover is building a CLI that has too many arguments for the terraform plan command.

⚠️ NOTE: I am not modifying the configuration for launchpad. Simply trying to get a foundational understanding of the ecosystem and tooling.

To Reproduce

1. Follow setup guide for caf-terraform-landingzones-starter demo
2. Complete prerequisites
   • Cloud Adoption Framework landing zones for Terraform - Starter template
3. Attempt to execute step one (with action == plan)

   rover -lz /tf/caf/landingzones/caf_launchpad \
     -launchpad \
     -var-folder /tf/caf/configuration/${environment}/level0/launchpad \
     -parallelism 30 \
     -level level0 \
     -env ${caf_environment} \
     -a plan

4. See error

Expected behavior
Expected this step to run clean and without issue so that I could move on to step 2 😄

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS: Windows
  • using the CAF Dev Container (image tagged aztfmod/rover:1.0.1-2106.3012)
• Rover Version
  • rover:1.0.1-2106.3012
  • aztfmod/caf/azurerm 5.4.2

Additional context
I am sure this is going to end up being something simple, 😉, please forgive the ignorance! FWIW, I attempted to run the following directly and got the same error:

terraform plan -var-file /tf/caf/configuration/demo/level0/launchpad/configuration.tfvars \ 
-var-file /tf/caf/configuration/demo/level0/launchpad/dynamic_secrets.tfvars \ 
-var-file /tf/caf/configuration/demo/level0/launchpad/iam_role_mapping.tfvars \ 
-var-file /tf/caf/configuration/demo/level0/launchpad/keyvaults.tfvars \ 
-var-file /tf/caf/configuration/demo/level0/launchpad/storage_accounts.tfvars \ 
- parallelism 30

Describe the bug
We have a Jumpbox subnet in our RI even though it is not in use by any required resource

Expected behavior
Remove the jumpbox subnet and its NSG from our RI

Describe the bug
Deploying the AKS reference implementation 104-private-cluster fails.

To Reproduce

1. Follow the instructions in /tf/caf/reference_implementations/azure_kubernetes_services/aks/104-private-cluster/readme.md
2. Attempt to 'Deploy enhanced networking model'
3. Get error /tf/rover/functions.sh: line 166: cd: /tf/caf/public/landingzones/caf_networking/: No such file or directory
4. Amend command to replace /tf/caf/public/landingzones/caf_networking with /tf/caf/landingzones/caf_networking/ to match where the caf-terraform-landingzones repository has been cloned to
5. Discover that the caf-terraform-landingzones does not contain a /caf_networking/ folder

Expected behavior
Expect the enhanced networking to be deployed.

Screenshots
N/A

Environment (please complete the following information):

• OS: Windows
• Rover Version aztfmod/rover:0.15.4-2105.2603

Additional context
The guides here and here were used to prepare the environment.

Describe the bug
This is creating a free cluster.

To Reproduce
Just follow the deployment steps.

Expected behavior
A production/standard configuration for an AKS cluster should use Uptime SLA.

Develop Terraform based solution for the Wingtip reference architecture :

https://github.com/Azure/Enterprise-Scale/tree/main/docs/reference/wingtip

Describe the bug
The level4 deployment does not complete.

bash script stop and prompts for a value for var.aks_cluster_key :
var.aks_cluster_key
Enter a value:

Branch: starter

To Reproduce
Run bash script as per "/tf/caf/configuration/sandpit/level4/argocd/README.md" - Lines 31 to 36 (as below)

rover -lz /tf/caf/landingzones/caf_solution/add-ons/aks_applications/ \
  -tfstate ${application}1.tfstate \
  -var-folder /tf/caf/configuration/${environment}/level4/${application} \
  -var tags={application=\"${application}\"} \
  -level level4 \
  -a plan

Error:
script does not complete and prompts user to enter value for: var.aks_cluster_key

Expected behavior
bash script should not prompt for any parameters

Screenshots

• none

Environment (please complete the following information):

• OS: Windows
• Rover Version: 1.0.1-2106.3012

Additional context

• none

Add a partner ID or any other form of ID into the reference implementation so that we can track the usage of our AKS construction set

Describe the bug
Error during devops integration after commit "#433 aztfmod/kv-access-policy" on aztfmod / terraform-azurerm-caf module

To Reproduce

Ran the rover command for devops integration in sandpit:

rover -lz /tf/caf/landingzones/caf_solution/add-ons/azure_devops
-var-folder /tf/caf/configuration/${environment}/level0/azure_devops
-tfstate azure_devops_contoso_demo.tfstate
-parallelism 30
-level level0
-env ${environment}
-a apply

Error:

Error: Invalid index

on /home/vscode/.terraform.cache/modules/caf/modules/security/keyvault_access_policies/policies.tf line 12, in module "azuread_apps":
12: object_id = var.azuread_apps[try(try(each.value.azuread_app_lz_key, each.value.lz_key),var.client_config.landingzone_key)][each.value.azuread_app_key].azuread_service_principal.object_id
|----------------
| each.value is object with 3 attributes
| each.value.lz_key is "launchpad"
| var.azuread_apps is object with 1 attribute "azdo-contoso_demo"
| var.client_config.landingzone_key is "azdo-contoso_demo"

The given key does not identify an element in this collection value.

Error: Invalid index

on /home/vscode/.terraform.cache/modules/caf/modules/security/keyvault_access_policies/policies.tf line 12, in module "azuread_apps":
12: object_id = var.azuread_apps[try(try(each.value.azuread_app_lz_key, each.value.lz_key),var.client_config.landingzone_key)][each.value.azuread_app_key].azuread_service_principal.object_id
|----------------
| each.value is object with 3 attributes
| each.value.lz_key is "launchpad"
| var.azuread_apps is object with 1 attribute "azdo-contoso_demo"
| var.client_config.landingzone_key is "azdo-contoso_demo"

The given key does not identify an element in this collection value.

Error: Invalid index

on /home/vscode/.terraform.cache/modules/caf/modules/security/keyvault_access_policies/policies.tf line 12, in module "azuread_apps":
12: object_id = var.azuread_apps[try(try(each.value.azuread_app_lz_key, each.value.lz_key),var.client_config.landingzone_key)][each.value.azuread_app_key].azuread_service_principal.object_id
|----------------
| each.value is object with 3 attributes
| each.value.lz_key is "launchpad"
| var.azuread_apps is object with 1 attribute "azdo-contoso_demo"
| var.client_config.landingzone_key is "azdo-contoso_demo"

The given key does not identify an element in this collection value.

Expected behavior
Successful integration with devops

Environment (please complete the following information):

OS: Windows 10
Rover Version aztfmod/rover:0.14.10-2104.2704

Additional context
Using the CAF starter files before merge from #56 from Azure/AL-ADOpatches

Describe the bug
var.global_settings.regions is null on deploy of level1

To Reproduce
Update regions to custom
Successfully deploy level0/launchpad
Deploy level1 with command

rover -lz /tf/caf/walkthrough/landingzones/caf_solution \
-var-folder /tf/caf/walkthrough/configuration/sandpit/level1/gitops/azure_devops_agents_vm \
-tfstate azure_devops_agents_vm.tfstate \
-level level1 \
-env sandpit \
-a apply

Produces:

│ Warning: Value for undeclared variable
│ 
│ The root module does not declare a variable named "azure_devops" but a
│ value was found in file
│ "/tf/caf/walkthrough/configuration/sandpit/level1/gitops/azure_devops_agents_vm/landingzone.tfvars".
│ If you meant to use this value, add a "variable" block to the
│ configuration.
│ 
│ To silence these warnings, use TF_VAR_... environment variables to provide
│ certain "global" settings to all configurations in your organization. To
│ reduce the verbosity of these warnings, use the -compact-warnings option.
╵
Terraform plan return code: 1
Terraform returned errors:
╷
│ Error: Attempt to index null value
│ 
│   on /home/vscode/.terraform.cache/modules/solution/modules/resource_group/module.tf line 15, in resource "azurerm_resource_group" "rg":
│   15:   location = var.global_settings.regions[lookup(var.settings, "region", var.global_settings.default_region)]
│     ├────────────────
│     │ var.global_settings.default_region is "region1"
│     │ var.global_settings.regions is null
│     │ var.settings is object with 1 attribute "name"
│ 
│ This value is null, so it does not have any indices.
╵
╷
│ Error: Attempt to index null value
│ 
│   on /home/vscode/.terraform.cache/modules/solution/modules/resource_group/module.tf line 15, in resource "azurerm_resource_group" "rg":
│   15:   location = var.global_settings.regions[lookup(var.settings, "region", var.global_settings.default_region)]
│     ├────────────────
│     │ var.global_settings.default_region is "region1"
│     │ var.global_settings.regions is null
│     │ var.settings is object with 1 attribute "name"
│ 
│ This value is null, so it does not have any indices.
╵
╷
│ Error: Attempt to index null value
│ 
│   on /home/vscode/.terraform.cache/modules/solution/modules/resource_group/module.tf line 15, in resource "azurerm_resource_group" "rg":
│   15:   location = var.global_settings.regions[lookup(var.settings, "region", var.global_settings.default_region)]
│     ├────────────────
│     │ var.global_settings.default_region is "region1"
│     │ var.global_settings.regions is null
│     │ var.settings is object with 1 attribute "name"
│ 
│ This value is null, so it does not have any indices.
╵
╷
│ Error: Attempt to index null value
│ 
│   on /home/vscode/.terraform.cache/modules/solution/modules/resource_group/module.tf line 15, in resource "azurerm_resource_group" "rg":
│   15:   location = var.global_settings.regions[lookup(var.settings, "region", var.global_settings.default_region)]
│     ├────────────────
│     │ var.global_settings.default_region is "region1"
│     │ var.global_settings.regions is null
│     │ var.settings is object with 1 attribute "name"
│ 
│ This value is null, so it does not have any indices.
╵
╷
│ Error: Attempt to index null value
│ 
│   on /home/vscode/.terraform.cache/modules/solution/modules/resource_group/module.tf line 15, in resource "azurerm_resource_group" "rg":
│   15:   location = var.global_settings.regions[lookup(var.settings, "region", var.global_settings.default_region)]
│     ├────────────────
│     │ var.global_settings.default_region is "region1"
│     │ var.global_settings.regions is null
│     │ var.settings is object with 1 attribute "name"
│ 
│ This value is null, so it does not have any indices.
╵
╷
│ Error: Invalid index
│ 
│   on /home/vscode/.terraform.cache/modules/solution/modules/security/keyvault_access_policies/policies.tf line 48, in module "azuread_group":
│   48:   object_id     = try(each.value.lz_key, null) == null ? var.azuread_groups[var.client_config.landingzone_key][each.value.azuread_group_key].id : var.azuread_groups[each.value.lz_key][each.value.azuread_group_key].id
│     ├────────────────
│     │ each.value.azuread_group_key is "keyvault_level1_rw"
│     │ each.value.lz_key is "launchpad"
│     │ var.azuread_groups is object with 4 attributes
│ 
│ The given key does not identify an element in this collection value.
╵
╷
│ Error: Invalid index
│ 
│   on /home/vscode/.terraform.cache/modules/solution/modules/security/keyvault_access_policies/policies.tf line 48, in module "azuread_group":
│   48:   object_id     = try(each.value.lz_key, null) == null ? var.azuread_groups[var.client_config.landingzone_key][each.value.azuread_group_key].id : var.azuread_groups[each.value.lz_key][each.value.azuread_group_key].id
│     ├────────────────
│     │ each.value.azuread_group_key is "keyvault_level1_rw"
│     │ each.value.lz_key is "launchpad"
│     │ var.azuread_groups is object with 4 attributes
│ 
│ The given key does not identify an element in this collection value.
╵
╷
│ Error: Invalid index
│ 
│   on /home/vscode/.terraform.cache/modules/solution/modules/security/keyvault_access_policies/policies.tf line 48, in module "azuread_group":
│   48:   object_id     = try(each.value.lz_key, null) == null ? var.azuread_groups[var.client_config.landingzone_key][each.value.azuread_group_key].id : var.azuread_groups[each.value.lz_key][each.value.azuread_group_key].id
│     ├────────────────
│     │ each.value.azuread_group_key is "keyvault_level1_rw"
│     │ each.value.lz_key is "launchpad"
│     │ var.azuread_groups is object with 4 attributes
│ 
│ The given key does not identify an element in this collection value.
╵
╷
│ Error: Invalid index
│ 
│   on /home/vscode/.terraform.cache/modules/solution/modules/security/keyvault_access_policies/policies.tf line 48, in module "azuread_group":
│   48:   object_id     = try(each.value.lz_key, null) == null ? var.azuread_groups[var.client_config.landingzone_key][each.value.azuread_group_key].id : var.azuread_groups[each.value.lz_key][each.value.azuread_group_key].id
│     ├────────────────
│     │ each.value.azuread_group_key is "keyvault_level1_rw"
│     │ each.value.lz_key is "launchpad"
│     │ var.azuread_groups is object with 4 attributes
│ 
│ The given key does not identify an element in this collection value.
╵
╷
│ Error: Invalid index
│ 
│   on /home/vscode/.terraform.cache/modules/solution/modules/security/keyvault_access_policies/policies.tf line 48, in module "azuread_group":
│   48:   object_id     = try(each.value.lz_key, null) == null ? var.azuread_groups[var.client_config.landingzone_key][each.value.azuread_group_key].id : var.azuread_groups[each.value.lz_key][each.value.azuread_group_key].id
│     ├────────────────
│     │ each.value.azuread_group_key is "keyvault_level1_rw"
│     │ each.value.lz_key is "launchpad"
│     │ var.azuread_groups is object with 4 attributes
│ 
│ The given key does not identify an element in this collection value.

Expected behavior
Level 1 deploy success

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS: macOS
• Rover Version: aztfmod/rover:1.0.1-2106.3012

Additional context
Add any other context about the problem here.

Describe the bug
Add tests examples to the demo configuration. I would want to verify what was deployed using CAF.

Expected behavior
Ability to run terratest on my environment after deployment

Develop Terraform based solution for the Adventure works reference architecture:

https://github.com/Azure/Enterprise-Scale/tree/main/docs/reference/adventureworks

Describe the bug
Deploying the sandpit fails with error Error: At least one logormetric must be enabled

To Reproduce
Steps to reproduce the behaviour:

1. Sync code on branch starter
2. Set $environment to sandpit as per docs.
3. Run rover -lz /tf/caf/public/landingzones/caf_launchpad -var-folder /tf/caf/configuration/${environment}/level0/launchpad -parallelism 30 -level level0 -env ${environment} -launchpad -a apply
4. See error

Expected behavior
Deploy succeeds

Screenshots
Error log on console is:

Terraform apply return code: 0
Terraform returned errors:

Error: At least one `log` or `metric` must be enabled

  on /home/vscode/.terraform.cache/modules/launchpad/modules/diagnostics/module.tf line 1, in resource "azurerm_monitor_diagnostic_setting" "diagnostics":
   1: resource "azurerm_monitor_diagnostic_setting" "diagnostics" {


Error on or near line 457: Error running terraform apply; exiting with status 2001

Environment (please complete the following information):

• OS: Windows 10.0.19042 Build 19042, running dev container aztfmod/rover:2011.3012 in VS Code 1.51.1
• Rover Version 2011.3012

Additional context
The problem appears to be the nic setting on line 93 in configuration/sandpit/level0/launchpad/diagnostics_definition.tfvars. Enabling metrics for this settings allows the deploy to succeed. I'm not sure if this is the best fix, or if a logs setting needs to be added instead?

I have a PR with this change which I'll submit with this issue #.

Describe the bug
I have an error when I try to instantiate version pipelines in sandpit folder

The given key does not identify an element in this collection value.
Error on or near line 449: Error running terraform plan; exiting with status 2000

configuration/sandpit/pipelines/README-pipelines.md

step 2.1

When

To Reproduce
environment=sandpit

rover -lz /tf/caf/public/landingzones/caf_launchpad
-var-folder /tf/caf/configuration/${environment}/level0/launchpad
-parallelism 30
-level level0
-env ${environment}
-launchpad
-a apply

Expected behavior
Create the launchpad

Screenshots

Terraform returned errors:

Error: Invalid index

  on /home/vscode/.terraform.cache/modules/launchpad/modules/diagnostics/module.tf line 17, in resource "azurerm_monitor_diagnostic_setting" "diagnostics":
  17:   storage_account_id = each.value.destination_type == "storage" ? var.diagnostics.storage_accounts[var.diagnostics.diagnostics_destinations.storage[each.value.destination_key][var.resource_location].storage_account_key].id : null
    |----------------
    | each.value.destination_key is "all_regions"
    | var.diagnostics.diagnostics_destinations.storage is object with 1 attribute "all_regions"
    | var.resource_location is "westeurope"

The given key does not identify an element in this collection value.


Error: Invalid index

  on /home/vscode/.terraform.cache/modules/launchpad/modules/diagnostics/module.tf line 17, in resource "azurerm_monitor_diagnostic_setting" "diagnostics":
  17:   storage_account_id = each.value.destination_type == "storage" ? var.diagnostics.storage_accounts[var.diagnostics.diagnostics_destinations.storage[each.value.destination_key][var.resource_location].storage_account_key].id : null
    |----------------
    | each.value.destination_key is "all_regions"
    | var.diagnostics.diagnostics_destinations.storage is object with 1 attribute "all_regions"
    | var.resource_location is "westeurope"

The given key does not identify an element in this collection value.


Error: Invalid index

  on /home/vscode/.terraform.cache/modules/launchpad/modules/diagnostics/module.tf line 17, in resource "azurerm_monitor_diagnostic_setting" "diagnostics":
  17:   storage_account_id = each.value.destination_type == "storage" ? var.diagnostics.storage_accounts[var.diagnostics.diagnostics_destinations.storage[each.value.destination_key][var.resource_location].storage_account_key].id : null
    |----------------
    | each.value.destination_key is "all_regions"
    | var.diagnostics.diagnostics_destinations.storage is object with 1 attribute "all_regions"
    | var.resource_location is "westeurope"

The given key does not identify an element in this collection value.


Error: Invalid index

  on /home/vscode/.terraform.cache/modules/launchpad/modules/diagnostics/module.tf line 17, in resource "azurerm_monitor_diagnostic_setting" "diagnostics":
  17:   storage_account_id = each.value.destination_type == "storage" ? var.diagnostics.storage_accounts[var.diagnostics.diagnostics_destinations.storage[each.value.destination_key][var.resource_location].storage_account_key].id : null
    |----------------
    | each.value.destination_key is "all_regions"
    | var.diagnostics.diagnostics_destinations.storage is object with 1 attribute "all_regions"
    | var.resource_location is "westeurope"

The given key does not identify an element in this collection value.


Error: Invalid index

  on /home/vscode/.terraform.cache/modules/launchpad/modules/diagnostics/module.tf line 17, in resource "azurerm_monitor_diagnostic_setting" "diagnostics":
  17:   storage_account_id = each.value.destination_type == "storage" ? var.diagnostics.storage_accounts[var.diagnostics.diagnostics_destinations.storage[each.value.destination_key][var.resource_location].storage_account_key].id : null
    |----------------
    | each.value.destination_key is "all_regions"
    | var.diagnostics.diagnostics_destinations.storage is object with 1 attribute "all_regions"
    | var.resource_location is "westeurope"

The given key does not identify an element in this collection value.


Error: Invalid index

  on /home/vscode/.terraform.cache/modules/launchpad/modules/diagnostics/module.tf line 17, in resource "azurerm_monitor_diagnostic_setting" "diagnostics":
  17:   storage_account_id = each.value.destination_type == "storage" ? var.diagnostics.storage_accounts[var.diagnostics.diagnostics_destinations.storage[each.value.destination_key][var.resource_location].storage_account_key].id : null
    |----------------
    | each.value.destination_key is "all_regions"
    | var.diagnostics.diagnostics_destinations.storage is object with 1 attribute "all_regions"
    | var.resource_location is "westeurope"

The given key does not identify an element in this collection value.


Error: Invalid index

  on /home/vscode/.terraform.cache/modules/launchpad/modules/diagnostics/module.tf line 17, in resource "azurerm_monitor_diagnostic_setting" "diagnostics":
  17:   storage_account_id = each.value.destination_type == "storage" ? var.diagnostics.storage_accounts[var.diagnostics.diagnostics_destinations.storage[each.value.destination_key][var.resource_location].storage_account_key].id : null
    |----------------
    | each.value.destination_key is "all_regions"
    | var.diagnostics.diagnostics_destinations.storage is object with 1 attribute "all_regions"
    | var.resource_location is "westeurope"

The given key does not identify an element in this collection value.


Error: Invalid index

  on /home/vscode/.terraform.cache/modules/launchpad/modules/diagnostics/module.tf line 17, in resource "azurerm_monitor_diagnostic_setting" "diagnostics":
  17:   storage_account_id = each.value.destination_type == "storage" ? var.diagnostics.storage_accounts[var.diagnostics.diagnostics_destinations.storage[each.value.destination_key][var.resource_location].storage_account_key].id : null
    |----------------
    | each.value.destination_key is "all_regions"
    | var.diagnostics.diagnostics_destinations.storage is object with 1 attribute "all_regions"
    | var.resource_location is "westeurope"

The given key does not identify an element in this collection value.


Error: Invalid index

  on /home/vscode/.terraform.cache/modules/launchpad/modules/diagnostics/module.tf line 17, in resource "azurerm_monitor_diagnostic_setting" "diagnostics":
  17:   storage_account_id = each.value.destination_type == "storage" ? var.diagnostics.storage_accounts[var.diagnostics.diagnostics_destinations.storage[each.value.destination_key][var.resource_location].storage_account_key].id : null
    |----------------
    | each.value.destination_key is "all_regions"
    | var.diagnostics.diagnostics_destinations.storage is object with 1 attribute "all_regions"
    | var.resource_location is "westeurope"

The given key does not identify an element in this collection value.

Error on or near line 449: Error running terraform plan; exiting with status 2000

Environment (please complete the following information):
Rover
January 2021 Version of caf-terraform-landingzones-starter

Additional context
Tenant with restriction

Describe the bug
There is no role assignment to allow AKS pull from ACR currently in RI

Expected behavior
Add role assignment allowing privileged AKS users pull from ACR

Describe the bug
We have a jumpbox subscription with a bastion PIP that is not needed for the online version of our reference implementation. P

Expected behavior
Please remove the subscription and the PIP resource within it from our RI

Add examples to deploy the sandpit environment via Azure DevOps pipelines, in addition to interactive method.

Describe the bug
From Step 3. Customize and deploy the Azure DevOps Agents (runners)
Error of type

module.vm_extensions["level0"].azurerm_virtual_machine_extension.devops_selfhosted_agent["devops_selfhosted_agent"]: Still creating... [1m0s elapsed]
module.vm_extensions["level3"].azurerm_virtual_machine_extension.devops_selfhosted_agent["devops_selfhosted_agent"]: Still creating... [1m0s elapsed]
module.vm_extensions["level1"].azurerm_virtual_machine_extension.devops_selfhosted_agent["devops_selfhosted_agent"]: Still creating... [1m0s elapsed]
module.vm_extensions["level2"].azurerm_virtual_machine_extension.devops_selfhosted_agent["devops_selfhosted_agent"]: Still creating... [1m0s elapsed]
module.vm_extensions["level4"].azurerm_virtual_machine_extension.devops_selfhosted_agent["devops_selfhosted_agent"]: Still creating... [1m0s elapsed]
Terraform apply return code: 0
Terraform returned errors:
╷
│ Error: Code="VMExtensionProvisioningError" Message="VM has reported a failure when processing extension 'install_azure_devops_agent'. Error message: "Enable failed: failed to execute command: command terminated with exit status=1\n[stdout]\ndll\n./bin/Microsoft.TeamFoundation.Test.WebApi.dll\n./bin/System.Diagnostics.Tools.dll\n./bin/System.Web.HttpUtility.dll\n./bin/Microsoft.Azure.Storage.DataMovement.dll\n./bin/System.Security.Cryptography.Encoding.dll\n./bin/es-ES/\n./bin/es-ES/strings.json\n./bin/Microsoft.TeamFoundation.Core.WebApi.dll\n./bin/System.Private.DataContractSerialization.dll\n./bin/System.Net.WebProxy.dll\n./bin/System.Security.Cryptography.Cng.dll\n./bin/System.Private.Uri.dll\n./bin/Agent.Worker.dll\n./bin/Microsoft.VisualBasic.dll\n./bin/System.Xml.XPath.XmlDocument.dll\n./bin/Minimatch.dll\n./bin/Microsoft.TeamFoundation.Policy.WebApi.dll\n./bin/Microsoft.TeamFoundation.TestClient.PublishTestResults.dll\n./bin/System.Net.WebSockets.dll\n./bin/System.Globalization.dll\n./bin/Agent.Listener.runtimeconfig.json\n./bin/CommandLine.dll\n./bin/System.Threading.AccessControl.dll\n./bin/System.IO.Compression.Brotli.dll\n./bin/update.sh.template\n./bin/System.CodeDom.dll\n./bin/CodeSignSummary-a05ae9f4-33f9-45d4-9a39-6a2e91593084.md\n./bin/System.IO.Abstractions.dll\n./bin/System.IO.MemoryMappedFiles.dll\nextracted\n--------OS Information--------\nNAME="Ubuntu"\nVERSION="20.04.2 LTS (Focal Fossa)"\nID=ubuntu\nID_LIKE=debian\nPRETTY_NAME="Ubuntu 20.04.2 LTS"\nVERSION_ID="20.04"\nHOME_URL="https://www.ubuntu.com/\"\nSUPPORT_URL=\"https://help.ubuntu.com/\"\nBUG_REPORT_URL=\"https://bugs.launchpad.net/ubuntu/\"\nPRIVACY_POLICY_URL=\"https://www.ubuntu.com/legal/terms-and-policies/privacy-policy\"\nVERSION_CODENAME=focal\nUBUNTU_CODENAME=focal\n------------------------------\nThe current OS is Debian based\n--------Debian Version--------\nbullseye/sid\n------------------------------\n/usr/bin/apt\nHit:1 http://azure.archive.ubuntu.com/ubuntu focal InRelease\nHit:2 http://azure.archive.ubuntu.com/ubuntu focal-updates InRelease\nHit:3 http://azure.archive.ubuntu.com/ubuntu focal-backports InRelease\nHit:4 https://packages.microsoft.com/repos/azure-cli focal InRelease\nGet:5 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]\nFetched 114 kB in 0s (229 kB/s)\nReading package lists...\nBuilding dependency tree...\nReading state information...\n14 packages can be upgraded. Run 'apt list --upgradable' to see them.\nReading package lists...\nBuilding dependency tree...\nReading state information...\nliblttng-ust0 is already the newest version (2.11.0-1).\nlibkrb5-3 is already the newest version (1.17-6ubuntu4.1).\nzlib1g is already the newest version (1:1.2.11.dfsg-2ubuntu1.2).\n0 upgraded, 0 newly installed, 0 to remove and 14 not upgraded.\nReading package lists...\nBuilding dependency tree...\nReading state information...\nlibssl1.1 is already the newest version (1.1.1f-1ubuntu2.4).\n0 upgraded, 0 newly installed, 0 to remove and 14 not upgraded.\nReading package lists...\nBuilding dependency tree...\nReading state information...\nReading package lists...\nBuilding dependency tree...\nReading state information...\nlibicu66 is already the newest version (66.1-2ubuntu2).\n0 upgraded, 0 newly installed, 0 to remove and 14 not upgraded.\n-----------------------------\n Finish Install Dependencies\n-----------------------------\ndependencies installed\n\n ___ ______ _ _ \n / _ \ | ___ () | ()\n/ /\ \_____ _ _ __ ___ | |/ / _ __ | | _ __ ___ \n| _ | / | | | '/ _ \ | /| | ' \ / _ \ | | ' \ / _ \/ __|\n| | | |/ /| || | | | __/ | | | | |) | / | | | | | /\ \\n\| |/|\,|| \| \| || ./ \|||| ||\||/\n | |\n agent v2.187.2 || (commit 0cfc45c)\n\n\n>> End User License Agreements:\n\nBuilding sources from a TFVC repository requires accepting the Team Explorer Everywhere End User License Agreement. This step is not required for building sources from Git repositories.\n\nA copy of the Team Explorer Everywhere license agreement can be found at:\n /home/adminuser/agent/agent-1/externals/tee/license.html\n\n\n>> Connect:\n\nError reported in diagnostic logs. Please examine the log for more details.\n - /home/adminuser/agent/agent-1/_diag/Agent_20210620-194325-utc.log\n\n\n[stderr]\nWARNING: Error loading config file: .dockercfg: $HOME is not defined\nsudo: ./svc.sh: command not found\nsudo: ./svc.sh: command not found\n\nWARNING: apt does not have a stable CLI interface. Use with caution in scripts.\n\n\nWARNING: apt does not have a stable CLI interface. Use with caution in scripts.\n\n\nWARNING: apt does not have a stable CLI interface. Use with caution in scripts.\n\n\nWARNING: apt does not have a stable CLI interface. Use with caution in scripts.\n\nE: Unable to locate package libicu67\n\nWARNING: apt does not have a stable CLI interface. Use with caution in scripts.\n\nVS30063: You are not authorized to access https://dev.azure.com.\n\x1b[41mError on or near line 101; exiting with status 1\x1b[0m\n"\r\n\r\nMore information on troubleshooting is available at https://aka.ms/VMExtensionCSELinuxTroubleshoot "
│
│ with module.vm_extensions["level0"].azurerm_virtual_machine_extension.devops_selfhosted_agent["devops_selfhosted_agent"],
│ on extensions/devops_selfhosted_agent.tf line 2, in resource "azurerm_virtual_machine_extension" "devops_selfhosted_agent":
│ 2: resource "azurerm_virtual_machine_extension" "devops_selfhosted_agent" {
│
╵

Screenshots

• OS:Win 10 / Docker 3.4
• Rover Version aztfmod/rover:0.15.1-2104.2711

Describe the bug
Error listing landing zones deployed

To Reproduce
Steps to reproduce the behavior:

1. Following information in https://github.com/Azure/caf-terraform-landingzones-starter/tree/starter/configuration
2. Created two level 0 environments in same subscription
   3.Running the command "rover landingzone list -level level0"
3. Error message: "ERROR: argument --ids: expected at least one argument" - see screenshot

Expected behavior
Expect to see listed environments at level 0. In my case the two that have been successfully deployed

Screenshots

Environment:

• OS: Windows 10
• Rover Version aztfmod/rover:0.13.6-2103.0304

Describe the bug
Broken link exists under:
https://github.com/Azure/caf-terraform-landingzones-starter/blob/starter/reference_implementations/data_analytics/README.md
'Cloud Adoption Framework layered approach'

To Reproduce

1. Go to 'Cloud Adoption Framework layered approach' section:
2. Click on 'the following documentation'
3. See '404' error page

Expected behavior
User should get link to Cloud Adoption Framework layered approach documentation

Environment (please complete the following information):

• OS: macOS
• Monterey v12.1

Additional context
Add any other context about the problem here.

Describe the bug
RI images are currently being pulled from public registries that doesn't conform to our best practices.

Expected behavior
Update our RI to import images from ACR setup for the RI instead of public registries

A link in the file README-piplines.md is broken (404).
https://github.com/Azure/caf-terraform-landingzones-starter/blob/starter/configuration/sandpit/pipelines/README-pipelines.md

Customize your Azure DevOps environment as discussed here.

System Nodepool - DS2v2 - 80gb
User Nodepool - DS3v2 - 120gb
os_disk_size_gb - (Optional) The size of the OS Disk which should be used for each agent in the Node Pool. Changing this forces a new resource to be created.

os_disk_type - (Optional) The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Defaults to Managed. Changing this forces a new resource to be created.