Giter Club home page Giter Club logo

terraform-provider-cockroach's Introduction

Terraform provider for CockroachDB Cloud

terraform-provider-cockroach is the Terraform provider for the CockroachDB Cloud API [reference, getting started].

For information on developing terraform-provider-cockroach see DEVELOPMENT.md.

Note: This is a preview release, suitable only for experimental use.

Get Started

Prerequisites

Before you use terraform-provider-cockroach you must install Terraform and git on your local machine.

Run the Serverless Example

  1. In a terminal clone the terraform-provider-cockroach repository:

    git clone https://github.com/cockroachdb/terraform-provider-cockroach.git
  2. Go to the examples/workflows/cockroach_serverless_cluster directory.

    cd examples/workflows/cockroach_serverless_cluster
  3. The provider requires an API key set in an environment variable named COCKROACH_API_KEY. Copy the API key from the CockroachDB Cloud console and create the COCKROACH_API_KEY environment variable.

    export COCKROACH_API_KEY=<YOUR_API_KEY>

    Where <your API key> is the API key you copied from the CockroachDB Cloud Console.

  4. In a text editor create a new file terraform.tfvars in cockroach_serverless_cluster with the following settings.

    cluster_name = "<cluster name>"
    sql_user_name = "<SQL user name>"
    sql_user_password = "<SQL user password>"
    

    Where:
    - <cluster name> is the name of the cluster you want to create.
    - <SQL user name> is the name of the SQL user you want to create.
    - <SQL user password> is the password for the SQL user you want to create.

  5. Initialize the provider.

    terraform init

    This reads the main.tf configuration file, which contains the information on how the provider will create the Serverless cluster. The terraform.tfvars file sets the cluster name, SQL user name, and SQL user password.

  6. Create the Terraform plan. This shows the actions the provider will take, but won't perform them.

    terraform plan
  7. Create the cluster.

    terraform apply

    Enter yes when prompted to apply the plan and create the cluster.

  8. (optional) Delete the cluster when you are done.

    terraform destroy

    Enter yes when prompted to delete the cluster.

Run the Dedicated Example

Running this example will incur charges to your CockroachDB account. It defaults to a minimal hardware spec, but please remember to run terraform destroy when you're done if you don't need to keep your cluster.

  1. In a terminal clone the terraform-provider-cockroach repository:

    git clone https://github.com/cockroachdb/terraform-provider-cockroach.git
  2. In a terminal go to the examples/workflows/cockroach_dedicated_cluster directory.

    cd examples/workflows/cockroach_dedicated_cluster
  3. The provider requires an API key set in an environment variable named COCKROACH_API_KEY. Copy the API key from the CockroachDB Cloud console and create the COCKROACH_API_KEY environment variable.

    export COCKROACH_API_KEY=<YOUR_API_KEY>

    Where <your API key> is the API key you copied from the CockroachDB Cloud Console.

  4. In a text editor create a new file terraform.tfvars in cockroach_dedicated_cluster with the following settings.

    cluster_name = "<cluster name>"
    database = "<database name>"
    sql_user_name = "<SQL user name>"
    sql_user_password = "<SQL user password>"
    cloud_provider = "<cloud provider>"
    cloud_provider_regions = ["<cloud provider region>"]
    cluster_node_count = <number of nodes>
    storage_gib = <storage in GiB>
    machine_type = "<cloud provider machine type>"
    allow_list_name = "<allow list name>"
    cidr_ip = "<allow list CIDR IP>"
    cidr_mask = <allow list CIDR prefix>
    os = "<OS name>"
    

    Where:
    - <cluster name> is the name of the cluster you want to create.
    - <database name> is the name that will be used for the database created within the cluster. This database is in addition to defaultdb which is created by default.
    - <SQL user name> is the name of the SQL user you want to create.
    - <SQL user password> is the password for the SQL user you want to create.
    - <cloud provider> is the cloud infrastructure provider. Possible values are GCP or AWS or AZURE (limited access).
    - <cloud provider region> is the region code or codes for the cloud infrastructure provider. For multi-region clusters, separate each region with a comma.
    - <number of nodes> is the number of nodes in each region. Cockroach Labs recommends at least 3 nodes per region, and the same number of nodes in each region for multi-region clusters.
    - <storage in GiB> is the amount of storage specified in GiB.
    - <cloud provider machine type> is the machine type for the cloud infrastructure provider.
    - <allow list name> is the name for the IP allow list. Use a descriptive name to identify the IP allow list.
    - <allow list CIDR IP> is the Classless Inter-Domain Routing (CIDR) IP address base.
    - <allow list CIDR prefix> is the CIDR prefix. This should be a number from 0 to 32. Use 32 to only allow the single IP Address passed in cidr_ip.
    - <OS name> is the name of the OS that will be used to connect from for connection string output. Possible values are ('WINDOWS', 'MAC', and 'LINUX').

  5. Initialize the provider.

    terraform init

    This reads the main.tf configuration file, which contains the information on how the provider will create the Serverless cluster. The terraform.tfvars file sets the cluster name, SQL user name, and SQL user password.

  6. Create the Terraform plan. This shows the actions the provider will take, but won't perform them.

    terraform plan
  7. Create the cluster.

    terraform apply

    Enter yes when prompted to apply the plan and create the cluster.

  8. (optional) Delete the cluster when you are done.

    terraform destroy

    Enter yes when prompted to delete the cluster.

terraform-provider-cockroach's People

Contributors

abhishekdwivedi3060 avatar andy-kimball avatar arjunmahishi avatar dasrirez avatar dhartunian avatar erademacher avatar fantapop avatar jason-crl avatar jaylim-crl avatar jenngeorge avatar jmcarp avatar kpatron-cockroachlabs avatar marksoper avatar pjtatlow avatar prafull01 avatar samdotci avatar santamaura avatar vonniklasson avatar yecs1999 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

terraform-provider-cockroach's Issues

Could not enable log export: log channel 'TELEMETRY' 'DEV' is unknown

Using cockroach_log_export_config, I encounter errors when specifying DEV and TELEMETRY channels.

│ Error: Error enabling log export
│
│   with module.cockroachdb_dedicated.cockroach_log_export_config.example[0],
│   on ../../metric_log_export.tf line 82, in resource "cockroach_log_export_config" "example":
│   82: resource "cockroach_log_export_config" "example" {
│
│ Could not enable log export: invalid argument: invalid field
│ ClusterSpec: log channel 'DEV' is unknown


│ Error: Error enabling log export
│
│   with module.cockroachdb_dedicated.cockroach_log_export_config.export[0],
│   on ../../metric_log_export.tf line 82, in resource "cockroach_log_export_config" "export":
│   82: resource "cockroach_log_export_config" "export" {
│
│ Could not enable log export: invalid argument: invalid
│ field ClusterSpec: log channel 'TELEMETRY' is unknown

I am able to create the cockroach_log_export_config resource after removing these channels. I don't understand why this is the case, official documentation mentions that this is a supported channel name.

Code

resource "cockroach_log_export_config" "export" {
  count          = var.enable_cloudwatch_integration ? 1 : 0
  id             = cockroach_cluster.cluster.id
  auth_principal = aws_iam_role.crdb_cloudwatch_role[count.index].arn
  log_name       = "${var.cluster_name}-${local.cockroachdb_short_name}"
  type           = "AWS_CLOUDWATCH"
  redact         = false
  groups         = [
    {
      log_name : "sql-perf",
      channels : ["SQL_PERF", "SQL_EXEC"],
      redact : false
    },
    {
      log_name : "ops",
      channels : ["OPS", "HEALTH", "SQL_SCHEMA"]
      min_level : "WARNING"
    },
    {
      log_name : "security",
      channels : ["SESSIONS", "USER_ADMIN", "PRIVILEGES", "SENSITIVE_ACCESS"]
    },
    {
      log_name : "debug",
      channels : ["DEV", "TELEMETRY", "STORAGE"],
      redact : true
    },
  ]
}

Terraform versions

Terraform v1.3.7
on darwin_amd64
+ provider registry.terraform.io/cockroachdb/cockroach v0.4.7
+ provider registry.terraform.io/hashicorp/aws v4.66.1

Feature request: create non-admin database users with cockroach_sql_user

Database users created through the provider are placed in the admin role and I haven't found a way to create a user that does not have this role.

One use case is creating readonly developer accounts for use with sso sql login. We would like to create a database user for a developer then grant them SELECT on all tables in a database, but no other privileges.

Provider crashes given a not uuid id to data cockroach_cluster

Given an invalid id to data cockroach_cluster, the provider crashes.

E.g., here the id contains the name of the cluster, and not the uuid.

data "cockroach_cluster" "foo" {
  id = "sunset-laika"
}

The fix is to indicate the uuid as the value.

data "cockroach_cluster" "foo" {
  id = "6ac25c84-adff-11ed-afa1-0242ac120002"
}

Consider validating the id and returning an error.

It looks like a return is missing in internal/provider/cockroach_cluster_data_source.go near

	cockroachCluster, httpResp, err := d.provider.service.GetCluster(ctx, cluster.ID.ValueString())
	if httpResp != nil && httpResp.StatusCode == http.StatusNotFound {
		resp.Diagnostics.AddError(
			"Cluster not found",
			fmt.Sprintf("Couldn't find a cluster with ID %s", cluster.ID.ValueString()))
		return
	}
	if err != nil {
		resp.Diagnostics.AddError(
			"Error getting cluster info",
			fmt.Sprintf("Unexpected error while retrieving cluster info: %v", formatAPIErrorMessage(err)))
          // TODO(add the missing return to prevent crashes): return
	}

	cluster.Name = types.StringValue(cockroachCluster.Name)

Here is the panic.

Stack trace from the terraform-provider-cockroach_v0.4.1 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0xafb7ad]

goroutine 53 [running]:
github.com/cockroachdb/terraform-provider-cockroach/internal/provider.(*clusterDataSource).Read(0xc00011c030, {0xe25980, 0xc000588ab0}, {{{{0xe29268, 0xc0005cc1b0}, {0xbf3f40, 0xc000589b90}}, {0xe2abc0, 0xc00023b680}}, {{{0x0, ...}, ...}, ...}}, ...)
        github.com/cockroachdb/terraform-provider-cockroach/internal/provider/cockroach_cluster_data_source.go:174 +0x4cd
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).ReadDataSource(0xc000408160, {0xe25980, 0xc000588ab0}, 0xc000588ba0, 0xc000171618)
        github.com/hashicorp/[email protected]/internal/fwserver/server_readdatasource.go:75 +0x46d
github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(*Server).ReadDataSource(0xc000408160, {0xe25980?, 0xc000588960?}, 0xc00011e120)
        github.com/hashicorp/[email protected]/internal/proto6server/server_readdatasource.go:52 +0x28d
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ReadDataSource(0xc000312dc0, {0xe25980?, 0xc0005881b0?}, 0xc00058c0a0)
        github.com/hashicorp/[email protected]/tfprotov6/tf6server/server.go:656 +0x3ef
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ReadDataSource_Handler({0xcb5620?, 0xc000312dc0}, {0xe25980, 0xc0005881b0}, 0xc000182000, 0x0)
        github.com/hashicorp/[email protected]/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:421 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc00041c000, {0xe294a0, 0xc0004824e0}, 0xc000590000, 0xc0004189c0, 0x1315db0, 0x0)
        google.golang.org/[email protected]/server.go:1340 +0xd13
google.golang.org/grpc.(*Server).handleStream(0xc00041c000, {0xe294a0, 0xc0004824e0}, 0xc000590000, 0x0)
        google.golang.org/[email protected]/server.go:1713 +0xa1b
google.golang.org/grpc.(*Server).serveStreams.func1.2()
        google.golang.org/[email protected]/server.go:965 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
        google.golang.org/[email protected]/server.go:963 +0x28a

Error: The terraform-provider-cockroach_v0.4.1 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Feature request: Resources for creating Schema and SQL User permissions

Overview

I have the following limitation with the Terraform provider for CockroachDB. We are trying to automate the creation of Database, Schema, Users and User's permissions like GRANTS and DEFAULT PRIVILEGES. Unfortunately, the Provider currently supports only the creation of Database and SQL Users.

Issue

I have tried to create the resources with the PostgreSQL Provider but there are engine compatibility issues with this provider and CockroachDB. I am receiving the following errors when trying to create the schema and the permissions:

│ Error: Error reading schema: pq: incompatible COALESCE expressions: type "aclitem" does not exist
│ 
│   with module.apps["test-app-1-migration"].postgresql_schema.cockroachdb_schema[0],
│   on .terraform/modules/apps/cockroachdb.tf line 5, in resource "postgresql_schema" "cockroachdb_schema":
│    5: resource "postgresql_schema" "cockroachdb_schema" {
│ Error: could not get advisory lock for role test_app_1_migration: pq: unknown function: pg_advisory_xact_lock(): function undefined
│ 
│   with module.apps["test-app-1-migration"].postgresql_grant.cockroachdb_schema_permissions[0],
│   on .terraform/modules/apps/cockroachdb.tf line 42, in resource "postgresql_grant" "cockroachdb_schema_permissions":
│   42: resource "postgresql_grant" "cockroachdb_schema_permissions" {

Request

It would be nice if the CockroachDB provider is able to create these resources

  • postgresql_schema
  • postgresql_grant
  • postgresql_default_privileges
    like the PostgreSQL Provider or make the CockroachDB compatible with Vanilla Postgresql.

Error when deleting cockroach_metric_export_datadog_config

I get errors when I delete the cockroach_metric_export_datadog_config resource. At first, I thought it was an ordering thing, so I added a depends_on to guarantee the order but it didn't help.

│ Error: Error deleting Datadog metric export config
│
│ Could not delete Datadog metric export config: couldn't lock cluster

After running the terraform destroy command again, I get another error which blocks me from proceeding unless I do terraform state rm. This error tells me that the resource on CockroachDB side has actually been deleted.

│ Error: Error getting Datadog metric export info
│
│   with module.cockroachdb_dedicated.cockroach_metric_export_datadog_config.datadog_export[0],
│   on ../../metric_log_export.tf line 25, in resource "cockroach_metric_export_datadog_config" "datadog_export":
│   25: resource "cockroach_metric_export_datadog_config" "datadog_export" {
│
│ Unexpected error retrieving Datadog metric export info: couldn't find a Datadog metric export config for the cluster

Terraform code

resource "cockroach_metric_export_datadog_config" "datadog_export" {
  count   = var.enable_datadog_integration ? 1 : 0
  id      = cockroach_cluster.cluster.id
  site    = "US1"
  api_key = var.datadog_api_key

  depends_on = [
    cockroach_cluster.cluster,
  ]
}

Terraform versions

▶ terraform version
Terraform v1.3.7
on darwin_amd64
+ provider registry.terraform.io/cockroachdb/cockroach v0.4.7
+ provider registry.terraform.io/hashicorp/aws v4.67.0

Error messages for invalid conditions with CRDB Terraform Provider

If I attempt to explicitly specify the version for a Serverless cluster I get the following error:

Attempting to create a serverless cluster with an explicit version ("v22.1.1")

│ Error: Provider produced invalid plan

│ Provider "registry.terraform.io/hashicorp/cockroach" planned an invalid value for cockroach_cluster.cockroach.cockroach_version: planned value
│ cty.NullVal(cty.String) does not match config value cty.StringVal("v22.1.1").

│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

This should return a more descriptive error message.

issue updating name and flag fields for cockroach_allow_list resource

There is currently an issue with updating the name, ui and sql flags. When updating an existing state, the apply will appear to succeed but the state is not actually updated, and you will be able to run plan / apply again and see the same state. The result is these fields never get updated in state and therefore the infrastructure can't change.

Below lines where state is used instead of the plan for the SQL, UI and Name fields:

name := state.Name.ValueString()
existingAllowList := client.AllowlistEntry1{
Ui: state.Ui.ValueBool(),
Sql: state.Sql.ValueBool(),
Name: &name,
}
_, _, err := r.provider.service.UpdateAllowlistEntry(
ctx, clusterId, entryCIDRIp, entryCIDRMask, &existingAllowList)

Unable to modify resources with CRDB Terraform Provider

Whilst it is possible to successfully create serverless and dedicated clusters, any attempt to modify a cluster returns an error.

Serverless example - attempting to increase the spend from $0 to $5 :

│ Error: Provider produced inconsistent result after apply

│ When applying changes to cockroach_cluster.cockroach, provider "provider["registry.terraform.io/hashicorp/cockroach"]" produced an unexpected new value:
│ .create_spec.serverless.spend_limit: was cty.NumberIntVal(5), but now cty.NumberIntVal(0).

│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Dedicated example #1 - attempting to add a 4th node via Terraform
dedicated: {
region_nodes = {
"eu-west-2": 4
}

cockroach_cluster.cockroach: Modifying... [id=8b7dd8da-b654-429f-96a8-bf9fee7cc710]

│ Error: Provider produced inconsistent result after apply

│ When applying changes to cockroach_cluster.cockroach, provider "provider["registry.terraform.io/hashicorp/cockroach"]" produced an unexpected new value:
│ .create_spec.dedicated.region_nodes["eu-west-2"]: was cty.NumberIntVal(4), but now cty.NumberIntVal(3).

│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Dedicated example #2 - Attempting to change machine type from "m5.large" to "m5.xlarge" ...

│ Error: Provider produced inconsistent result after apply

│ When applying changes to cockroach_cluster.cockroach, provider "provider["registry.terraform.io/hashicorp/cockroach"]" produced an unexpected new value:
│ .create_spec.dedicated.hardware.machine_spec.machine_type: was cty.StringVal("m5.xlarge"), but now cty.StringVal("m5.large").

│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

terraform-registry-manifest.json specifies protocol_versions "5.0" but built binary appears to be "6.0"

terraform-registry-manifest.json suggests that the supported protocol_version is 5, but attempting to use the provider in a version of terraform using version 5 (0.13.7 in this case) reports the following:

Failed to instantiate provider "registry.terraform.io/cockroachdb/cockroach"
to obtain schema: Incompatible API version with plugin. Plugin version: 6,
Client versions: [5]

Looking at https://developer.hashicorp.com/terraform/registry/providers/publishing#terraform-registry-manifest-file and https://pkg.go.dev/github.com/hashicorp/terraform-plugin-framework/providerserver#ServeOpts.ProtocolVersion suggests that version 6 will be used unless overridden in ServeOpts.ProtocolVersion which does not appear to be the case: https://github.com/cockroachdb/terraform-provider-cockroach/blob/main/main.go#L53

It would be helpful if terraform 0.13 were supported, but otherwise I imagine protocol_versions should be changed to "6.0" in terraform-registry-manifest.json to avoid any confusion.

Thanks!

Dedicated example throws an "unsupported version" error

I got an "unsupported version" error when running terraform apply using the dedicated workflow example:
https://github.com/cockroachdb/terraform-provider-cockroach/blob/main/examples/workflows/cockroach_dedicated_cluster/main.tf

cockroach_cluster.example: Creating...
╷
│ Error: Error creating cluster
│
│   with cockroach_cluster.example,
│   on main.tf line 89, in resource "cockroach_cluster" "example":
│   89: resource "cockroach_cluster" "example" {
│
│ Could not create cluster: invalid argument: unsupported version: v22.1.10

I used the following terraform.tfvars file:

cluster_name = "ian-dim-dog"
sql_user_name = "maxroach"
sql_user_password = "NotAGoodPassword"
cloud_provider = "GCP"
cloud_provider_regions = ["us-west2"]
cluster_node_count = 3
storage_gib = 15
machine_type = "n1-standard-2"
allow_list_name = "Ian's home network"
cidr_ip = "x.x.x.x"
cidr_mask = 32

The example main.tf specifies the default version as v22.1. Changing it to v22.2 allowed the plan to successfully apply.

Port is missing from the `cockroach_cluster` response

There is no connection port indicated in the cockroach_cluster results. I've been guessing that it would always be 26257 to save a connection URL into another TF resource.

It would be great to have the port explicitly stated though!

Unable to retrieve connection URL for Serverless clusters

I can't seem to figure out a way to construct the connection URL for a serverless cluster from a cockroach_cluster resource. There's the sql_dns field of a region, but that only provides an endpoint like free-tier.gcp-us-central1.cockroachlabs.cloud, which lacks the tenant routing ID, and is not usable to determine the proxy endpoint which looks like tenant-name-1234.5xj.cockroachlabs.cloud.

Although I could technically use the options DSN parameter and pass the routing ID via the cluster option, this is inconvenient as it's not supported by configuration in some applications.

This might be a shortcoming of the Cockroach Cloud API as well as I don't see it exposed there either.

It would be nice if this could be added to the provider, and for the data source as well.

Provider produced inconsistent result after apply

Received an error after applying, although the cluster itself was actually created.

│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to module.cockroachdb_dedicated.cockroach_cluster.cluster, provider "provider[\"registry.terraform.io/cockroachdb/cockroach\"]" produced an
│ unexpected new value: .regions[0].primary: was cty.True, but now cty.False.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Running a plan after this forced a recreation of the cluster


Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.cockroachdb_dedicated.cockroach_cluster.cluster is tainted, so must be replaced

Terraform code

resource "cockroach_cluster" "cluster" {
  cloud_provider = "AWS"
  name           = "test"
  regions = [
    {
      name       = "ap-southeast-1"
      node_count = 3
      primary = true
    }
  ]

  dedicated = {
    num_virtual_cpus           = 2
    storage_gib                = 100
    private_network_visibility = true
  }

}

Terraform versions

Terraform v1.3.7
on darwin_amd64
+ provider registry.terraform.io/cockroachdb/cockroach v0.4.7
+ provider registry.terraform.io/hashicorp/aws v4.66.1

Error when scaling down cluster

Trying to scale down a cluster from a 4vCPU node to 2vCPU node but encountered an inconsistent apply error.

Terraform plan

  # module.crdb_foo.cockroach_cluster.cluster will be updated in-place
  ~ resource "cockroach_cluster" "cluster" {
      ~ dedicated         = {
          ~ disk_iops                  = 1500 -> (known after apply)
          ~ machine_type               = "m6i.xlarge" -> (known after apply)
          ~ memory_gib                 = 16 -> (known after apply)
          ~ num_virtual_cpus           = 4 -> 2
            # (2 unchanged attributes hidden)
        }
        id                = "<REDACTED>"
        name              = "foo-cluster"
      ~ operation_status  = "UNSPECIFIED" -> (known after apply)
      ~ regions           = [
          ~ {
                name       = "<REDACTED>"
              ~ sql_dns    = "<REDACTED>" -> (known after apply)
              ~ ui_dns     = "<REDACTED>" -> (known after apply)
                # (2 unchanged attributes hidden)
            },
        ]
      ~ state             = "CREATED" -> (known after apply)
      ~ upgrade_status    = "FINALIZED" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

Error

│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to module.crdb_foo.cockroach_cluster.cluster, provider "provider[\"registry.terraform.io/cockroachdb/cockroach\"]" produced an unexpected new value: .dedicated.num_virtual_cpus: was cty.NumberIntVal(2), but now cty.NumberIntVal(4).
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Versions

▶ terraform version
Terraform v1.3.7
on darwin_amd64
+ provider registry.terraform.io/cockroachdb/cockroach v0.4.7

Had to manually apply the changes via the CockroachDB console.

Plugin crash when trying to import existing cluster into terraform

When trying to import an existing cluster into terraform, the plugin crashed due to a memory panic (see error below).

The terraform import command run was terraform import module.databases.cockroach_cluster.main_cluster <cluster uuid, copied from URL bar of console>

Please let me know if you need any further information on my configuration, or have any other questions.

│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the plugin6.(*GRPCProvider).ReadResource call. The plugin logs may contain
│ more details.
╵


Stack trace from the terraform-provider-cockroach_v0.6.0 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0xb33d20]

goroutine 27 [running]:
github.com/cockroachdb/terraform-provider-cockroach/internal/provider.loadClusterToTerraformState(0xc00027e0f0, 0xc00029c360, 0xc00029c360)
        github.com/cockroachdb/terraform-provider-cockroach/internal/provider/cluster_resource.go:790 +0x4a0
github.com/cockroachdb/terraform-provider-cockroach/internal/provider.(*clusterResource).Read(0xc000124b70, {0xe9d680, 0xc0004dd8f0}, {{{{0xea11a8, 0xc000299710}, {0xc53280, 0xc0002994a0}}, {0xea33b8, 0xc000100370}}, 0xc000124b88, ...}, ...)
        github.com/cockroachdb/terraform-provider-cockroach/internal/provider/cluster_resource.go:441 +0x665
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).ReadResource(0xc0003f4160, {0xe9d680, 0xc0004dd8f0}, 0xc0004dd9e0, 0xc0002b1608)
        github.com/hashicorp/[email protected]/internal/fwserver/server_readresource.go:97 +0x60e
github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(*Server).ReadResource(0xc0003f4160, {0xe9d680?, 0xc0004dd7a0?}, 0xc000128380)
        github.com/hashicorp/[email protected]/internal/proto6server/server_readresource.go:52 +0x27b
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ReadResource(0xc00030f7c0, {0xe9d680?, 0xc0004dcff0?}, 0xc000516780)
        github.com/hashicorp/[email protected]/tfprotov6/tf6server/server.go:746 +0x49e
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ReadResource_Handler({0xd1e7a0?, 0xc00030f7c0}, {0xe9d680, 0xc0004dcff0}, 0xc00017acb0, 0x0)
        github.com/hashicorp/[email protected]/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:349 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0000001e0, {0xea13e0, 0xc000416680}, 0xc0004d7560, 0xc0003f71d0, 0x13b9290, 0x0)
        google.golang.org/[email protected]/server.go:1340 +0xd13
google.golang.org/grpc.(*Server).handleStream(0xc0000001e0, {0xea13e0, 0xc000416680}, 0xc0004d7560, 0x0)
        google.golang.org/[email protected]/server.go:1713 +0xa1b
google.golang.org/grpc.(*Server).serveStreams.func1.2()
        google.golang.org/[email protected]/server.go:965 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
        google.golang.org/[email protected]/server.go:963 +0x28a

Error: The terraform-provider-cockroach_v0.6.0 plugin crashed!```

Cannot create a SQL user after creating a dedicate cluster (right away)

Since new clusters take about 30 minutes to provision, it is not actually possible to create a SQL user right away.
Basically ends up with a 404 error until the provisioning is complete.

GKE suffers from slow start-up as well and typically we see terraform just waiting and checking for status update until considering that the cluster is fully up.

module.cockroach_global_user_service.cockroach_sql_user.user: Creating...
module.cockroach_global_user_service.google_secret_manager_secret_version.version: Creation complete after 1s [id=projects/664563636243/secrets/cockroach_sql_connection_global_user_service/versions/1]
╷
│ Error: Error creating sql user
│ 
│   with module.cockroach_global_user_service.cockroach_sql_user.user,
│   on ../modules/cockroach/main.tf line 23, in resource "cockroach_sql_user" "user":
│   23: resource "cockroach_sql_user" "user" {
│ 
│ Could not create sql user, unexpected error: &{404 Not Found 404 HTTP/2.0 2 0 map[Alt-Svc:[h3=":443"; ma=2592000,h3-29=":443"; ma=2592000] Cc-Version:[2022-03-31] Content-Length:[130]
│ Content-Security-Policy:[sandbox] Content-Type:[application/json] Date:[Fri, 14 Oct 2022 21:39:54 GMT] Strict-Transport-Security:[max-age=63072000; preload] Via:[1.1 google]] {{
│   "code": 5,
│   "message": "internal call failed: cluster '<<uuid-redacted>>' is not found",
│   "details": []
│ }} 130 [] false false map[] 0x14000396600 0x1400009f6b0} %!v(MISSING) 404 Not Found

TF Code:

resource "cockroach_cluster" "main" {
  name           = "test-name"
  cloud_provider = "GCP"
  create_spec = {
    dedicated = {
      region_nodes  = {
        us-west2 = 3
      }
      hardware = {
        machine_spec = {
          machine_type = "n1-standard-2"
        }
      }
    }
  }
}

resource "cockroach_sql_user" "user" {
  id       = cockroach_cluster.cluster.id
  name     = "test
  password = "test"
}

Provider fails to implement pagination for >100 SQL users, causing user creation conflicts

Issue Description:

The Terraform provider incorrectly handles pagination when verifying SQL users in the state. When more than 100 SQL users are present, it only retrieves the first 100 due to the default page size. This causes warnings during each Terraform run for a SQL cluster with more than 100 users and errors when applying changes due to conflicts with existing, unfetched users.

The warning during the plan appears as below:

This cluster doesn't have a SQL user named... Removing from state..

Expected Behavior

The provider should correctly implement pagination to verify all SQL users when their count exceeds 100.

Steps to Reproduce:

  1. Write Terraform code that would provision >100 SQL users for a cluster
  2. Execute a terraform apply
  3. Observe the success of the apply (assuming there were no existing users in the state)
  4. Execute a terraform apply again
  5. Observe the warnings indicating that some SQL users are missing, and that they will need to be recreated
  6. Try to apply changes and observe the resultant errors due to user creation conflicts.

Proposed Solution

I have created a PR to resolve this issue.

Documentation is out of date for dedicated clusters

The schema in the documentation at https://registry.terraform.io/providers/cockroachdb/cockroach/latest/docs/data-sources/cluster#nested-schema-for-configdedicated

Does not match how I actually needed to set up my TF configuration:

resource "cockroach_cluster" "main" {
  name           = local.cluster_name
  cloud_provider = "GCP"
  create_spec = {
    dedicated = {
      region_nodes  = {
        us-west2 = 3
      }
      hardware = {
        machine_spec = {
          machine_type = "n1-standard-2"
        }
      }
    }
  }
}

Specify cockroach_connection_string connection_params attributes in docs

When trying to get just the host uri of a cluster, the most straightforward option seems to be connection_params from cockroach_connection_string -- but the documentation only lists connection_params as a "map of string" without specifying its attributes. The underlying cockroach API documentation actually doesn't specify it either, you need to manually call the API to get the parameters.
Can we specify the connection_params attributes in the documentation? From calling the cockroach API manually, I got Database, Host, and Port.

Provide a data sources for ListClusters, ListAvailableRegions and ListDatabases

Feature Request

Presently, there exists a way of fetching a cluster by a known id, however, there is no ability to fetch all clusters which exist. It could be great to have a data source for doing this as it allows for more dynamic configuration whereby it is not possible to do this today.

What is the problem right now?

Currently, we need to define out the cluster ID's and set them to internal variables to check against, rather than being able to dynamically create and/or fetch a cluster and get the id value.

Use Case

In our development environments, we prefer to reuse a single cluster (it's wasteful to continually create new clusters for each development environment), therefore, we want to be able to dynamically check against the clusters configurations to check and see if we have a development cluster available or not and then we create different databases for each dev environment since they're utilized very infrequently.

Ideally this would provide back information about each cluster in a list, this method already exists within the go SDK which is the backbone for the terraform provider: ListClusters, likewise, the ListAvailableRegions would certainly aide in associating the proper regions for a cluster. Then we would like to be able to fetch the currently configured databases, this lets us know if we need to create one or not ListDatabases in the event we are sharing this database for a different stack, think of a situation whereby you are reusing the database to test out a hotfix / derived stack.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.