Invalid index error when changing `number_of_broker_nodes` variable about terraform-aws-msk-apache-kafka-cluster HOT 7 CLOSED

I've encountered this issue as well, using module version 0.5.2. @vsimon were you able to find a workaround?

from terraform-aws-msk-apache-kafka-cluster.

No workaround found yet unfortunately :(

from terraform-aws-msk-apache-kafka-cluster.

+1

from terraform-aws-msk-apache-kafka-cluster.

+1 .

from terraform-aws-msk-apache-kafka-cluster.

@ashwini-hopin @sekka1 @vsimon @gitbrad

Please try the latest version 0.6.1 released on august 16 from pr #24.

module "kafka" {
  source = "https://github.com/cloudposse/terraform-aws-msk-apache-kafka-cluster.git?ref=0.6.1"

  # your arguments
}

If that doesn't work, try using this feature branch (from pr #29) and see if it works for you

module "kafka" {
  source = "https://github.com/cloudposse/terraform-aws-msk-apache-kafka-cluster.git?ref=module-hostname-count"

  # your arguments
}

from terraform-aws-msk-apache-kafka-cluster.

@nitrocode Amazing ( pr #29 ) fixed my issue. However, just discovered that the broker endpoints is outputting only 3 brokers even though the cluster runs 4 brokers

from terraform-aws-msk-apache-kafka-cluster.

I reached out to AWS support to understand why only 3 endpoints are being outputted. Here's the response from them. In case it helps someone,

In order to get more insight, I reached out to our MSK service team and they informed me that this is an expected behavior that both UI and getBootstrapBrokers API gives maximum 3 brokers as the output as you will only need a maximum of 3 broker node details to successfully connect or programmatically connect to the MSK cluster at any given moment. The broker node details presented may randomly change as part of the connection string when retrieving the broker node connection string, alternatively you can still connect to any other available broker nodes on your cluster or make use of, as a connection string (Thus, you will only ever need a maximum of 3 broker nodes to ensure high availability when connecting, producing or consuming from the cluster).

Having said that, only 3 brokers will be listing in bootstrap list irrespective of the number of brokers configured in the cluster and this is by design. However, all brokers will be part of the cluster.

If you're looking for a workaround, you can make use of the 'list-nodes' API [1] , with paginate or no-paginate option to control returned results when retrieving all the broker node details on your cluster. See sample command below:

aws kafka list-nodes --cluster-arn arn:aws:kafka:us-east-1:accountID:cluster/demo-cluster-1/6357e0b2-0e6a-4b86-a0b4-70df934c2e31-5 --no-paginate

[1] https://docs.aws.amazon.com/cli/latest/reference/kafka/list-nodes.html

from terraform-aws-msk-apache-kafka-cluster.