Comments (7)
I've encountered this issue as well, using module version 0.5.2. @vsimon were you able to find a workaround?
from terraform-aws-msk-apache-kafka-cluster.
No workaround found yet unfortunately :(
from terraform-aws-msk-apache-kafka-cluster.
+1
from terraform-aws-msk-apache-kafka-cluster.
+1 .
from terraform-aws-msk-apache-kafka-cluster.
@ashwini-hopin @sekka1 @vsimon @gitbrad
Please try the latest version 0.6.1 released on august 16 from pr #24.
module "kafka" {
source = "https://github.com/cloudposse/terraform-aws-msk-apache-kafka-cluster.git?ref=0.6.1"
# your arguments
}
If that doesn't work, try using this feature branch (from pr #29) and see if it works for you
module "kafka" {
source = "https://github.com/cloudposse/terraform-aws-msk-apache-kafka-cluster.git?ref=module-hostname-count"
# your arguments
}
from terraform-aws-msk-apache-kafka-cluster.
@nitrocode Amazing ( pr #29 ) fixed my issue. However, just discovered that the broker endpoints is outputting only 3 brokers even though the cluster runs 4 brokers
from terraform-aws-msk-apache-kafka-cluster.
I reached out to AWS support to understand why only 3 endpoints are being outputted. Here's the response from them. In case it helps someone,
In order to get more insight, I reached out to our MSK service team and they informed me that this is an expected behavior that both UI and getBootstrapBrokers API gives maximum 3 brokers as the output as you will only need a maximum of 3 broker node details to successfully connect or programmatically connect to the MSK cluster at any given moment. The broker node details presented may randomly change as part of the connection string when retrieving the broker node connection string, alternatively you can still connect to any other available broker nodes on your cluster or make use of, as a connection string (Thus, you will only ever need a maximum of 3 broker nodes to ensure high availability when connecting, producing or consuming from the cluster).
Having said that, only 3 brokers will be listing in bootstrap list irrespective of the number of brokers configured in the cluster and this is by design. However, all brokers will be part of the cluster.
If you're looking for a workaround, you can make use of the 'list-nodes' API [1] , with paginate or no-paginate option to control returned results when retrieving all the broker node details on your cluster. See sample command below:
aws kafka list-nodes --cluster-arn arn:aws:kafka:us-east-1:accountID:cluster/demo-cluster-1/6357e0b2-0e6a-4b86-a0b4-70df934c2e31-5 --no-paginate
[1] https://docs.aws.amazon.com/cli/latest/reference/kafka/list-nodes.html
from terraform-aws-msk-apache-kafka-cluster.
Related Issues (20)
- server properties change does not apply latest revision in the MSK cluster only creates a new revision HOT 1
- Version 2.4.0 tries to downscale storage back to original value
- MSK Multi-VPC Connectivity and Cluster Policy
- Outputs of broker lists creating perpetual diff on MSK clusters with more than 1 broker per AZ HOT 1
- Behaviour when SASL IAM and SCRAM enabled together HOT 1
- Client TLS auth and SASL auth cannot be enabled at the same time HOT 1
- exemple need to upgrade dynamic-subnets to 0.39.8 , otherwise it fails with Call to function "map" failed: the "map" function was deprecated.... HOT 1
- Only create `aws_msk_configuration` if `parameters` are supplied
- MSK cluster to get recreated when sasl is reverted to null HOT 6
- Add Open Monitoring with Prometheus for MSK HOT 1
- Enabling IAM and SCRAM auth at same time fails HOT 1
- Error Connecting to Service Discovery Domains
- Allow creation of Topics & ACLs with Terraform
- Please update the module to use the latest cloudposse / terraform-aws-security-group tag HOT 1
- Error: creating Security Group: InvalidVpcID.NotFound: The vpc ID does not exist HOT 5
- Cluster gets recreated(deleted then created) for the latest version v2.3.0 HOT 5
- Invalid required aws provider version
- Protocols used in allowed_cidr_blocks don't include SASL/SCRAM public cluster
- Cluster gets recreated on enabling open monitoring
- broker storage: BadRequestException: The request does not include any updates to the EBS volumes of the cluster HOT 14
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from terraform-aws-msk-apache-kafka-cluster.