Comments (23)
This is probably miss configuration.
How are your kafka brokers and kafka clients configured, which versions are used ?
How do you start kafka-proxy ?
from kafka-proxy.
I am getting this as well. My topology is VPC peering my AWS to confluent.cloud and trying to get my localhost (outside of AWS env) to be able to reach confluent.cloud.
from kafka-proxy.
Could you provide server, proxy and client configuration ?
from kafka-proxy.
This is the environment I have setup:
https://docs.confluent.io/current/cloud/vpc.html
I don't have many settings on the server side available to me but I do know that I can connect to it on server:9092
from an instance running on my AWS env.
The proxy settings I am currently trying are:
kafka-proxy server --bootstrap-server-mapping "server:9092,0.0.0.0:9092" \
--log-level debug \
--tls-enable \
--tls-insecure-skip-verify
The client settings I am using are something to the affect of:
// kafka client version 2.4.0
import org.apache.kafka.clients.admin.{AdminClient, AdminClientConfig}
import java.util.Properties
val config = ConfigFor("environment")
val properties = new Properties()
properties.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, config.bootstrapServerUrl)
properties.put(AdminClientConfig.REQUEST_TIMEOUT_MS_CONFIG, "300000")
properties.put("ssl.endpoint.identification.algorithm", "https")
properties.put("sasl.mechanism", "PLAIN")
properties.put("sasl.jaas.config", s"""org.apache.kafka.common.security.plain.PlainLoginModule required username="${config.clusterAPIKey}" password="${config.clusterAPISecret}";""")
properties.put("security.protocol", "SASL_SSL")
AdminClient.create(properties)
I did try with jaas
on the proxy side with the same credentials, but was receiving the same error.
My specific error looks like:
INFO[2020-02-04T12:10:10Z] Starting kafka-proxy version 0.1.6
INFO[2020-02-04T12:10:10Z] Bootstrap server server:9092 advertised as 0.0.0.0:9092
INFO[2020-02-04T12:10:10Z] Listening on 0.0.0.0:9092 ([::]:9092) for remote server:9092
INFO[2020-02-04T12:10:10Z] Ready for new connections
INFO[2020-02-04T12:11:31Z] New connection for server:9092
DEBU[2020-02-04T12:11:31Z] Sending SaslHandshakeRequest mechanism: PLAIN version: 0
DEBU[2020-02-04T12:11:31Z] Successful SASL handshake. Available mechanisms: [PLAIN OAUTHBEARER]
DEBU[2020-02-04T12:11:31Z] Sending authentication opaque packets, mechanism PLAIN
DEBU[2020-02-04T12:11:31Z] Kafka request key -18175, version 0, length 369296128
INFO[2020-02-04T12:11:31Z] Reading data from local connection on 172.28.196.187:9092 from 172.20.80.195:60232 (server:9092) had error: api key -18175 is invalid
Where 172.28.196.187
is the kafka-proxy
IP and 172.20.80.195
is my localhost.
LMK if there is anything else I can provide and I am digging into our env more to see what I can find out. Thanks!
from kafka-proxy.
- Is the broker server:9092 really a TLS port (check e.g. with openssl s_client -connect server:9092) ?
in most cases the port 9092 is a plain listener https://docs.confluent.io/current/installation/configuration/broker-configs.html
control.plane.listener.name INTERNAL://192.1.1.8:9092, listener.security.protocol.map = INTERNAL:PLAINTEXT
if it is not, use the correct TLS port or do not use "--tls-enable" in the proxy.
If you want terminate TLS on proxy please use "--proxy-listener-tls-enable"
- What is the version of the broker ? please provide output from bin/kafka-broker-api-versions.sh
if your kafka client version is 2.4.0 and the server is also 2.4.0, the proxy will not work as the were changes in the kafka protocol which have not been implemented yet (it will be soon).
However if you downgrade the client to 2.3.x, older message version version will be negotiated and proxy will work.
from kafka-proxy.
@hekaldama Did you get kafka-proxy working properly with socks or http proxying towards Confluent Cloud? Very interested in any working configs if that is the case. Cheers.
from kafka-proxy.
@datafisk I was not able to work on it anymore after my original comment. I will definitely post back here if I figure anything out. We are going to be setting up AWS transit gateway to at least allow VPN -> confluent.cloud access.
from kafka-proxy.
@hekaldama Did you get kafka-proxy working properly with socks or http proxying towards Confluent Cloud? Very interested in any working configs if that is the case. Cheers.
I can forward-proxy to Confluent Cloud via squid but not via sasl_ssl. Only when I set sasl_plaintext on the client side.
from kafka-proxy.
I think ii got this sorted @sarwarbhuiyan , i will DM you a link.
from kafka-proxy.
I am running in to similar issues when connecting to confluent cloud from the kafka-proxy and wanted to see if anyone was successful with that:
had error: api key XXXX is invalid
My server command:
./kafka-proxy server --bootstrap-server-mapping "confluent-server:9092,0.0.0.0:9092" \
--debug-enable \
--log-level="debug" \
--tls-enable \
--tls-insecure-skip-verify
And the kafka properties on the server:
bootstrap.servers=confluent-server:9092
ssl.endpoint.identification.algorithm=https
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="{{ API_KEY }}" password="{{ API_SECRET }}";
Does anything jump out from my configuration or kafka-proxy setup?
from kafka-proxy.
@afoley-st don't you miss --sasl-enable
option?
from kafka-proxy.
@afoley-st don't you miss
--sasl-enable
option?
The same issue arises:
./kafka-proxy server --bootstrap-server-mapping "confluent-server:9092,0.0.0.0:9092" \
--debug-enable \
--log-level="debug" \
--sasl-enable \
--sasl-username="${SASL_USERNAME}" \
--sasl-password="${SASL_PASSWORD}" \
--tls-enable --tls-insecure-skip-verify
INFO[2021-06-02T13:22:40Z] Starting kafka-proxy version 0.2.8
INFO[2021-06-02T13:22:40Z] Bootstrap server confluent-server:9092 advertised as 0.0.0.0:9092
INFO[2021-06-02T13:22:40Z] Listening on 0.0.0.0:9092 ([::]:9092) for remote confluent-server:9092
INFO[2021-06-02T13:22:40Z] Ready for new connections
INFO[2021-06-02T13:22:50Z] New connection for confluent-server:9092
DEBU[2021-06-02T13:22:50Z] Sending SaslHandshakeRequest mechanism: PLAIN version: 0
DEBU[2021-06-02T13:22:50Z] Successful SASL handshake. Available mechanisms: [PLAIN OAUTHBEARER]
DEBU[2021-06-02T13:22:50Z] Sending authentication opaque packets, mechanism PLAIN
DEBU[2021-06-02T13:22:50Z] Kafka request key -3327, version 1, length 369296129
INFO[2021-06-02T13:22:50Z] Reading data from local connection on XXXXX:9092 from XXXXX:61791 (confluent-server:9092) had error: api key -3327 is invalid
Is the --proxy-listener-tls-enable
setting a requirement?
from kafka-proxy.
@afoley-st no --proxy-listener-tls-enable
should not be requirement. What version of client/server you have? What is your listener.security.protocol.map
from kafka-proxy.
@afoley-st no
--proxy-listener-tls-enable
should not be requirement. What version of client/server you have? What is yourlistener.security.protocol.map
It's running Apache Kafka Version 2.7.0
. I can't get the entire listener.security.protocol.map
(don't have access because its a managed service via confluent), but the bootstrap endpoint is SASL_SSL://confluent-server:9092
from kafka-proxy.
and client kafka version?
from kafka-proxy.
and client kafka version?
I've tried with both 2.7.0
and 2.8.0
from kafka-proxy.
@afoley-st did you try also with 2.3 see #28 (comment)
from kafka-proxy.
@afoley-st skip the properties for your kafka client, the kafka-proxy has the required settings as per this config and your client will just initiate the connection to the proxy with just the bootstrap as required config, the proxy takes care of the rest.
@afoley-st don't you miss
--sasl-enable
option?The same issue arises:
./kafka-proxy server --bootstrap-server-mapping "confluent-server:9092,0.0.0.0:9092" \ --debug-enable \ --log-level="debug" \ --sasl-enable \ --sasl-username="${SASL_USERNAME}" \ --sasl-password="${SASL_PASSWORD}" \ --tls-enable --tls-insecure-skip-verify INFO[2021-06-02T13:22:40Z] Starting kafka-proxy version 0.2.8 INFO[2021-06-02T13:22:40Z] Bootstrap server confluent-server:9092 advertised as 0.0.0.0:9092 INFO[2021-06-02T13:22:40Z] Listening on 0.0.0.0:9092 ([::]:9092) for remote confluent-server:9092 INFO[2021-06-02T13:22:40Z] Ready for new connections INFO[2021-06-02T13:22:50Z] New connection for confluent-server:9092 DEBU[2021-06-02T13:22:50Z] Sending SaslHandshakeRequest mechanism: PLAIN version: 0 DEBU[2021-06-02T13:22:50Z] Successful SASL handshake. Available mechanisms: [PLAIN OAUTHBEARER] DEBU[2021-06-02T13:22:50Z] Sending authentication opaque packets, mechanism PLAIN DEBU[2021-06-02T13:22:50Z] Kafka request key -3327, version 1, length 369296129 INFO[2021-06-02T13:22:50Z] Reading data from local connection on XXXXX:9092 from XXXXX:61791 (confluent-server:9092) had error: api key -3327 is invalidIs the
--proxy-listener-tls-enable
setting a requirement?
from kafka-proxy.
@afoley-st you should connect to your kafka-proxy server without SASL credentials/SSL (if are doing it right now, kafka-proxy already authenticates/encrypts connection with confluent kafka) as @datafisk mentioned
from kafka-proxy.
If I enable SASL, it's working perfectly but, If I remove --sasl-enable
flag and try to connect from client, this error will appear.
I was following this guide to connect to cluster.
https://docs.conduktor.io/kafka-cluster-connection/setting-up-a-connection-to-kafka/connecting-to-a-secure-kafka
from kafka-proxy.
If I enable SASL, it's working perfectly but, If I remove
--sasl-enable
flag and try to connect from client, this error will appear.I was following this guide to connect to cluster. https://docs.conduktor.io/kafka-cluster-connection/setting-up-a-connection-to-kafka/connecting-to-a-secure-kafka
After few hours debug, I realized the problem is protocol Iām using. Just changed it to SASL_PLAINTEXT
and then everything works perfectly.
from kafka-proxy.
Updating this ticket for the SASL passthrough issue against Confluent Cloud, that generates the API key error. Since Confluent Cloud uses. SASL_PLAN over TLS the proxy needs to be setup with TLS listeners and it will translate/proxy/passthrough the connection details to Confluent Cloud.
Below is an examle:
kafka-proxy server \
--bootstrap-server-mapping \
"pkc-lgwgm.eastus2.azure.confluent.cloud:9092,localhost:9092" \
--log-level DEBUG \
--proxy-listener-cert-file ./proxy.cert \
--proxy-listener-tls-enable \
--proxy-listener-key-file ./proxy.key \
--tls-enable
Then you just setup the kafka client to talk to the proxy, localhost in my example, so from kafkacat:
export USERNAME=<API_KEY/SASL_USERNAME>
export PASSWORD=<API_SECRET_KEY/SASL_PASSWORD>
export BROKERS=localhost:9092
kafkacat -b $BROKERS -L -X security.protocol=SASL_SSL -X sasl.mechanisms=PLAIN -X sasl.username=$USERNAME -X sasl.password=$PASSWORD
If you use self-signed certs for some testing you want to add -X enable.ssl.certificate.verification=false
to kafkacat to make testing the passthrough simpler.
from kafka-proxy.
can anyone post a fully working config + start scripts for:
kafka-proxy -> squid -> confluent kafka ?
I'm not able to get this to work even without squid. I've tried a number of different CLI options to kafka-proxy and with confluent-7.3.0/bin/kafka-console-producer, none of it works for me.
from kafka-proxy.
Related Issues (20)
- [Question] Can I attach 3 bootstrap server endpoints to a single port? HOT 1
- [Question] If my Kafka brokers are running version 2.8.1, should I be using kafka-proxy version 0.2.9? HOT 1
- "Metadata" request (ApiKey=3 and ApiVersion=5) in the Kafka Proxy is not following the protocol structure defined by Kafka protocol guide HOT 1
- [Question] is there a plan to release a Java implementation of Kafka Proxy ? HOT 1
- [Need Help] Sending Custom METADATA response through Kafka Proxy
- [Question] is there a plan to support HTTPS proxy ?
- will there be an update to resovle 7 vulnerabilitys
- tls: failed to parse private key AWS MSK HOT 6
- bad performance when executing kafka-producer-perf-test.sh HOT 3
- Can not use grepplabs/kafka-proxy ARM image as base image HOT 1
- one port mapping to 6 broker HOT 2
- AWS Invalid API Key. What did I miss ?
- tls: failed to parse private key HOT 2
- Can't get proxy-listener tls to work HOT 2
- Kafka 3.7.0 and producer error "produce version 10 is not supported" HOT 2
- Is Kafka-proxy support Kafka Cluster in KRaft mode?
- Experiencing issue with AWS MSK IAM between Proxy and Brokers, and SASL Plain between Client and Kafka Proxy
- Running kafka-proxy in k8s with more then 1 replica HOT 1
- MSK Serverless Net Address Not Found HOT 1
- Exposing proxy through istio virtual service HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
š Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ššš
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ā¤ļø Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kafka-proxy.