Comments (19)
Glad to hear, it would be great!
As far as SASL is concerned - it's quite a simple thing when you get it, so I can describe it here.
To setup SASL authentification in a kafka cluster you shoul fulfill several steps:
- Add relevant properties in properies files
- Make jaas-file with authentication information
- Pass JVM a special environment parametr
- Run zookeeper/kafka/producers/consumers explicitly passing modifyed config files
Now in details:
- Zookeper
add this at config/zookeeper.properies
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000
make jaas file, for example config/jaas_zookeeper.conf with lines
Server {
org.apache.zookeeper.server.auth.DigestLoginModule required
username="admin"
password="admin-secret"
user_zooclient="client-secret";
};
pass JVM a special param, in linux you can do it this way:
export SERVER_JVMFLAGS="-Djava.security.auth.login.config=/home/nkm/apps/kafka_2.11-2.0.0/config/jaas_zookeeper.conf"
and run zookeeper
bin/zookeeper-server-start.sh config/zookeeper.properties
- Kafka broker
modify or add this properies at config/server.properies
advertised.host.name=localhost
advertised.listeners=SASL_PLAINTEXT://localhost:9092
allow.everyone.if.no.acl.found=true
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
listeners=SASL_PLAINTEXT://localhost:9092
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
security.inter.broker.protocol=SASL_PLAINTEXT
super.users=User:admin
make jaas file, for example config/jaas_kafka_server.conf with lines
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="admin"
password="admin-secret"
user_admin="admin-secret"
user_kafkaclient="client-secret";
};
Client {
org.apache.zookeeper.server.auth.DigestLoginModule required
username="zooclient"
password="client-secret";
};
pass JVM a special param, in linux you can do it this way:
export KAFKA_OPTS="-Djava.security.auth.login.config=/home/nkm/apps/kafka_2.11-2.0.0/config/jaas_kafka_server.conf"
and run kafka broker
bin/kafka-server-start.sh config/server.properties
- Kafka producer/consumer
make config/client.properties and add this
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
make jaas file, for example config/jaas_kafka_client.conf with lines
KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="kafkaclient"
password="client-secret";
};
pass JVM a special param, in linux you can do it this way:
export KAFKA_OPTS="-Djava.security.auth.login.config=/home/nkm/apps/kafka_2.11-2.0.0/config/jaas_kafka_client.conf"
and run producer
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test_topic --producer.config config/client.properties
run consumer
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test_topic --from-beginning --consumer.config config/client.properties
So after all kafka broker connects to Zookeper using Client data in jaas file (it must correspond to zookeeper Server jaas data) and consumers and producers connect to broker using KafkaClient jaas data (it must correspond to KafkaServer authentication data)
And don't forget to replace "/home/nkm/apps/kafka_2.11-2.0.0" with your own kafka directory)
from kafka-webview.
SASL JAAS Config (only applies to SASL based security) - used for yahoo kafka manager
com.sun.security.auth.module.Krb5LoginModule required doNotPrompt=true useTicketCache=false principal="[email protected]" useKeyTab=true serviceName="kafka" keyTab="/home/user/user.keytab" client=true;
here is the jaas.conf file that clients uses
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
useTicketCache=false
principal="[email protected]"
useKeyTab=true
serviceName="kafka"
keyTab="/home/user/user.keytab"
client=true;
};
Client {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
useTicketCache=false
principal="[email protected]"
useKeyTab=true
serviceName="kafka"
keyTab="/home/user/user.keytab"
client=true;
};
from kafka-webview.
@Crim yes you are right.
One thing to point out is when we have SASL_SSL; we just use truststore(no ssl Auth - so keystore is not needed).
from kafka-webview.
Do you know of a good online resource/tutorial for setting up SASL authentication on brokers? If I can configure it locally I can likely work out how to include it in the app.
from kafka-webview.
@Crim - I can help with this. What distro of Kafka you need? (Apache/Horotonworks/Cloudera/Confluent)...
This is a good doc which covers all vendor distros... Probably you need to setup a KDC, if you don't have one already.
from kafka-webview.
just standard Apache's Kafka, or whatever is easiest to get up and going really.
Thanks!
from kafka-webview.
That worked a treat @nadeevkm Thanks a ton! I'll poke around this weekend and get this working in the webapp.
from kafka-webview.
How does this UI look for setting up a cluster using SASL? Any suggestions or improvements?
Plain SASL authentication
GSSAPI / Custom Mechanism
Selecting GSSAPI or a Custom mechanism would basically just prompt you to enter your own JAAS configuration manually.
from kafka-webview.
Looks Great. We Use GSSAPI(Kerberos) with SSL as SASL_SSL. So It should also take path to trust store and its password in SSL Settings(when no ssl auth). Also, it should ask for Kafka security protocol options as - PLAINTEXT/SASL_PLAINTEXT/SASL_SSL/SSL.
from kafka-webview.
Just to make sure I'm understanding correctly....since the app already supports PLAINTEXT and SSL, when I add SASL support,
- PLAIN when you don't have SSL or SASL options enabled
- SSL when you have just the currently supported SSL options enabled.
- SASL_PLAIN when you do NOT have the currently supported SSL settings enabled, and have the new SASL settings enabled.
- SASL_SSL a combination of the existing SSL settings enabled, as well as the new SASL settings
This sounds correct?
from kafka-webview.
Good to know!
from kafka-webview.
I've published a new release that should support this. Thanks!
from kafka-webview.
Thank you very much for your work! I'll check It in a few days!
from kafka-webview.
Thanks @Crim. when we have SASL auth, we dont need SSL auth; can you check on this; for SASL_SSL we just need SASL conf and SSL truststore.
from kafka-webview.
I am able to setup with SASL. I can use it. But, i am getting below error in Cluster Broker View:
Error: org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_CONFIGS
from kafka-webview.
@thammuio I've created issue #115 for your error.
from kafka-webview.
Sounds like when SASL is enabled, there's no need for the keystore and associated password?
from kafka-webview.
Thanks @Crim . Yes, when SASL is enabled no need of keystore details.
from kafka-webview.
created issue #116 to deal with no longer requiring a KEYSTORE in this scenario and closing out this ticket.
Thanks!
from kafka-webview.
Related Issues (20)
- Lets add warning log msgs when ignoring or overwriting user defined configuration values for deserializers HOT 1
- Kafka-Webview should provide option to "Stringify" ByteArray messages HOT 2
- Failing to connect to GSSAPI/SSL cluster HOT 2
- Custom protobuf deserializer failed because of pb version conflict HOT 2
- On View page, the "previous button" jump is twice as big as the "next button" jump. HOT 1
- use custom config.yml with docker
- Repository with ID="orgsourcelab-1031" not found HOT 1
- Expose actuator/health path without login HOT 1
- Selecting a Partition filter from a 'Stream' persists the partition as an enforced filter.
- Allow Filtering on Views Page HOT 3
- Consumer poll timeout is hardcoded
- How to Build this as a Single Jar/War file to deploy in server which doesn't have MVN/JDK HOT 2
- Disable server host name verification HOT 3
- "Unknown magic byte!" when deserializing avro message with TopicRecordNameStrategy HOT 2
- Mistake in environment variable name HOT 2
- Feature Request: Make `requestTimeoutMs` configurable per cluster HOT 3
- Security Update HOT 3
- [Docker] Execute web application fail while upgrade from v2.4.0 to 2.8.1 HOT 5
- Is webview vulnerable to Spring4Shell vulnerablity (CVE-2022-22965)? HOT 3
- unable to connect to AWS MSK clusters HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kafka-webview.