kow3ns / kubernetes-kafka Goto Github PK

Hello

I don't understand where consumers messages volumes are mapped to disk
i see that messages info is at /tmp/kafka-logs

I can see
--override log.dir=/var/lib/kafka \

may - logs.dir - ??

how can i map /tmp/kafka-logs using volume and pvc ?

Hi,
I'm trying to use your template to setup Kafka/Zookeeper on Kubernetes but am having a hard time understanding the link between PersistentVolume and volumeClaimTemplates.
Would you mind commenting on how to create the PersistentVolume pointing to a local path?
I have this setup, but when I apply your manifest I get the error below...

apiVersion: v1
kind: PersistentVolume
metadata:
  name: datadir
  labels:
    type: local
spec:
  storageClassName: datadir
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  hostPath:
    path: "/kubernetes/zk/config"

Error is:

PersistentVolumeClaim is not bound: "datadir-zk-0" (repeated 6 times)

and the Persistent Volume Claims stay in pending.

Any help is greatly appreciated.

Kind regards,

Eric V.

I am creating kafka and zookeeper cluster through given manifest file. Clusters and created and running good. But when I created NodePort to access kafka through external IP, it is throwing me error saying

WARN [Consumer clientId=consumer-1, groupId=console-consumer-25337] Error while fetching 
metadata with correlation id 11 : {test=LEADER_NOT_AVAILABLE}
 (org.apache.kafka.clients.NetworkClient)

I have tried Exposing kafka service externally issue suggestion, but still I am getting the error.
Any suggestion will help. Thank You

Hi,
We are able to run the Kafka console producer and consumer from within the cluster, please add documentation/suggestion on how to run the service from outside the cluster (maybe through NodePort) without nodectl run

Hi,
I am trying to expose my kafka externally but I am facing issues with it. I have created separate load balanced services for each broker(my nodes are private). Operations like listing topics work fine but my publisher/consumer fail as they are not able to resolve the brokers.

% ERROR: Local: Host resolution failure: kafka-0.kafka-hs.default.svc.cluster.local:9093/0: Failed to resolve 'kafka-0.kafka-hs.default.svc.cluster.local:9093': nodename nor servname provided, or not known
% ERROR: Local: Host resolution failure: kafka-1.kafka-hs.default.svc.cluster.local:9093/1: Failed to resolve 'kafka-1.kafka-hs.default.svc.cluster.local:9093': nodename nor servname provided, or not known

Can anyone please guide me towards the best way to do this.
Thanks.

When i tried to run this command "kubectl create -f zookeeper_mini.yaml" with the zookeeper config file , it came up with an error .

It says :
error: unable to recognize "zookeeper.yml": no matches for kind "StatefulSet" in version "apps/v1beta1"

And i changed "apps/v1beta2" but still same error. What is the problem that i cant see?

Please include this in mini:

---
apiVersion: v1
kind: Service
metadata:
  name: kafka-hs
  labels:
    app: kafka
spec:
  ports:
  - port: 9092
    name: client
  clusterIP: None
  selector:
    app: kafka