Giter Club home page Giter Club logo

test-clients's Introduction

Strimzi

Run Apache Kafka on Kubernetes and OpenShift

Build Status GitHub release License Twitter Follow Artifact Hub

Strimzi provides a way to run an Apache Kafka® cluster on Kubernetes or OpenShift in various deployment configurations. See our website for more details about the project.

Quick Starts

To get up and running quickly, check our Quick Start for Minikube, OKD (OpenShift Origin) and Kubernetes Kind.

Documentation

Documentation for the current main branch as well as all releases can be found on our website.

Roadmap

The roadmap of the Strimzi Operator project is maintained as GitHub Project.

Getting help

If you encounter any issues while using Strimzi, you can get help using:

Strimzi Community Meetings

You can join our regular community meetings:

Resources:

Contributing

You can contribute by:

  • Raising any issues you find using Strimzi
  • Fixing issues by opening Pull Requests
  • Improving documentation
  • Talking about Strimzi

All bugs, tasks or enhancements are tracked as GitHub issues. Issues which might be a good start for new contributors are marked with "good-start" label.

The Dev guide describes how to build Strimzi. Before submitting a patch, please make sure to understand, how to test your changes before opening a PR Test guide.

The Documentation Contributor Guide describes how to contribute to Strimzi documentation.

If you want to get in touch with us first before contributing, you can use:

License

Strimzi is licensed under the Apache License, Version 2.0

Container signatures

From the 0.38.0 release, Strimzi containers are signed using the cosign tool. Strimzi currently does not use the keyless signing and the transparency log. To verify the container, you can copy the following public key into a file:

-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAET3OleLR7h0JqatY2KkECXhA9ZAkC
TRnbE23Wb5AzJPnpevvQ1QUEQQ5h/I4GobB7/jkGfqYkt6Ct5WOU2cc6HQ==
-----END PUBLIC KEY-----

And use it to verify the signature:

cosign verify --key strimzi.pub quay.io/strimzi/operator:latest --insecure-ignore-tlog=true

Software Bill of Materials (SBOM)

From the 0.38.0 release, Strimzi publishes the software bill of materials (SBOM) of our containers. The SBOMs are published as an archive with SPDX-JSON and Syft-Table formats signed using cosign. For releases, they are also pushed into the container registry. To verify the SBOM signatures, please use the Strimzi public key:

-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAET3OleLR7h0JqatY2KkECXhA9ZAkC
TRnbE23Wb5AzJPnpevvQ1QUEQQ5h/I4GobB7/jkGfqYkt6Ct5WOU2cc6HQ==
-----END PUBLIC KEY-----

You can use it to verify the signature of the SBOM files with the following command:

cosign verify-blob --key cosign.pub --bundle <SBOM-file>.bundle --insecure-ignore-tlog=true <SBOM-file>

Strimzi is a Cloud Native Computing Foundation incubating project.

CNCF ><

test-clients's People

Contributors

alesj avatar frawless avatar henryzrncik avatar im-konge avatar kun-lu20 avatar michalxo avatar ppatierno avatar scholzj avatar tombentley avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

test-clients's Issues

Improve logging of HttpProducer and possibly HttpConsumer clients

It seems that from the codebase we do not log any message sent to the KafkaBridge instance f.e.,

oc logs -f http-producer-kgh6d
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2024-01-14 14:47:12.646.793762779232966 INFO  HttpProducerClient:52 - Starting io.strimzi.http.producer.HttpProducerClient with configuration:
HttpProducerConfiguration:
hostname='my-bridge-bridge-service',
port='8080',
topic='my-topic',
delay='0',
messageCount='1000000',
message='Hello world',
uri='http://my-bridge-bridge-service:8080/topics/my-topic'

This is the log of the HTTP producer without any additional logging. When I tried to use our client examples the output was the following:

oc logs -f java-http-producer-74cdcb77f-z8pqt | head -n 150
[2024-01-14T14:52:04.017+0000] Using Serial
[2024-01-14T14:52:04.017+0000] Version: 17.0.9+9-LTS (release)
[2024-01-14T14:52:04.017+0000] CPUs: 8 total, 1 available
[2024-01-14T14:52:04.017+0000] Memory: 256M
[2024-01-14T14:52:04.017+0000] Large Page Support: Disabled
[2024-01-14T14:52:04.017+0000] NUMA Support: Disabled
[2024-01-14T14:52:04.017+0000] Compressed Oops: Enabled (32-bit)
[2024-01-14T14:52:04.017+0000] Heap Min Capacity: 8M
[2024-01-14T14:52:04.017+0000] Heap Initial Capacity: 8M
[2024-01-14T14:52:04.017+0000] Heap Max Capacity: 126M
[2024-01-14T14:52:04.017+0000] Pre-touch: Disabled
[2024-01-14T14:52:04.075+0000] CDS archive(s) mapped at: [0x00007f8c53000000-0x00007f8c53bc8000-0x00007f8c53bc8000), size 12353536, SharedBaseAddress: 0x00007f8c53000000, ArchiveRelocationMode: 1.
[2024-01-14T14:52:04.075+0000] Compressed class space mapped at: 0x00007f8c54000000-0x00007f8c94000000, reserved size: 1073741824
[2024-01-14T14:52:04.075+0000] Narrow klass base: 0x00007f8c53000000, Narrow klass shift: 0, Narrow klass range: 0x100000000
[2024-01-14T14:52:04.477+0000] GC(0) Pause Young (Allocation Failure)
[2024-01-14T14:52:04.479+0000] GC(0) DefNew: 2176K(2432K)->256K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 0K(256K)->256K(256K)
[2024-01-14T14:52:04.479+0000] GC(0) Tenured: 0K(5504K)->755K(5504K)
[2024-01-14T14:52:04.479+0000] GC(0) Metaspace: 540K(704K)->540K(704K) NonClass: 497K(576K)->497K(576K) Class: 42K(128K)->42K(128K)
[2024-01-14T14:52:04.479+0000] GC(0) Pause Young (Allocation Failure) 2M->0M(7M) 2.137ms
[2024-01-14T14:52:04.479+0000] GC(0) User=0.00s Sys=0.00s Real=0.01s
[2024-01-14T14:52:05.370+0000] GC(1) Pause Young (Allocation Failure)
[2024-01-14T14:52:05.375+0000] GC(1) DefNew: 2432K(2432K)->256K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 256K(256K)->256K(256K)
[2024-01-14T14:52:05.375+0000] GC(1) Tenured: 755K(5504K)->1124K(5504K)
[2024-01-14T14:52:05.375+0000] GC(1) Metaspace: 1581K(1728K)->1581K(1728K) NonClass: 1430K(1536K)->1430K(1536K) Class: 151K(192K)->151K(192K)
[2024-01-14T14:52:05.375+0000] GC(1) Pause Young (Allocation Failure) 3M->1M(7M) 5.123ms
[2024-01-14T14:52:05.375+0000] GC(1) User=0.01s Sys=0.00s Real=0.00s
[2024-01-14T14:52:05.771+0000] GC(2) Pause Young (Allocation Failure)
[2024-01-14T14:52:05.773+0000] GC(2) DefNew: 2432K(2432K)->69K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 256K(256K)->69K(256K)
[2024-01-14T14:52:05.773+0000] GC(2) Tenured: 1124K(5504K)->1371K(5504K)
[2024-01-14T14:52:05.773+0000] GC(2) Metaspace: 2206K(2368K)->2206K(2368K) NonClass: 1963K(2048K)->1963K(2048K) Class: 243K(320K)->243K(320K)
[2024-01-14T14:52:05.773+0000] GC(2) Pause Young (Allocation Failure) 3M->1M(7M) 1.995ms
[2024-01-14T14:52:05.773+0000] GC(2) User=0.00s Sys=0.00s Real=0.00s
[2024-01-14T14:52:06.169+0000] GC(3) Pause Young (Allocation Failure)
[2024-01-14T14:52:06.171+0000] GC(3) DefNew: 2245K(2432K)->149K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 69K(256K)->149K(256K)
[2024-01-14T14:52:06.171+0000] GC(3) Tenured: 1371K(5504K)->1371K(5504K)
[2024-01-14T14:52:06.171+0000] GC(3) Metaspace: 2835K(2944K)->2835K(2944K) NonClass: 2496K(2560K)->2496K(2560K) Class: 339K(384K)->339K(384K)
[2024-01-14T14:52:06.171+0000] GC(3) Pause Young (Allocation Failure) 3M->1M(7M) 1.404ms
[2024-01-14T14:52:06.171+0000] GC(3) User=0.00s Sys=0.00s Real=0.00s
[2024-01-14T14:52:07.171+0000] GC(4) Pause Young (Allocation Failure)
[2024-01-14T14:52:07.173+0000] GC(4) DefNew: 2325K(2432K)->255K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 149K(256K)->255K(256K)
[2024-01-14T14:52:07.173+0000] GC(4) Tenured: 1371K(5504K)->1674K(5504K)
[2024-01-14T14:52:07.174+0000] GC(4) Metaspace: 4135K(4288K)->4135K(4288K) NonClass: 3627K(3712K)->3627K(3712K) Class: 508K(576K)->508K(576K)
[2024-01-14T14:52:07.174+0000] GC(4) Pause Young (Allocation Failure) 3M->1M(7M) 2.844ms
[2024-01-14T14:52:07.174+0000] GC(4) User=0.01s Sys=0.00s Real=0.00s
[2024-01-14T14:52:07.778+0000] GC(5) Pause Young (Allocation Failure)
[2024-01-14T14:52:07.872+0000] GC(5) DefNew: 2431K(2432K)->256K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 255K(256K)->256K(256K)
[2024-01-14T14:52:07.872+0000] GC(5) Tenured: 1674K(5504K)->2471K(5504K)
[2024-01-14T14:52:07.872+0000] GC(5) Metaspace: 4604K(4800K)->4604K(4800K) NonClass: 4045K(4160K)->4045K(4160K) Class: 558K(640K)->558K(640K)
[2024-01-14T14:52:07.872+0000] GC(5) Pause Young (Allocation Failure) 4M->2M(7M) 93.676ms
[2024-01-14T14:52:07.872+0000] GC(5) User=0.01s Sys=0.00s Real=0.10s
[2024-01-14T14:52:08.381+0000] GC(6) Pause Young (Allocation Failure)
[2024-01-14T14:52:08.468+0000] GC(6) DefNew: 2432K(2432K)->241K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 256K(256K)->241K(256K)
[2024-01-14T14:52:08.468+0000] GC(6) Tenured: 2471K(5504K)->2722K(5504K)
[2024-01-14T14:52:08.468+0000] GC(6) Metaspace: 5391K(5568K)->5391K(5568K) NonClass: 4727K(4800K)->4727K(4800K) Class: 664K(768K)->664K(768K)
[2024-01-14T14:52:08.468+0000] GC(6) Pause Young (Allocation Failure) 4M->2M(7M) 86.922ms
[2024-01-14T14:52:08.468+0000] GC(6) User=0.00s Sys=0.00s Real=0.08s
[2024-01-14T14:52:09.268+0000] GC(7) Pause Young (Allocation Failure)
[2024-01-14T14:52:09.271+0000] GC(7) DefNew: 2417K(2432K)->256K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 241K(256K)->256K(256K)
[2024-01-14T14:52:09.271+0000] GC(7) Tenured: 2722K(5504K)->3177K(5504K)
[2024-01-14T14:52:09.271+0000] GC(7) Metaspace: 7204K(7424K)->7204K(7424K) NonClass: 6330K(6464K)->6330K(6464K) Class: 874K(960K)->874K(960K)
[2024-01-14T14:52:09.271+0000] GC(7) Pause Young (Allocation Failure) 5M->3M(7M) 3.343ms
[2024-01-14T14:52:09.271+0000] GC(7) User=0.01s Sys=0.00s Real=0.01s
[2024-01-14T14:52:10.067+0000] GC(8) Pause Young (Allocation Failure)
[2024-01-14T14:52:10.073+0000] GC(8) DefNew: 2432K(2432K)->255K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 256K(256K)->255K(256K)
[2024-01-14T14:52:10.073+0000] GC(8) Tenured: 3177K(5504K)->3863K(5504K)
[2024-01-14T14:52:10.073+0000] GC(8) Metaspace: 8607K(8832K)->8607K(8832K) NonClass: 7559K(7680K)->7559K(7680K) Class: 1047K(1152K)->1047K(1152K)
[2024-01-14T14:52:10.073+0000] GC(8) Pause Young (Allocation Failure) 5M->4M(7M) 5.947ms
[2024-01-14T14:52:10.073+0000] GC(8) User=0.01s Sys=0.00s Real=0.01s
[2024-01-14T14:52:10.968+0000] GC(9) Pause Young (Allocation Failure)
[2024-01-14T14:52:10.974+0000] GC(9) DefNew: 2431K(2432K)->255K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 255K(256K)->255K(256K)
[2024-01-14T14:52:10.974+0000] GC(9) Tenured: 3863K(5504K)->4399K(5504K)
[2024-01-14T14:52:10.974+0000] GC(9) Metaspace: 10138K(10368K)->10138K(10368K) NonClass: 8926K(9024K)->8926K(9024K) Class: 1211K(1344K)->1211K(1344K)
[2024-01-14T14:52:10.974+0000] GC(9) Pause Young (Allocation Failure) 6M->4M(7M) 5.662ms
[2024-01-14T14:52:10.974+0000] GC(9) User=0.00s Sys=0.00s Real=0.01s
[2024-01-14T14:52:11.172+0000] GC(10) Pause Young (Allocation Failure)
[2024-01-14T14:52:11.175+0000] GC(10) DefNew: 2431K(2432K)->256K(2432K) Eden: 2176K(2176K)->0K(2176K) From: 255K(256K)->256K(256K)
[2024-01-14T14:52:11.175+0000] GC(10) Tenured: 4399K(5504K)->4939K(5504K)
[2024-01-14T14:52:11.175+0000] GC(10) Metaspace: 10189K(10432K)->10189K(10432K) NonClass: 8974K(9088K)->8974K(9088K) Class: 1215K(1344K)->1215K(1344K)
[2024-01-14T14:52:11.175+0000] GC(10) Pause Young (Allocation Failure) 6M->5M(7M) 3.354ms
[2024-01-14T14:52:11.175+0000] GC(10) User=0.00s Sys=0.00s Real=0.00s
[2024-01-14T14:52:11.569+0000] GC(11) Pause Young (Allocation Failure)
[2024-01-14T14:52:11.571+0000] GC(11) Pause Young (Allocation Failure) 7M->5M(7M) 2.381ms
[2024-01-14T14:52:11.571+0000] GC(11) User=0.00s Sys=0.00s Real=0.00s
[2024-01-14T14:52:11.571+0000] GC(12) Pause Full (Allocation Failure)
[2024-01-14T14:52:11.571+0000] GC(12) Phase 1: Mark live objects
[2024-01-14T14:52:11.581+0000] GC(12) Phase 1: Mark live objects 9.201ms
[2024-01-14T14:52:11.581+0000] GC(12) Phase 2: Compute new object addresses
[2024-01-14T14:52:11.583+0000] GC(12) Phase 2: Compute new object addresses 2.538ms
[2024-01-14T14:52:11.583+0000] GC(12) Phase 3: Adjust pointers
[2024-01-14T14:52:11.590+0000] GC(12) Phase 3: Adjust pointers 6.258ms
[2024-01-14T14:52:11.667+0000] GC(12) Phase 4: Move objects
[2024-01-14T14:52:11.668+0000] GC(12) Phase 4: Move objects 0.410ms
[2024-01-14T14:52:11.668+0000] GC(12) DefNew: 2432K(2432K)->0K(4160K) Eden: 2176K(2176K)->0K(3712K) From: 256K(256K)->0K(448K)
[2024-01-14T14:52:11.668+0000] GC(12) Tenured: 4939K(5504K)->5486K(9148K)
[2024-01-14T14:52:11.668+0000] GC(12) Metaspace: 10585K(10880K)->10585K(10880K) NonClass: 9318K(9472K)->9318K(9472K) Class: 1267K(1408K)->1267K(1408K)
[2024-01-14T14:52:11.668+0000] GC(12) Pause Full (Allocation Failure) 5M->5M(12M) 96.977ms
[2024-01-14T14:52:11.668+0000] GC(12) User=0.02s Sys=0.00s Real=0.09s
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2024-01-14 14:52:12 INFO  HttpProducer:76 - Scheduling periodic send: null messages every 1000 ms ...
2024-01-14 14:52:12 INFO  HttpProducer:78 - ... 0 messages sent
2024-01-14 14:52:12 INFO  HttpProducer:59 - Waiting for sending all messages
2024-01-14 14:52:12 INFO  HttpProducer:100 - Sending message = {"key":"key-0","value":"Hello world-0"}
[2024-01-14T14:52:12.669+0000] GC(13) Pause Young (Allocation Failure)
[2024-01-14T14:52:12.673+0000] GC(13) DefNew: 3712K(4160K)->448K(4160K) Eden: 3712K(3712K)->0K(3712K) From: 0K(448K)->448K(448K)
[2024-01-14T14:52:12.673+0000] GC(13) Tenured: 5486K(9148K)->5589K(9148K)
[2024-01-14T14:52:12.673+0000] GC(13) Metaspace: 12933K(13248K)->12933K(13248K) NonClass: 11366K(11520K)->11366K(11520K) Class: 1566K(1728K)->1566K(1728K)
[2024-01-14T14:52:12.673+0000] GC(13) Pause Young (Allocation Failure) 8M->5M(12M) 3.465ms
[2024-01-14T14:52:12.673+0000] GC(13) User=0.01s Sys=0.00s Real=0.00s
[2024-01-14T14:52:13.073+0000] GC(14) Pause Young (Allocation Failure)
[2024-01-14T14:52:13.076+0000] GC(14) DefNew: 4158K(4160K)->143K(4160K) Eden: 3710K(3712K)->0K(3712K) From: 448K(448K)->143K(448K)
[2024-01-14T14:52:13.076+0000] GC(14) Tenured: 5589K(9148K)->6675K(9148K)
[2024-01-14T14:52:13.076+0000] GC(14) Metaspace: 14108K(14336K)->14108K(14336K) NonClass: 12490K(12608K)->12490K(12608K) Class: 1617K(1728K)->1617K(1728K)
[2024-01-14T14:52:13.076+0000] GC(14) Pause Young (Allocation Failure) 9M->6M(12M) 3.586ms
[2024-01-14T14:52:13.076+0000] GC(14) User=0.00s Sys=0.00s Real=0.00s
[2024-01-14T14:52:13.874+0000] GC(15) Pause Young (Allocation Failure)
[2024-01-14T14:52:13.878+0000] GC(15) DefNew: 3796K(4160K)->448K(4160K) Eden: 3653K(3712K)->0K(3712K) From: 143K(448K)->448K(448K)
[2024-01-14T14:52:13.878+0000] GC(15) Tenured: 6675K(9148K)->6685K(9148K)
[2024-01-14T14:52:13.878+0000] GC(15) Metaspace: 15397K(15680K)->15397K(15680K) NonClass: 13650K(13824K)->13650K(13824K) Class: 1746K(1856K)->1746K(1856K)
[2024-01-14T14:52:13.879+0000] GC(15) Pause Young (Allocation Failure) 10M->6M(12M) 4.524ms
[2024-01-14T14:52:13.879+0000] GC(15) User=0.01s Sys=0.00s Real=0.00s
[2024-01-14T14:52:15.573+0000] GC(16) Pause Young (Allocation Failure)
[2024-01-14T14:52:15.578+0000] GC(16) DefNew: 4160K(4160K)->448K(4160K) Eden: 3712K(3712K)->0K(3712K) From: 448K(448K)->448K(448K)
[2024-01-14T14:52:15.578+0000] GC(16) Tenured: 6685K(9148K)->7298K(9148K)
[2024-01-14T14:52:15.578+0000] GC(16) Metaspace: 17434K(17728K)->17434K(17728K) NonClass: 15415K(15552K)->15415K(15552K) Class: 2018K(2176K)->2018K(2176K)
[2024-01-14T14:52:15.578+0000] GC(16) Pause Young (Allocation Failure) 10M->7M(12M) 4.780ms
[2024-01-14T14:52:15.578+0000] GC(16) User=0.00s Sys=0.00s Real=0.00s
2024-01-14 14:52:15 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":0}]}
2024-01-14 14:52:15 INFO  HttpProducer:100 - Sending message = {"key":"key-1","value":"Hello world-1"}
2024-01-14 14:52:15 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":1}]}
2024-01-14 14:52:15 INFO  HttpProducer:100 - Sending message = {"key":"key-2","value":"Hello world-2"}
2024-01-14 14:52:15 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":2}]}
2024-01-14 14:52:15 INFO  HttpProducer:100 - Sending message = {"key":"key-3","value":"Hello world-3"}
2024-01-14 14:52:15 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":3}]}
2024-01-14 14:52:16 INFO  HttpProducer:100 - Sending message = {"key":"key-4","value":"Hello world-4"}
2024-01-14 14:52:16 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":4}]}
2024-01-14 14:52:17 INFO  HttpProducer:100 - Sending message = {"key":"key-5","value":"Hello world-5"}
2024-01-14 14:52:17 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":5}]}
2024-01-14 14:52:18 INFO  HttpProducer:100 - Sending message = {"key":"key-6","value":"Hello world-6"}
2024-01-14 14:52:18 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":6}]}
2024-01-14 14:52:19 INFO  HttpProducer:100 - Sending message = {"key":"key-7","value":"Hello world-7"}
2024-01-14 14:52:19 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":7}]}
2024-01-14 14:52:20 INFO  HttpProducer:100 - Sending message = {"key":"key-8","value":"Hello world-8"}
2024-01-14 14:52:20 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":8}]}
2024-01-14 14:52:21 INFO  HttpProducer:100 - Sending message = {"key":"key-9","value":"Hello world-9"}
2024-01-14 14:52:21 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":9}]}
2024-01-14 14:52:22 INFO  HttpProducer:100 - Sending message = {"key":"key-10","value":"Hello world-10"}
2024-01-14 14:52:22 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":10}]}
2024-01-14 14:52:23 INFO  HttpProducer:100 - Sending message = {"key":"key-11","value":"Hello world-11"}
2024-01-14 14:52:23 INFO  HttpProducer:115 - code = 200, metadata = {"offsets":[{"partition":0,"offset":11}]}
2024-01-14 14:52:24 INFO  HttpProducer:100 - Sending message = {"key":"key-12","value":"Hello world-12"}

so I would assume the same from the test clients. Specically such logging could be added here

public void sendMessages() {
        ProducerRecord producerRecord = configuration.getDelay() == 0 ? generateMessages() : generateMessage(messageIndex);

        try {
            HttpResponse httpResponse = httpHandle.finish(client.send(httpHandle.build(producerRecord.context()), HttpResponse.BodyHandlers.ofString()));

            if (httpResponse.statusCode() != HttpResponseStatus.OK.code()) {
                LOGGER.error("Error while sending message {} : {}", producerRecord.message(), httpResponse.body());
                throw new RuntimeException("Failed to send message due to: " + httpResponse.body());
            } else if (httpResponse.body().equals("[]") && httpResponse.statusCode() == HttpResponseStatus.OK.code()) {
                LOGGER.info("Array with messages is empty, no messages were received!");
            }

            OffsetRecordSent[] offsetRecordSent = OffsetRecordSentUtils.parseOffsetRecordsSent(httpResponse.body().toString());
            OffsetRecordSentUtils.logOffsetRecordsSent(offsetRecordSent);
            messageSuccessfullySent += offsetRecordSent.length;
        } catch (Exception e) {
            LOGGER.error("Caught exception during message send");
            e.printStackTrace();
            throw new RuntimeException("Failed to send message due to: " + e.getMessage());
        }
    }

Clients configuration classes should be merged

We have different classes for each client inside kafka and http modules. But for example each client in kafka module shares almost the same configuration, which can be merged into one class - that means have one class f.e. for security and one for Kafka config, where will be default configuration (such as bootstrap server) - same should be for http module.

Move from OpenTelemetry Jaeger exporter to OTLP exporter

Referring to this work strimzi/strimzi-kafka-operator#7597 about the Strimzi operator moving from the usage of OpenTelemetry Jaeger exporter to the OTLP exporter, we should do the same for the test-clients.
This is needed because operator is using the test-clients in the KafkaTracingClients so corresponding STs for the above PR cannot pass without this change here.
We would need updating the test-clients and a new patch release to be used in that PR to allow STs passing (leaving the main operator branch still using the current test-clients release in order to have all the other operator PRs working fine).

Admin-client doesn't work on other architectures - aarch64, ppc64le, s390x

As part of our build pipeline, we are building multi-arch images. The clients like (HTTP) producer, (HTTP) consumer, streams work fine, there is no issue regarding different architecture.
But for the admin-client, we are getting errors like:

exec /admin-client/admin-client: exec format error
command terminated with exit code 1

when used on different architecture than x86.
This issue have to be investigated and fixed.

Admin client returns wrong return code on failure or exception

Current implementation of admin client is wrong - if the client hits an exception during creating/deleting/altering KafkaTopics, it just returns the 0 return code and the Job (when the client is used in K8s Job) shows that it's successful.

This behavior is wrong, it should return proper return code and, which would be the best, pop the exception somewhere in the Job status.

Admin client cannot create topics with additional config

For Kroxylicious system tests (related to this issue: kroxylicious/kroxylicious#732) we will need to make the topics more configurable. Currently we only have --topic-partitions and --topic-rep-factor as parameters and we would like to be able to configure things like:

  • min.insync.replicas
  • compression type
  • or any other possible configuration

For that, as a suggestion, we can use a parameter such as --config and add additional config separated by spaces:

--config=min.insync.replicas=2 compression.type=lz4 blah=2

Tracing - OpenTelemetry part is not working

When we try to use OpenTelemetry system for the tracing clients, it will just simply not create desired traces, even thought the OTel is initialized and everything seems to be set correctly.

OpenTracing part is working without a problem.

There is no hint in logs what should cause this.

Admin-client: add `if-exists` option to delete a topic

As part of the development of the Kroxylicious system tests, we have found the need of having an option like kafka already have to delete a topic only if it exists and not report and error if don't.

I guess it should be straight forward to add this --if-exists option already supported and nothing else need to be done.

Admin-client doesn't handle situations when topics are in "marked for deletion" state

During our testing we are usually creating, deleting, and then recreating the Kafka topics in relatively quick way (and after few seconds/minutes), so we are sometimes hitting issues that topic that we want to recreate is in marked for deletion state.

This "label" cannot be listed using the listTopics() or anything like that.

We should investigate how we can fix this - either waiting in the delete method for the actual deletion (trying to delete it with some error handling + having some kind of --wait flag) or try to create the topic multiple times.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.