sngular / kloadgen Goto Github PK

Hi is it possible to parameterize the Kafka Load Generator Config field list values ?

my scenarios is message.key.value have to be unique for every 4 consecutive messages that we post to Kafka

1. i need to generate a unique value for the message.key.value
2. parameterize the Kafka Load Generator Config field list values

will normal CSV data config will work here to pass the messages ( this is to mimic a scenario and need to send specific messages and not random generated ones )

let me know how do i achieve this

example

message Key value : 123456 then is send four messages taken from CSV and send to kafka topic and then change the message key value for the next iteration .

I'm trying to test Kloadgen. I built the standalone jar from master branch and use the included Example-Test-Plan.jmx. I only updated the bootstrap.servers and kafka.topic.name in this jmx file. When I try to run it, I'm getting the following exception:

Exception in thread "Thread Group 1-1" java.lang.Error: This should never happen. Tried to invoke:
net.coru.kloadgen.config.valueserialized.ValueSerializedConfigElement#setValueSchemaProperties(String )
at org.apache.jmeter.testbeans.TestBeanHelper.invokeOrBailOut(TestBeanHelper.java:168)
at org.apache.jmeter.testbeans.TestBeanHelper.prepare(TestBeanHelper.java:96)
at org.apache.jmeter.threads.TestCompiler.trackIterationListeners(TestCompiler.java:182)
at org.apache.jmeter.threads.TestCompiler.subtractNode(TestCompiler.java:136)
at org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:997)
at org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:994)
at org.apache.jorphan.collections.HashTree.traverse(HashTree.java:976)
at org.apache.jmeter.threads.JMeterThread.initRun(JMeterThread.java:704)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:252)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.lang.IllegalArgumentException: argument type mismatch
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.apache.jmeter.testbeans.TestBeanHelper.invokeOrBailOut(TestBeanHelper.java:166)
... 9 more

Is there anything else I need to update in the jmx file to use it?

On clicking load subject in Kafka Load generator config, getting below error.

Org.apache.avro.AvroRuntimeException: Not an array:["null", {"type" :"array", "items" :{"type" :"record", "name" :"UserRule", "namespace" :"com.user.details","fields":[{"name":"code", "type" :"int"}, {"name" :"override", "type" :{"type" :"string", "avro.java.string":"string"}}]}}]

Below is the avro schema.
{
"type":"record",
"name":"User",
"namespace":"com.user.details",
"fields":[{
"name":"username",
"type":"string"
},
{"name":"blocks",
"type":["null", {
"type" :"array",
"items" :{
"type" :"record",
"name" :"UserRule",
"fields" :[{
"name" :"code",
"type" :"int"},
{
"name" :"override",
"type" :["null", "string"],
"default":null
}]}}],
"default":null
}]}

Could you please look into why the error is coming up?

With client version 5.5.1 I can't work with JSON schemas - this is my problem.

In SchemaExtractorImpl.java schemaRegistryClient use deprecated method getById(int id), and in this method:
ParsedSchema schema = this.getSchemaById(id); return schema instanceof AvroSchema ? ((AvroSchema)schema).rawSchema() : null;

If our schema is JsonSchema we got NPE. And main problem - ParsedSchema in case AvroSchema by method rawSchema returns Schema object, in other case like JsonSchema returns ANOTHER Schema object.

Please fix it!

2020-09-08 23:26:19,331 ERROR o.a.j.t.JMeterThread: Error while processing sampler: 'Java Request'.
org.apache.avro.AvroRuntimeException: Not an array: ["null",{"type":"array","items":{"type":"record","name":"PCC_Array_Details","namespace":"com.bell.pec.out.data","fields":[{"name":"LOB_PCC","type":{"type":"array","items":{"type":"record","name":"LOB_Details_PCC","fields":[{"name":"LOB_Code","type":"int"},{"name":"Billed_from","type":"int"},{"name":"Nbr_Of_Days","type":"int"},{"name":"LOB_PCC_BLDT1","type":"float"},{"name":"LOB_PCC_BLDT2","type":"float"},{"name":"LOB_PCC_BLDT3","type":"float"}]}}},{"name":"Total","type":{"type":"record","name":"Total_PCC_Details","fields":[{"name":"PCC_BLDT1","type":"float"},{"name":"PCC_BLDT2","type":"float"},{"name":"PCC_BLDT3","type":"float"}]}}]}}]
at org.apache.avro.Schema.getElementType(Schema.java:356) ~[kloadgen-plugin-2.0.1.jar:?]
at net.coru.kloadgen.processor.AvroSchemaProcessor.next(AvroSchemaProcessor.java:76) ~[kloadgen-2.0.1.jar:?]
at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.nextMessage(AvroLoadGenerator.java:60) ~[kloadgen-2.0.1.jar:?]
at net.coru.kloadgen.sampler.ConfluentKafkaSampler.runTest(ConfluentKafkaSampler.java:253) ~[kloadgen-2.0.1.jar:?]
at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:197) ~[ApacheJMeter_java.jar:5.3]
at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:630) ~[ApacheJMeter_core.jar:5.3]
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:558) ~[ApacheJMeter_core.jar:5.3]
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:489) [ApacheJMeter_core.jar:5.3]
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256) [ApacheJMeter_core.jar:5.3]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_201]

Hi,

First of all, thanks for building this wonderful plugin! I have used it successfully for some cases but seeing few issues:

1- I've built the Jar but it's missing Kafka File Load Generator Config. Does it have any prerequisites?
2- When I use Schema Registry Config Element (successfully), and Value Schema Load Generator Config, my Java request throws "error serializing avro message".
3- I changed the approach and removed "Schema Registry Config Element", and tried using "Schema File Load Generator Config" instead. With this element, my producer "net.coru.kloadgen.sampler.KafkaSampler" throws this error:
net.coru.kloadgen.exception.KLoadGenException: Unsupported Serializer

Can you please help??
Thanks,
Raheel

i can access the schema registry URL in browser and its behind a proxy , need to know how do i use that in schema registry config element

Hi all, I use the plugin to send Avro-Kafka records. With the schema registry it works fine, great plugin.

Unfortunately loading Avro-schemes via file does not work for me. I use the Example-Test-Plan.jmx and replaced the Kafka Load Generator Config with a Kafka File Load Generator Config element.

This leads to the following error messages:

Error 1:

2020-06-08 14:17:26,190 ERROR n.c.k.l.i.AvroLoadGenerator: Please make sure that properties data type and expression function return type are compatible with each other
io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Subject not found.; error code: 40401
	at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:230) ~[kloadgen-1.5.0.jar:?]
	at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:256) ~[kloadgen-1.5.0.jar:?]
	at io.confluent.kafka.schemaregistry.client.rest.RestService.getLatestVersion(RestService.java:515) ~[kloadgen-1.5.0.jar:?]
	at io.confluent.kafka.schemaregistry.client.rest.RestService.getLatestVersion(RestService.java:507) ~[kloadgen-1.5.0.jar:?]
	at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getLatestSchemaMetadata(CachedSchemaRegistryClient.java:275) ~[kloadgen-1.5.0.jar:?]
	at net.coru.kloadgen.input.AvroSchemaProcessor.getSchemaBySubject(AvroSchemaProcessor.java:266) ~[kloadgen-1.5.0.jar:?]
	at net.coru.kloadgen.input.AvroSchemaProcessor.<init>(AvroSchemaProcessor.java:86) ~[kloadgen-1.5.0.jar:?]
	at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.<init>(AvroLoadGenerator.java:18) [kloadgen-1.5.0.jar:?]
	at net.coru.kloadgen.config.fileserialized.FileSerializedConfigElement.iterationStart(FileSerializedConfigElement.java:39) [kloadgen-1.5.0.jar:?]
	at org.apache.jmeter.control.GenericController.fireIterationStart(GenericController.java:399) [ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.control.GenericController.fireIterEvents(GenericController.java:391) [ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.control.GenericController.next(GenericController.java:160) [ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.control.LoopController.next(LoopController.java:135) [ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.threads.AbstractThreadGroup.next(AbstractThreadGroup.java:92) [ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:255) [ApacheJMeter_core.jar:5.2.1]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]

Error 2:

2020-06-08 14:17:26,191 ERROR n.c.k.c.f.FileSerializedConfigElement: Failed to create AvroLoadGenerator instance
net.coru.kloadgen.exception.KLoadGenException: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Subject not found.; error code: 40401
	at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.<init>(AvroLoadGenerator.java:21) ~[kloadgen-1.5.0.jar:?]
	at net.coru.kloadgen.config.fileserialized.FileSerializedConfigElement.iterationStart(FileSerializedConfigElement.java:39) [kloadgen-1.5.0.jar:?]
	at org.apache.jmeter.control.GenericController.fireIterationStart(GenericController.java:399) [ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.control.GenericController.fireIterEvents(GenericController.java:391) [ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.control.GenericController.next(GenericController.java:160) [ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.control.LoopController.next(LoopController.java:135) [ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.threads.AbstractThreadGroup.next(AbstractThreadGroup.java:92) [ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:255) [ApacheJMeter_core.jar:5.2.1]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Subject not found.; error code: 40401
	at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:230) ~[kloadgen-1.5.0.jar:?]
	at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:256) ~[kloadgen-1.5.0.jar:?]
	at io.confluent.kafka.schemaregistry.client.rest.RestService.getLatestVersion(RestService.java:515) ~[kloadgen-1.5.0.jar:?]
	at io.confluent.kafka.schemaregistry.client.rest.RestService.getLatestVersion(RestService.java:507) ~[kloadgen-1.5.0.jar:?]
	at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getLatestSchemaMetadata(CachedSchemaRegistryClient.java:275) ~[kloadgen-1.5.0.jar:?]
	at net.coru.kloadgen.input.AvroSchemaProcessor.getSchemaBySubject(AvroSchemaProcessor.java:266) ~[kloadgen-1.5.0.jar:?]
	at net.coru.kloadgen.input.AvroSchemaProcessor.<init>(AvroSchemaProcessor.java:86) ~[kloadgen-1.5.0.jar:?]
	at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.<init>(AvroLoadGenerator.java:18) ~[kloadgen-1.5.0.jar:?]

Error 3:

2020-06-08 14:17:26,202 ERROR n.c.k.s.ConfluentKafkaSampler: Failed to Generate message

I have tried different schemes. For example the Avro scheme as JSON file:

{
    "type" : "record",
    "namespace" : "test",
    "name" : "test",
    "fields" : [
       { "name" : "value1" , "type" : "string" },
       { "name" : "value2" , "type" : "int" }
    ]
 }

JMeter configuration (based on the Example-Test.json file):

Kafka File Load Generator Config:

Schema-Registry Config Element:

Kafka Producer Properties;

I suspect that this is a configuration problem. Does anyone see the problem?

Hi team, thank you for this plugin it helped a lot.

I am facing an issue while passing the below field value in the payload.

Field Type: String
Filed Value: [{"client_code":"ABC","market_codes":["popfa"],"permissions":[{"app_code":"TEAA","resource_codes":[{"code":"","action":""}]}]}]

this string gets break from comma and I only able to get half string in the output

*[{"client_code":"ABC"*

can you please help me with the same how I can pass comma in the string?

I'm running JMeter 5.3 on Mac, and added your plugin. I would like to use JMeter to load test Kafka with Schema Registry, where the messages are AVRO encoded.

I have configured Schema Registry Config Element as attached. When I clicked on Test Registry, it shows successful connection to schema registry, with number of subject in Registry: 1 .

Then I move on to Kafka Load Generator Config, and click on Load Subjects, but it failed with :

Failed retrieve schema properties: Subject not found, error code 40401

Under Kafka Load Generator Config, is it using the Schema Registry URL from Schema Registry Config Element for the connection?

Any step I'm missing? Appreciate your help.

I got this issue

net.coru.kloadgen.config.schemaregistry.SchemaRegistryConfigElement#setSchemaRegistryProperties(LinkedList [PropertyMapping(propertyName=schema.registry.auth.enabled, propertyValue=NO), PropertyMapping(propertyName=schema.registry.auth.method, propertyValue=BASIC), PropertyMapping(propertyName=schema.registry.username, propertyValue=<username>), PropertyMapping(propertyName=schema.registry.password, propertyValue=<password>), PropertyMapping(propertyName=schema.registry.bearer, propertyValue=<bearer>)]

and this is the detail.

java.lang.Error: This should never happen. Tried to invoke:
net.coru.kloadgen.config.schemaregistry.SchemaRegistryConfigElement#setSchemaRegistryProperties(LinkedList [PropertyMapping(propertyName=schema.registry.auth.enabled, propertyValue=NO), PropertyMapping(propertyName=schema.registry.auth.method, propertyValue=BASIC), PropertyMapping(propertyName=schema.registry.username, propertyValue=<username>), PropertyMapping(propertyName=schema.registry.password, propertyValue=<password>), PropertyMapping(propertyName=schema.registry.bearer, propertyValue=<bearer>)] )
	at org.apache.jmeter.testbeans.TestBeanHelper.invokeOrBailOut(TestBeanHelper.java:169) ~[ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.testbeans.TestBeanHelper.prepare(TestBeanHelper.java:97) ~[ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.threads.TestCompiler.trackIterationListeners(TestCompiler.java:183) ~[ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.threads.TestCompiler.subtractNode(TestCompiler.java:137) ~[ApacheJMeter_core.jar:5.2.1]
	at org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:998) ~[jorphan.jar:5.2.1]
	at org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:995) ~[jorphan.jar:5.2.1]
	at org.apache.jorphan.collections.HashTree.traverse(HashTree.java:977) ~[jorphan.jar:5.2.1]
	at org.apache.jmeter.threads.JMeterThread.initRun(JMeterThread.java:703) ~[ApacheJMeter_core.jar:5.2.1]
	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:253) ~[ApacheJMeter_core.jar:5.2.1]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_252]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_252]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_252]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
	at org.apache.jmeter.testbeans.TestBeanHelper.invokeOrBailOut(TestBeanHelper.java:167) ~[ApacheJMeter_core.jar:5.2.1]
	... 9 more
Caused by: java.lang.NullPointerException
	at java.util.Hashtable.put(Hashtable.java:460) ~[?:1.8.0_252]
	at java.util.Properties.setProperty(Properties.java:166) ~[?:1.8.0_252]
	at net.coru.kloadgen.config.schemaregistry.SchemaRegistryConfigElement.serializeProperties(SchemaRegistryConfigElement.java:58) ~[kloadgen-1.2.0.jar:?]
	at net.coru.kloadgen.config.schemaregistry.SchemaRegistryConfigElement.setSchemaRegistryProperties(SchemaRegistryConfigElement.java:45) ~[kloadgen-1.2.0.jar:?]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_252]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_252]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_252]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
	at org.apache.jmeter.testbeans.TestBeanHelper.invokeOrBailOut(TestBeanHelper.java:167) ~[ApacheJMeter_core.jar:5.2.1]
	... 9 more

From Jmeter source

/**
     * Utility method that invokes a method and does the error handling around
     * the invocation.
     *
     * @param invokee
     *            the object on which the method should be invoked
     * @param method
     *            the method which should be invoked
     * @param params
     *            the parameters for the method
     * @return the result of the method invocation.
     */
    private static Object invokeOrBailOut(Object invokee, Method method, Object[] params) {
        try {
            return method.invoke(invokee, params);
        } catch (IllegalArgumentException | IllegalAccessException | InvocationTargetException e) {
            throw new Error(createMessage(invokee, method, params), e);
        }
    }

I guess the method invoke one of those three exceptions which is illegalArgumentException, illegalAccessException and InvocationTargetException.

What am I supposed to do with that ?

Hi Team,

When using the Kafka Load Generator config, I have an integer field Customer Id for which I haven't set a definitive list and would like random number to be generated.

However, I see same number getting duplicated 3 out of 6 times. Is there a way to ensure the duplicates aren't generated or atleast reduce the frequency? I checked the documentation provided but couldn't find any solution in this regard.

Note: I am setting up load generator for testing a kafka publisher. So raising questions as and when I encounter. Appreciate your quick response/clarification so far.

Field type having Long or Int with field length 11 or more is generating only 10 digits long value

Hi,

thanks for building this wonderful plugin. I have successfully built this plugin with no error and able to use it in my jmeter. I am using java 11 to build the plugin.

But I am missing a config element which kafka file load generator config in my config element list. I tried different build and java configurations but still its missing. I dont know what i am doing wrong as it gives no error. I downloaded the latest kloadgen release.

Please help me to build it correctly so that i can use the Kafka file load generator config element.

Hello!
trying to send a message to kafka - key and value. (see message_for_Kafka.json)
but in kafka consumer are empty
posting .jmx, avro schema and sample message for Kafka
Jmeter.zip

and I`m using Classname .GenericKafkaSampler
if you connect ConfluentKafkaSampler then you get an error .NullPointerException: null

The guava version used in project has a solved vulnerability in the version 24.1.1 it should be a good practice to update the library to solve this vulnerability.

https://github.com/corunet/kloadgen/network/alert/pom.xml/com.google.guava:guava/open

When opening the Example-Test-Plan.jmx file, I am getting the following error:

io/confluent/kafka/schemaregistry/client/rest/exceptions/RestClientException

I performed the settings mentioned, clean and install, and added the .jar in the lib folder.

Can anyone help?

I have a weird schema with an array of objects
Something like this:
{
"uuid": "d7fa5207-3ba5-4517-8d5a-256066d26e88",
"type": "whatever",
"data": [{
"attribute": "name",
"valueList": ["John"],
"operator": "match"
}]
}

When i use the plugin as in the below screenshot:

a) I am unable to limit the number of values in the array even if i set the field-length = 1. The resulting message is
{
"uuid": "365b57ad-d9e6-44a9-af57-1112b6b1a822",
"type": "whatever",
"data": [{
"attribute": "name",
"valueList": ["John", "John", "John"],
"operator": "match"
}]
}
"John" gets added at least thrice (random sizes in the list). Is there a way to limit the number of entries in the array?

b) is there a way to add multiple objects into the above list with specific data. I understand keeping data[2] would give me 2 items in the list. But i want a way to be able to control the data in the "attribute" and "valueList" fields. For example:
if I put values as "Name", "LastName" for attribute and "John", "Smith" in valueList, is there a way to ensure that the resulting message looks like this:

{
"uuid": "365b57ad-d9e6-44a9-af57-1112b6b1a822",
"type": "whatever",
"data": [{
"attribute": "name",
"valueList": ["John"],
"operator": "match"
},
{
"attribute": "last_name",
"valueList": ["Smith"],
"operator": "match"
}
]
}

instead of everything being randomly picked and mixed and matched in the request ?

Can you please help?

Hi Team,
I have requirement to test kafka with avro schema message for which I came across your plugin. I tried to create the jar with the steps mentiond in the Build Project section. But I get dependency issues. Below is the output. I tried with eclipse and with maven both but every time build fails.
I have tried with master branch and with release-3.0.0 branch. but every time it gives dependency issues.
Is it possible to provide the JAR directly or will be possible to provide more detailed steps to use this with versions of tool and all. I have added manually some 20-30 but still it gives more... I am using home computer so to clarify I am not running this behind any proxy.

Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/39/commons-parent-39.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/50/commons-parent-50.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/48/commons-parent-48.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/github/luben/zstd-jni/1.4.4-7/zstd-jni-1.4.4-7.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/lz4/lz4-java/1.7.1/lz4-java-1.7.1.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/xerial/snappy/snappy-java/1.1.7.3/snappy-java-1.1.7.3.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.10.2/jackson-databind-2.10.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/module/jackson-module-scala_2.13/2.10.2/jackson-module-scala_2.13-2.10.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-csv/2.10.2/jackson-dataformat-csv-2.10.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.10.2/jackson-datatype-jdk8-2.10.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/net/sf/jopt-simple/jopt-simple/5.0.4/jopt-simple-5.0.4.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/yammer/metrics/metrics-core/2.2.0/metrics-core-2.2.0.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/scala-lang/modules/scala-collection-compat_2.13/2.1.6/scala-collection-compat_2.13-2.1.6.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/scala-lang/modules/scala-java8-compat_2.13/0.9.1/scala-java8-compat_2.13-0.9.1.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/scala-lang/scala-library/2.13.2/scala-library-2.13.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/scala-lang/scala-reflect/2.13.2/scala-reflect-2.13.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/typesafe/scala-logging/scala-logging_2.13/3.9.2/scala-logging_2.13-3.9.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/zookeeper/zookeeper/3.5.8/zookeeper-3.5.8.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/30.0-jre/guava-parent-30.0-jre.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/avro/avro-parent/1.10.0/avro-parent-1.10.0.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-base/2.10.5/jackson-base-2.10.5.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/51/commons-parent-51.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/ee4j/project/1.0.5/project-1.0.5.pom
Downloading from confluent: https://packages.confluent.io/maven/org/eclipse/ee4j/project/1.0.5/project-1.0.5.pom
Downloading from jitpack.io: https://jitpack.io/org/eclipse/ee4j/project/1.0.5/project-1.0.5.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.10/jackson-parent-2.10.pom
Downloading from confluent: https://packages.confluent.io/maven/com/fasterxml/jackson/jackson-parent/2.10/jackson-parent-2.10.pom
Downloading from jitpack.io: https://jitpack.io/com/fasterxml/jackson/jackson-parent/2.10/jackson-parent-2.10.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/2.0.0-alpha1/slf4j-parent-2.0.0-alpha1.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.5.2/junit-platform-commons-1.5.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/assertj/assertj-parent-pom/2.2.8/assertj-parent-pom-2.2.8.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.3/jackson-core-2.6.3.pom
Downloading from central: https://repo.maven.apache.org/maven2/net/minidev/json-smart/2.3/json-smart-2.3.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.3/jackson-databind-2.6.3.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/assertj/assertj-core/3.8.0/assertj-core-3.8.0.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/mockito/mockito-core/3.6.0/mockito-core-3.6.0.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/mockito/mockito-junit-jupiter/3.6.0/mockito-junit-jupiter-3.6.0.pom
Downloading from central: https://repo.maven.apache.org/maven2/commons-beanutils/commons-beanutils/1.9.4/commons-beanutils-1.9.4.pom
Downloading from central: https://repo.maven.apache.org/maven2/com/github/tomakehurst/wiremock/2.27.2/wiremock-2.27.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/ru/lanwen/wiremock/wiremock-junit5/1.3.1/wiremock-junit5-1.3.1.pom
Downloading from central: https://repo.maven.apache.org/maven2/io/rest-assured/rest-assured/4.3.2/rest-assured-4.3.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/javax/xml/bind/jaxb-api/2.3.1/jaxb-api-2.3.1.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.5.2/junit-jupiter-params-5.5.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.5.2/junit-jupiter-engine-5.5.2.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/jmeter/ApacheJMeter_java/5.3/ApacheJMeter_java-5.3.pom
Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-jdk14/2.0.0-alpha1/slf4j-jdk14-2.0.0-alpha1.pom

It will be great help you there can be any assistance on this.

Add avro serializer with binary encoder, remove code duplication. #72

In order to clean up and make sampler easy, lets move the serialize and naming strategy chose to where schema is configured. So will be then another 2 properties to set up in this screen. That will reduce the number of samplers to one.

Hi,

I'm trying to disable it by setting empty:

By after that, when I press "Start" in JMeter, it just restores to default value:

I'm using plugin from "master" branch and JMeter 5.4 on Mac.

Any ideas what could the problem?

When I click on the option "Test Registry" I am getting the error below, has anyone experienced this problem, any tips to help me?

Failed retrieve schema properties : sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Thanks!

If the schema Registry has security activated, we need to be able to pass username/password or bearer token to the client.

Currently we are required to send 20+ key-value pairs (preset values and not random values) as part of the String-Map field and what we have observed is that the plugin randomly chooses a few key-value pairs and sends it to Kafka Topic. Is it possible to change the behavior of this to send the preset-value as it is, if the Field-Length value is set to "0"?

Hi
I'm having an issue with Kafka Headers Configuration.
It doesn't send any headers in the producer.
I tried sending custom string as well as headers from example test plan.

pom.zip

Getting below error even though I don't have "log4j-slf4j-impl-2.11.1.jar" in JMeter/lib folder. Attached the pom.xml that I used to build the kloadgen-plugin-2.1.3.jar.

C:\gitRepos\jmeter-5.1.1\bin>jmeter.bat
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/gitRepos/jmeter-5.1.1/lib/log4j-slf4j-impl-2.11.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/gitRepos/jmeter-5.1.1/lib/ext/kloadgen-plugin-2.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
java.lang.NoClassDefFoundError: Could not initialize class org.apache.logging.log4j.util.PropertiesUtil
at org.apache.logging.log4j.status.StatusLogger.(StatusLogger.java:78)
at org.apache.logging.log4j.LogManager.(LogManager.java:60)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:46)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.apache.jmeter.JMeter.(JMeter.java:124)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.jmeter.NewDriver.main(NewDriver.java:251)

If an Avro uses an Union with only one element, a NullPointerException is raised.

Hi,
I am trying to pass string-array field in the schema and getting below error can you please help me with the same.

Hi,

Is there a way to send partitionkey along with topic name and message to Kafka through this plugin?

Hi!

In readme we can find

Above properties are added by default in sampler as those are more significant in terms of performance in most of the cases. But you can add other non listed kafka properties with prefix "_".

For example to enable SSL properties you can add below properties

_ssl.key.password
_ssl.keystore.location

I am trying to add
_ssl.enabled.protocols = [TLSv1.2]

and some other props to Kafka Producer Properties below default props. But it seems that it's not working. After each start jmeter deletes this custom props, and I see only defaults in log output.

Is this feature work properly, or I do something wrong?

Apache jmeter 5.2.1
KLoadGen 2.1.2

HI

there is a scenarios to use unique message key value for each iteration of the loop . but even if i use a time function for example
${__threadNum}_${__time(ddMMyyyyHHMMSS)} like this it always takes the first value as message key for the next iteration also .

but i need this to change , is there a way to achieve this ?

In Kafka Load Generator config option in JMeter, when we hard code one value for an array field, the load generator is duplicating the value multiple times. Is there a way to restrict the number random generator to generate values only once in this case?

Example: There is an array as address[] and 2 fields under it as address[].addressId and address[].city and I have set addressId as 1 and city as abc. Field Length for both of them is set to 0. However when I run the script, it shows address multiple times as below. Due to this the functionality is failing as duplicate addressId is not allowed in my application.

"address":[{
"addressId" :1,
"city":abc},
{"addressId":1,
"city" :abc},
{"addressId" :1,
"city":abc}]

I need to add a consumer so I can receive messages and deserialize them base on the AVRO/Json-Schema configuration.
Message received will be populate into JMeter Context for further evaluation.

Can you post the avsc file for the example? Thanks.

After last update, a wrong behavior has been detected introducing values in the Field Value List, adding extra [ ]when a value is introduced.

In RandomTool there exists one method which is used only for schema Avro, the objective is to move this method to another place. Also the second parameter of castValue method should be a string.

In the Avro Field and Expression Mappings - Kafka Load Generator Config, for the String-Map field whatever value that we give it gets duplicated or split and sent. We are not able to send the data properly.
For eg, if we have field name as "ExcelValues" and field type as "string-map" and if we give this in the field value list [{"area":"LASK","area":"JSKL"}] then it goes to kafka as {"area":"LASK"}:{"area":"LASK"} or some times only one key-pair values.
Is it something that, we are not properly formatting and supplying the values, or is it an issue with string-map format? Please let us know.

This is the only plugin that I fond that declare ability to send kafka message with a header. But unfortunately it doesn't work for me.

Details:

1. There is an issue in Jmeter 5.1.1 while sending a kafka message. I used Example-Test-Plan.jmx from the git folder.
   Log from jmeter:

020-03-31 20:21:19,802 ERROR o.a.j.t.JMeterThread: Error while processing sampler: 'Kafka Producer Properties'.
java.lang.NullPointerException: null
at java.util.Hashtable.put(Hashtable.java:460) ~[?:1.8.0_241]
at net.coru.kloadgen.sampler.ConfluentKafkaSampler.setupTest(ConfluentKafkaSampler.java:136) ~[kloadgen-1.0.jar:?]
at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:196) ~[ApacheJMeter_java.jar:5.1.1 r1855137]
at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:622) ~[ApacheJMeter_core.jar:5.1.1 r1855137].....

2. Issue during " mvn clean install" to create a jar file.
   I had to add the lines below to pom.xml file, because confluent.io files are not available by default path during compiling.

 <repositories>
    <repository>
     <id>confluent</id>
     <url>http://packages.confluent.io/maven/</url>
   </repository>
  </repositories>

The issue with building a jar file is still exists for me on Windows if I use the provided Setting.xml file.
What I done:
I've put the setting file that you sent me to my .m2\ folder; there is no effect, it looks that it doesn't work for windows.
I've put it to maven home directory, the logs while building the project were changed, but in the end I have the same issue.

My schema has an array of map. And I believe that feature does not exist. Am I correct on this? Or Am I missing something?