When run the exe with ./mqtt-stresser-linux-amd64-static -broker tcp://45.76.163.6:1883 -num-clients 32219 -num-messages 100 -publisher-qos 2 -subscriber-qos 2 at the platform of Ubuntu 14.04 x64.

(45.76.163.6 is my centos6 1G 1CPU vps in vultr.com in Singapore, if you want, you can use it to test, and it's broker's web service port is 18083)

Here is the output,

100 worker started - waiting 500ms
200 worker started - waiting 500ms
300 worker started - waiting 500ms
400 worker started - waiting 500ms
500 worker started - waiting 500ms
600 worker started - waiting 500ms
700 worker started - waiting 500ms
800 worker started - waiting 500ms
900 worker started - waiting 500ms
1000 worker started - waiting 500ms
1100 worker started - waiting 500ms
1200 worker started - waiting 500ms
1300 worker started - waiting 500ms
1400 worker started - waiting 500ms
1500 worker started - waiting 500ms
1600 worker started - waiting 500ms
1700 worker started - waiting 500ms
1800 worker started - waiting 500ms
1900 worker started - waiting 500ms
2000 worker started - waiting 500ms
2100 worker started - waiting 500ms
2200 worker started - waiting 500ms
2300 worker started - waiting 500ms
2400 worker started - waiting 500ms
2500 worker started - waiting 500ms
2600 worker started - waiting 500ms
2700 worker started - waiting 500ms
2800 worker started - waiting 500ms
2900 worker started - waiting 500ms
3000 worker started - waiting 500ms
3100 worker started - waiting 500ms
3200 worker started - waiting 500ms
3300 worker started - waiting 500ms
3400 worker started - waiting 500ms
3500 worker started - waiting 500ms
3600 worker started - waiting 500ms
3700 worker started - waiting 500ms
3800 worker started - waiting 500ms
3900 worker started - waiting 500ms
4000 worker started - waiting 500ms
4100 worker started - waiting 500ms
4200 worker started - waiting 500ms
4300 worker started - waiting 500ms
4400 worker started - waiting 500ms
4500 worker started - waiting 500ms
4600 worker started - waiting 500ms
4700 worker started - waiting 500ms
4800 worker started - waiting 500ms
4900 worker started - waiting 500ms
5000 worker started - waiting 500ms
5100 worker started - waiting 500ms
5200 worker started - waiting 500ms
5300 worker started - waiting 500ms
5400 worker started - waiting 500ms
5500 worker started - waiting 500ms
5600 worker started - waiting 500ms
5700 worker started - waiting 500ms
5800 worker started - waiting 500ms
5900 worker started - waiting 500ms
6000 worker started - waiting 500ms
6100 worker started - waiting 500ms
panic: close of nil channel

goroutine 31121 [running]:
github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho%2emqtt%2egolang.(*client).closeStop(0xc01de78380)
	/Users/ab/Documents/dev/GOPATH/src/github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho.mqtt.golang/client.go:482 +0x12f
github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho%2emqtt%2egolang.(*client).disconnect(0xc01de78380)
	/Users/ab/Documents/dev/GOPATH/src/github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho.mqtt.golang/client.go:495 +0x2f
github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho%2emqtt%2egolang.(*client).Disconnect(0xc01de78380, 0x5)
	/Users/ab/Documents/dev/GOPATH/src/github.com/inovex/mqtt-stresser/vendor/github.com/eclipse/paho.mqtt.golang/client.go:439 +0x1f9
main.(*Worker).Run(0xc01dd71140, 0x719f40, 0xc000054420)
	/Users/ab/Documents/dev/GOPATH/src/github.com/inovex/mqtt-stresser/worker.go:127 +0xf23
created by main.main
	/Users/ab/Documents/dev/GOPATH/src/github.com/inovex/mqtt-stresser/main.go:174 +0x592

./mqtt-stresser -broker tcp://broker.mqttdashboard.com:1883 -num-clients 10 -num-messages 150 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s
open : no such file or directory

time.Sleep(rampUpDelay).    // <-- here is first 
select {
case <-time.NewTimer(rampUpDelay).C:       // <-- here is second
case s := <-signalChan:
fmt.Printf("Got signal %s. Cancel test.\n", s.String())
cancelFunc()
stopStartLoop = true
}

why need two ?

hello:

I found your project very useful, but I had to add support for TLS authentication to make it work in our brokers.
I created a PR in case you find that feature of any value. Let me know if you would like me to change something.

thanks for sharing.

Hello,
i got an unexpected behavior while testing our broker: Test timeout. Wait 5s to allow disconnection of clients.

The Debug Log:
DEBUG: 13:03:17.305348 worker.go:190: [5] starting control loop test/test/76027e37ce19/worker5-1693134756
DEBUG: 13:03:17.305464 worker.go:207: [6] all messages published
DEBUG: 13:03:17.305663 worker.go:207: [0] all messages published
DEBUG: 13:03:17.305671 worker.go:207: [5] all messages published
DEBUG: 13:03:17.306942 worker.go:190: [2] starting control loop test/test/76027e37ce19/worker2-1550364868
DEBUG: 13:03:17.307255 worker.go:207: [2] all messages published
DEBUG: 13:03:17.307308 worker.go:179: [9] connecting publisher
DEBUG: 13:03:17.307320 worker.go:190: [8] starting control loop test/test/76027e37ce19/worker8-1457327270
DEBUG: 13:03:17.308286 worker.go:207: [8] all messages published
DEBUG: 13:03:17.313748 worker.go:190: [7] starting control loop test/test/76027e37ce19/worker7-112932577
DEBUG: 13:03:17.314074 worker.go:207: [7] all messages published
DEBUG: 13:03:17.327766 worker.go:190: [9] starting control loop test/test/76027e37ce19/worker9-1605856701
DEBUG: 13:03:17.327976 worker.go:207: [9] all messages published
Test timeout. Wait 5s to allow disconnection of clients.
DEBUG: 13:06:37.242693 worker.go:240: [5] received abort signal due to test timeout
DEBUG: 13:06:37.242730 worker.go:260: [5] worker finished

no results are displayed.

Is there any opportunity to send file as payload.

How to start the stesser with user name and password for mqtt authentication?

After a certain time the stresser stops to generate load and does not receive any messages.

My receiving client deserializes the paylod from cbor format. Is there a way to send an array of bytes as payload?

using the existing Dockerfile as base

i have been setting the -constant-payload flag, but I seem to be getting the default messages each time. this seems to happen with the topic set by the flag -topic-base-path(see the logs below). I am using rust's paho_mqtt for my client and mosquitto as the broker.

Here is my command:
docker run --rm inovex/mqtt-stresser -broker tcp://xxxxxxx:1883 -num-clients 1 -num-messages 1 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s -skip-tls-verification true -constant-payload hello -topic-base-path 1231asss/1212asss/event

I am using rust to deserialize the payload using the serde crate on my client side:

    let report = std::str::from_utf8(payload_str);
    info!("THIS IS A TEST: {:?}", report);

and this is the result from my logs:

2021-09-09T12:57:32.240024471-04:00 INFO mqtt_bin::mqttmod::mqtt_bin - do async_wait_message topic received: internal/mqtt-stresser/b262e40f3baa/worker0-722306144
 

2021-09-09T12:57:32.242970317-04:00 INFO mqtt_bin::mqttmod::mqtt_event_process::event_process - event_porcess_func-payload: [116, 104, 105, 115, 32, 105, 115, 32, 109, 115, 103, 32, 35, 48, 33] 

2021-09-09T12:57:32.243107067-04:00 INFO mqtt_bin::mqttmod::mqtt_event_process::event_process - THIS IS A TEST: Ok("this is msg #0!") 

I want to simulate a situation where we have many clients, sending a message once a minute.

I'd suggest a new parameter that accepts a wait time in seconds, like wait-period and then in the worker, add a thread sleep (or similar) to wait for the required period of time.

Awesome tool by the way, thanks!

I want to use the MQTT-Stresser to test a MQTT broker which only supports secure links and self signed certificates.
I am using the docker but am not able to connect to the broker. Since it is a self signed certificate I added -skip-tls-verification Do I still have to provide a CA file wit this option.
I tried too with the HiveMQ broker from the example with gave the exact same result I running on Ubuntu

sudo docker run --rm inovex/mqtt-stresser -broker tcp://broker.mqttdashboard.com:8883 -skip-tls-verification -num-clients 2 -num-messages 2 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s

Error message:
failed to unsubscribe: Not Connected
failed to unsubscribe: Not Connected

btw if I do add a CA certificate with the -ca_file option I get, independent of which certificate file I use

panic: tls: failed to find any PEM data in certificate input

goroutine 12 [running]:
main.(*Worker).Run(0xc000066580, 0x77ee20, 0xc000046420)
	/go/src/github.com/inovex/mqtt-stresser/worker.go:131 +0x1b6d
created by main.main
	/go/src/github.com/inovex/mqtt-stresser/main.go:271 +0x910

It would be of value to be able to set the QoS level for subscribers and publishers. The retain value for publishers is also valuable when stress testing different scenarios.

I suggest:
-publisher-qos
-subscriber-qos
-retain

Any number of clients with -num-messages exceeding 220 will result in a crash.
ERROR: main.go:162: {0 TimeoutExceeded 3.4400055s 0s 220 300 true }

When exceeding 220 the script freezes until the global timeout hits.

I am facing error running sample emqx free server

Below is the command
docker run inovex/mqtt-stresser -broker mqtt://broker.emqx.io:1883 -num-clients 100 -num-messages 10 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s -skip-tls-verification true

Error i am getting is
Efailed to unsubscribe: Not Connected
Efailed to unsubscribe: Not Connected

Every-time I try to stress-test test.mosquitto.org, I am getting the following output/error(but works fine with other publicly hosted brokers). Please help me understand this and guide how can I get the full result.
thanks in advance.

./mqtt-stresser -broker tcp://test.mosquitto.org:1883 -nm-clients 100 -num-messages 10 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s
10 worker started - waiting 1s
20 worker started - waiting 1s
30 worker started - waiting 1s
40 worker started - waiting 1s
50 worker started - waiting 1s
60 worker started - waiting 1s
70 worker started - waiting 1s
80 worker started - waiting 1s
90 worker started - waiting 1s
100 worker started
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEpanic: close of nil channel

goroutine 23 [running]:
panic(0x6f54c0, 0xc820336670)
/usr/lib/go-1.6/src/runtime/panic.go:481 +0x3e6
github.com/eclipse/paho%2emqtt%2egolang.(*client).closeStop(0xc82015af00)
/home/install/go/src/github.com/eclipse/paho.mqtt.golang/client.go:482 +0x1b2
github.com/eclipse/paho%2emqtt%2egolang.(*client).disconnect(0xc82015af00)
/home/install/go/src/github.com/eclipse/paho.mqtt.golang/client.go:495 +0x2b
github.com/eclipse/paho%2emqtt%2egolang.(*client).Disconnect(0xc82015af00, 0x5)
/home/install/go/src/github.com/eclipse/paho.mqtt.golang/client.go:439 +0x314
main.(*Worker).Run.func2(0xc820092500, 0x7fc524d3d910, 0xc82015af00, 0xc820012900, 0x3f)
/home/install/Downloads/Brokers/benchmark/mqtt-stresser-master/worker.go:70 +0x304
main.(*Worker).Run(0xc820092500)
/home/install/Downloads/Brokers/benchmark/mqtt-stresser-master/worker.go:163 +0x23fe
created by main.main
/home/install/Downloads/Brokers/benchmark/mqtt-stresser-master/main.go:137 +0x96a

I was testing my mosquitto broker with the following command

mqtt-stresser -broker tcp://192.168.1.221:1883 \
   -num-clients 10000 \
   -num-messages 10 \
   -rampup-delay 200ms \
   -rampup-size 1000 \
   -global-timeout 180s \
   -timeout 20s

However the test stopped in the middle with the Not Connected error. Does this simply mean my broker can not handle 10K concurrent connections?

1000 worker started - waiting 200ms
2000 worker started - waiting 200ms
3000 worker started - waiting 200ms
4000 worker started - waiting 200ms
5000 worker started - waiting 200ms
6000 worker started - waiting 200ms
7000 worker started - waiting 200ms
8000 worker started - waiting 200ms
9000 worker started - waiting 200ms
10000 worker started
.........................(many-dots-omitted-here)...........................Not Connected

Hi guys,

I've been trying to use the stresser in the form of docker image on Windows 7 Ultimate using docker Toolbox. Without tls certificate it works great, but when i tried to pass a certificate through the "docker run" command, and even when i tried to bind mount, it still says the cert "doesn't exist" at the path i mounted it to. Here's the command:

$ docker run -v /e/Docker\ Toolbox/:/home inovex/mqtt-stresser -broker tcp://192.168.0.10:8883 -cert /home/stress_client.crt -num-clients 10 -num-messages 150 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20

I know i haven't specified CA cert yet, but i suspect to run into the same problem.
Could you let me know what is the intended way to pass in certificate when using the docker image, or if i'm doing anything wrong with the command above

Thank you for your time

To ease systematic benchmark, we should export the benchmark results in a machine readable format. This allows import in evaluation tool chains like jupyter notebooks, R or Excel.

This tool would be amazing if it could be used on ARM. Many mqtt scenarios use devices with the ARM architecture.

I would just suggest you to list pre-requisities for the build and using mqtt-stresser. You can figure out you are using go but would be great to put a reference to it etc.

root@ip-172-31-22-25:~/mqtttester/mqtt-stresser# make
GOOS=windows GOARCH=amd64 go build -o build/mqtt-stresser.exe
/bin/sh: 1: go: not found
Makefile:46: recipe for target 'build/mqtt-stresser-windows-amd64.zip' failed
make: *** [build/mqtt-stresser-windows-amd64.zip] Error 127

The problem occurs while stress testing some brokers like Bevywise MQTT Route and HiveMq as they support only 25 clients to be connected in their evaluation versions.

MQTT Route keeps a client connection count which works based on the Clean Session. Hence what happens while connecting to the Broker Server(using MQTT Stresser) the second connection request from the same client​ ​(MQTT Stresser ) to publish/subscribe some msgs is being treated as a New client and client counter gets increased by 1 and thus exceeds 25.

If we could initiate a client connection to the MQTTRoute setting the Clean session as false ( 0 ), It will treats the second connection request with the same unique Client ID as a re-connection and will not increase the connected client count and I can conduct some fare stress test keeping all the testing parameters constant for all the brokers that I put for evaluation of performace.

  < 147 msg/sec  75%
  < 155 msg/sec  93%
  < 164 msg/sec  96%
  < 181 msg/sec  97%
  < 190 msg/sec  98%
  < 215 msg/sec  99%
  < 232 msg/sec  100%

msg/sec only, but what's the the length of each msg?

Hello,
For clarification purposes, would it be possible to add some description covering which layers and protocols this tool explores? For reliability tests, this would be pretty useful! Thanks!

I`m with problem to execute mqtt-stresser. I already updated the modules

./build/mqtt-stresser-linux-amd64 -broker tcp://localhost:1883 -num-clients 100 -num-messages 150 -rampup-delay 1s -rampup-size 10 -global-timeout 180s -timeout 20s
open : no such file or directory

Hello:
this is not a big deal, but I'm finding something a little confusing in the report output. When, for example, I run the program with -num-clients 10 -num-messages 10

The report shows Messages / Client: 100 , that is the total number of messages (clients * messages)

My expectation is that this value would be the same as --num-messages to reflect the configuration of that run.

Sample output:

# Configuration
Concurrent Clients: 10
Messages / Client:  100

# Results
Published Messages: 100 (100%)
Received Messages:  100 (100%)
Completed:          10 (100%)
Errors:             0 (0%)

If this is something that should be fixed, I'll be happy to make the PR with the adjustment (it should be minimal), but if I'm understanding this wrong, please disregard.

All the best

Hello!
I'm stress testing a broker and and I noticed a lot bigger receiving througput with QoS 1 and 2. Otherwise same parameters.
Broker is on AWS.
No errors, all 100% published/received.
Why is that?

./mqtt-stresser-linux-amd64 \
-broker tcp://localhost:1883 \
-num-clients 1000 \
-num-messages 10 \
-rampup-delay 5s \
-global-timeout 180s \
-timeout 20s \
-publisher-qos 2 \
-constant-payload $(cat payload40kb.txt) \
> mqtt_stresser_40kb_10_qos2.txt
/

QoS 0

# Receiving Througput
Fastest: 126 msg/sec
Slowest: 23 msg/sec
Median: 77 msg/sec

QoS 1

# Receiving Througput
Fastest: 49685 msg/sec
Slowest: 71 msg/sec
Median: 550 msg/sec

QoS 2

# Receiving Througput
Fastest: 31005 msg/sec
Slowest: 69 msg/sec
Median: 567 msg/sec

Sorry if is not a bug :)

via CI builds?

Hello,
i tried many times to connect to a local broker (127.0.0.1) (docker image eclipse-mosquitto) without success (mqtt-stresser container).
With another broker in my local network (ip adress) it's working flawlessly.
I tried to debug without success.

Here some tries (i verified with mosquitto_pub that the broker is up)

~/bin/apache-jmeter-5.4/bin $ docker run inovex/mqtt-stresser -broker :1883 -num-clients 1 -num-messages 10 -log-level 2
1 worker started
DEBUG: 11:47:58.796291 worker.go:89: [0] initializing
DEBUG: 11:47:58.796361 worker.go:104: [0] topic=internal/mqtt-stresser/de1628ebb3d9/worker0-2132015686 subscriberClientId=mqtt-stresser-sub-de1628ebb3d9-worker0-2132015686 publisherClientId=mqtt-stresser-pub-de1628ebb3d9-worker0-2132015686
DEBUG: 11:47:58.796384 worker.go:133: [0] connecting subscriber
DEBUG: 11:48:03.796613 worker.go:155: [0] subscribing to topic
DEBUG: 11:48:03.796696 worker.go:146: [0] unsubscribe
DEBUG: 11:48:03.796758 main.go:291: 1/1 events received
failed to unsubscribe: Not Connected

~/bin/apache-jmeter-5.4/bin $ docker run inovex/mqtt-stresser -broker localhost:1883 -num-clients 1 -num-messages 10 -log-level 2
1 worker started
DEBUG: 11:48:47.676236 worker.go:89: [0] initializing
DEBUG: 11:48:47.676318 worker.go:104: [0] topic=internal/mqtt-stresser/9038eac90665/worker0-1771842039 subscriberClientId=mqtt-stresser-sub-9038eac90665-worker0-1771842039 publisherClientId=mqtt-stresser-pub-9038eac90665-worker0-1771842039
DEBUG: 11:48:47.676346 worker.go:133: [0] connecting subscriber
failed to unsubscribe: Not Connected
DEBUG: 11:48:52.676640 worker.go:155: [0] subscribing to topic
DEBUG: 11:48:52.676783 worker.go:146: [0] unsubscribe
DEBUG: 11:48:52.676858 main.go:291: 1/1 events received

~/bin/apache-jmeter-5.4/bin $ docker run inovex/mqtt-stresser -broker 127.0.0.1:1883 -num-clients 1 -num-messages 10 -log-level 2
1 worker started
DEBUG: 11:49:25.085916 worker.go:89: [0] initializing
DEBUG: 11:49:25.086003 worker.go:104: [0] topic=internal/mqtt-stresser/dd21babcd6b3/worker0-1451970135 subscriberClientId=mqtt-stresser-sub-dd21babcd6b3-worker0-1451970135 publisherClientId=mqtt-stresser-pub-dd21babcd6b3-worker0-1451970135
DEBUG: 11:49:25.086023 worker.go:133: [0] connecting subscriber
EDEBUG: 11:49:30.086227 worker.go:155: [0] subscribing to topic
DEBUG: 11:49:30.086319 main.go:291: 1/1 events received
DEBUG: 11:49:30.086382 worker.go:146: [0] unsubscribe

Working with mosquitto_pub:

~/bin/apache-jmeter-5.4/bin $ mosquitto_pub -m "{\"status\":\"off\"}" -t "test/topic" --repeat 1000 --repeat-delay 2 -p 1883 -d
Client (null) sending CONNECT
Client (null) received CONNACK (0)
Client (null) sending PUBLISH (d0, q0, r0, m1, 'test/topic', ... (16 bytes))
Client (null) sending PUBLISH (d0, q0, r0, m2, 'test/topic', ... (16 bytes))
Client (null) sending PUBLISH (d0, q0, r0, m3, 'test/topic', ... (16 bytes))

Sorry if I forgot something.

Thank you,
ruio

Ps: i tried to run the binaries without success

~/Downloads $ ./mqtt-stresser-linux-amd64 -broker :1883 -num-clients 1 -num-messages 10 -log-level 2
open : no such file or directory

Ps2: I tried to make without success

~/workspace/mqtt-stresser (master)$ make
GO111MODULE=on GOOS=windows GOARCH=amd64 go build -mod=vendor -o build/mqtt-stresser-windows-amd64.exe
flag provided but not defined: -mod
usage: build [-o output] [-i] [build flags] [packages]
Run 'go help build' for details.
Makefile:81: recipe for target 'build/mqtt-stresser-windows-amd64.exe' failed
make: *** [build/mqtt-stresser-windows-amd64.exe] Error 2

I am asking how we can start mqtt stresser to generate

• 50 users with 50 mqtt connect message
• maximum of publish on /api/test topic with publish message in josn { "test:" "this is lodaing test message"}
  I have right now:

 docker run --rm inovex/mqtt-stresser -broker tcp://xxxxxxx:8883 -num-clients 50 -num-messages 150 -rampup-delay 1s -rampup-size 10 -global-timeout 180s  -timeout 20s -username xxxx -password xxx -skip-tls-verification true