Unfortunately can't download docker image from Dockerhub:

~> docker run \
                            -e BROKER_ENDPOINT=tcp://mosquitto:1883 \
                            -p 9234:9234 sapcc/mosquitto-exporter
Unable to find image 'sapcc/mosquitto-exporter:latest' locally
docker: Error response from daemon: manifest for sapcc/mosquitto-exporter:latest not found: manifest unknown: manifest unknown.
See 'docker run --help'.

Hello,

is there a list of metrics available with this exporter ?

Thank you

Hi and thank you for the exporter!

I'm trying to build it with the Nix package manager, and it seems to me (a Go novice) that the vendor/ directory is missing something. I get the following while building:

go/src/github.com/sapcc/mosquitto-exporter/main.go:12:2: cannot find package "github.com/codegangsta/cli" in any of:
        /tmp/nix-build-mosquitto_exporter-v0.2.0.drv-0/go/src/github.com/sapcc/mosquitto-exporter/vendor/github.com/codegangsta/cli (vendor tree)

Is this just me missing something silly, or should the github.com/codegangsta/cli be present in vendor/?

The listed counter metrics are already added by Mosquitto. There is no need to add the given value to the metric counter.

Prometheus.Counter doesn't offer any set method, so we need a work around.

Hello,

according to trivy sapcc/mosquitto-exporter has some vulnerabilities in go libraries:

┌─────────────────────────────────────┬────────────────┬──────────┬────────────────────────────────────┬───────────────────────────────────┬──────────────────────────────────────────────────────────────┐
│               Library               │ Vulnerability  │ Severity │         Installed Version          │           Fixed Version           │                            Title                             │
├─────────────────────────────────────┼────────────────┼──────────┼────────────────────────────────────┼───────────────────────────────────┼──────────────────────────────────────────────────────────────┤
│ github.com/prometheus/client_golang │ CVE-2022-21698 │ HIGH     │ v1.11.0                            │ 1.11.1                            │ prometheus/client_golang: Denial of service using            │
│                                     │                │          │                                    │                                   │ InstrumentHandlerCounter                                     │
│                                     │                │          │                                    │                                   │ https://avd.aquasec.com/nvd/cve-2022-21698                   │
├─────────────────────────────────────┼────────────────┼──────────┼────────────────────────────────────┼───────────────────────────────────┼──────────────────────────────────────────────────────────────┤
│ golang.org/x/net                    │ CVE-2021-33194 │ HIGH     │ v0.0.0-20200625001655-4c5254603344 │ 0.0.0-20210520170846-37e1c6afe023 │ golang: x/net/html: infinite loop in ParseFragment           │
│                                     │                │          │                                    │                                   │ https://avd.aquasec.com/nvd/cve-2021-33194                   │
├─────────────────────────────────────┼────────────────┼──────────┼────────────────────────────────────┼───────────────────────────────────┼──────────────────────────────────────────────────────────────┤
│ golang.org/x/net                    │ CVE-2021-44716 │ HIGH     │ v0.0.0-20200625001655-4c5254603344 │ 0.0.0-20211209124913-491a49abca63 │ golang: net/http: limit growth of header canonicalization    │
│                                     │                │          │                                    │                                   │ cache                                                        │
│                                     │                │          │                                    │                                   │ https://avd.aquasec.com/nvd/cve-2021-44716                   │
│                                     ├────────────────┼──────────┤                                    ├───────────────────────────────────┼──────────────────────────────────────────────────────────────┤
│                                     │ CVE-2021-31525 │ MEDIUM   │                                    │ 0.0.0-20210428140749-89ef3d95e781 │ golang: net/http: panic in ReadRequest and ReadResponse when │
│                                     │                │          │                                    │                                   │ reading a very large...                                      │
│                                     │                │          │                                    │                                   │ https://avd.aquasec.com/nvd/cve-2021-31525                   │
├─────────────────────────────────────┼────────────────┼──────────┼────────────────────────────────────┼───────────────────────────────────┼──────────────────────────────────────────────────────────────┤
│ golang.org/x/sys                    │ CVE-2022-29526 │ MEDIUM   │ v0.0.0-20210603081109-ebe580a85c40 │ 0.0.0-20220412211240-33da011f77ad │ golang: syscall: faccessat checks wrong group                │
│                                     │                │          │                                    │                                   │ https://avd.aquasec.com/nvd/cve-2022-29526                   │
└─────────────────────────────────────┴────────────────┴──────────┴────────────────────────────────────┴───────────────────────────────────┴──────────────────────────────────────────────────────────────┘

way to reproduce:

trivy image sapcc/mosquitto-exporter

2022-07-29T12:45:33.719+0200    INFO    Need to update DB
2022-07-29T12:45:33.719+0200    INFO    DB Repository: ghcr.io/aquasecurity/triv                                                                                                                                                             y-db
2022-07-29T12:45:33.719+0200    INFO    Downloading DB...
33.27 MiB / 33.27 MiB [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 24.07 MiB p/s 1.6s
2022-07-29T12:45:36.525+0200    INFO    Vulnerability scanning is enabled
2022-07-29T12:45:36.525+0200    INFO    Secret scanning is enabled
2022-07-29T12:45:36.525+0200    INFO    If your scanning is slow, please try '--security-checks vuln' to disable secret scanning
2022-07-29T12:45:36.525+0200    INFO    Please see also https://aquasecurity.github.io/trivy/v0.29.2/docs/secret/scanning/#recommendation for faster secret detection
2022-07-29T12:45:38.557+0200    INFO    Number of language-specific files: 1
2022-07-29T12:45:38.557+0200    INFO    Detecting gobinary vulnerabilities...

trivy can be found here: https://github.com/aquasecurity/trivy

can someone update the go libraries ?

thanks!

You need to define a "ClientId" to avoid problems with brokers that have the flag "allow_zero_length_clientid false" in mosquitto.conf

it would be nice if we have a binary file to run

There seems to be no option to add a username and password to authenticate at the broker when using docker.

Is there a way to configure connection retries? I had to bounce the broker and the mosquitto-exporter log ended with:

2022/08/15 00:49:37 Error: Connection to tcp://mosquitto-mqtt.mosquitto:1883 lost: EOF

No sign of a retry, the program doesn't exit out to trigger a container restart policy.

I manually restarted mosquitto-exporter and connected fine.

2022/08/15 17:57:14 Starting mosquitto_broker 0.8.0 (e268064), go1.17.2
2022/08/15 17:57:14 Connected to tcp://mosquitto-mqtt.mosquitto:1883
[store]    memorystore wiped
2022/08/15 17:57:14 Listening on 0.0.0.0:9234...

If the exporter can't connect, it should retry a set number of times and then exit out to allow the restart policy to kick in. Then it becomes a condition that can be monitored and fixed.

Is there a way to determine if the broker is reachable? If my broker goes down the exporter will expose the last metrics it was able to scrape. Im currently using something like

abs(broker_heap_current - (broker_heap_current offset 10m))

to determine if there are any changes in the heap. But that is not very reliable so far.

I'm using the 0.4.0 tagged docker image of the exporter.

Please provide an example configuration file to use.

When you pass --endpoint, you get

Incorrect Usage. flag provided but not defined: -endpoint tcp://mosquitto:1883

Only -endpoint is written instead of --endpoint
This is taking time to understand that the problem is not the you pass the option but potentially the parameter after it.

the error message is also not clear about the problem to solve.

I made a PR in Prometheus documentation repository to add this exporter to the list of exporters. In the PR the revisor (@brian-brazil) pointed out that taking the password from the command line as a parameter is insecure.

Would there be another way to do it?

Thanks for your work, i want to use your repo for my project, how do i configure this repo according to my mqtt broker ? any help is appreciated

Hi,
Tried the exporter in kubernettes cluster.
here is the logs from pod
2018/11/19 16:55:48 Starting mosquitto_broker 0.5.0 (122a6dd), go1.9.3
2018/11/19 16:55:48 Connected to tcp://10.233.96.9:1883
2018/11/19 16:55:48 Listening on 0.0.0.0:9234...

After this if i try to curl http://:9234 , its getting timed out.
Any way to find out whats happening with the exporter and why its getting timed out?

I am not very familiar with go, so I'm not sure how to debug this.

I git cloned your repository, and then tried to run make build. I get these errors:

achin@bigbox:/nas/achin/devel/mosquitto-exporter$ make build
go build -o bin/mosquitto_exporter -ldflags="-s -w -X main.Version=0.5.0 -X main.GITCOMMIT=`git rev-parse --short HEAD`" github.com/sapcc/mosquitto-exporter
can't load package: package github.com/sapcc/mosquitto-exporter: cannot find package "github.com/sapcc/mosquitto-exporter" in any of:
        /nas/achin/devel/go/src/github.com/sapcc/mosquitto-exporter (from $GOROOT)
        /nas/achin/devel/gopath/src/github.com/sapcc/mosquitto-exporter (from $GOPATH)
Makefile:19: recipe for target 'build' failed
make: *** [build] Error 1

I'm using this version of go:

go version go1.12.4 linux/amd64

Any suggestions?

FROM scratch does not include any ca-certificates, so it will fail to connect to any TLS-enabled MQTT broker.

❯ docker pull sapcc/mosquitto-exporter:0.6.0
0.6.0: Pulling from sapcc/mosquitto-exporter
257963aec3e6: Pull complete 
Digest: sha256:109e3fbbf99943eacc66a57a6e535ea43878fc63c328d1e27a928e6e69c72ddb
Status: Downloaded newer image for sapcc/mosquitto-exporter:0.6.0
docker.io/sapcc/mosquitto-exporter:0.6.0
❯ docker run -e BROKER_ENDPOINT=tls://test.mosquitto.org:8883 -p 9234:9234 sapcc/mosquitto-exporter:0.6.0 
2020/06/20 19:06:52 Starting mosquitto_broker 0.6.0 (0ac92b5), go1.12.3
2020/06/20 19:06:52 Error: Failed to connect to broker: Network Error : x509: certificate signed by unknown authority

You don't need to make changes in your code, just build the Docker images in a different way.

Manual multi-arch build:

docker buildx create --use
docker buildx build -t sapcc/mosquitto-exporter:0.8.0 --platform linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x .

# add --push to publish to DockerHub

Automate Docker build and publishing with GitHub actions:
https://github.com/FlareSolverr/FlareSolverr/blob/master/.github/workflows/release-docker.yml

My file:

version: '3.8'

services:

  mosquitto:
    image: eclipse-mosquitto:2.0.14
    container_name: mosquitto
    environment:
      - TZ=Europe/Madrid
    ports:
      - "1883:1883"
    volumes:
      - mosquitto_volume/config:/mosquitto/config
      - mosquitto_volume/data:/mosquitto/data
      - mosquitto_volume/log:/mosquitto/log
    restart: unless-stopped

  mosquitto-exporter:
    image: sapcc/mosquitto-exporter:0.8.0
    container_name: mosquitto-exporter
    command: --endpoint tcp://mosquitto:1883
    environment:
      - TZ=Europe/Madrid
    ports:
      - "9234:9234"
    restart: unless-stopped

Download release. Unpack. Make build.

Doesn't work.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update module github.com/prometheus/client_golang to v1.11.1 [SECURITY]
• Update golang Docker tag to v1.20
• Update module github.com/codegangsta/cli to v1.22.15
• Update module github.com/eclipse/paho.mqtt.golang to v1.4.3
• Update module github.com/codegangsta/cli to v3
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

If not, whenever I get one together I'll tack it onto this issue. It would be handy.

I made a PR in Prometheus documentation repository to add this exporter to the list of exporters. In the PR the revisor (@brian-brazil) pointed out some changes that would be needed to comply with the naming conventions of Prometheus metrics.

• Add the sufix "_total" to all the counter metrics
• Change $SYS/broker/uptime and $SYS/broker/clients/maximum to type gauge

Also would be interesting to:

• Include $SYS/broker/version as a metric
• Include $SYS/broker/timestamp
• Include a metric for when the last update was received (this would be included in the PR #22 )

The most complex here would be adding the sufix to all the counters, due to the metrics would change from the current version to this one and all the queries using this metrics would need to be updated.

my docker running: sudo docker logs -f 25baf2f8d6b0
2019/03/26 20:50:05 Starting mosquitto_broker 0.5.0 (122a6dd), go1.11
2019/03/26 20:50:05 Listening on 0.0.0.0:9234...
^C
but curl output doesn't have any mosquitto related metrics. Does anyone have any clue? thx!!!

curl localhost:9234/metrics

HELP go_gc_duration_seconds A summary of the GC invocation durations.

TYPE go_gc_duration_seconds summary

go_gc_duration_seconds{quantile="0"} 6.4097e-05
go_gc_duration_seconds{quantile="0.25"} 8.333e-05
go_gc_duration_seconds{quantile="0.5"} 9.154e-05
go_gc_duration_seconds{quantile="0.75"} 0.000101585
go_gc_duration_seconds{quantile="1"} 0.000273718
go_gc_duration_seconds_sum 0.012934772
go_gc_duration_seconds_count 135

HELP go_goroutines Number of goroutines that currently exist.

TYPE go_goroutines gauge

go_goroutines 12

HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.

TYPE go_memstats_alloc_bytes gauge

go_memstats_alloc_bytes 2.47416e+06

HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.

TYPE go_memstats_alloc_bytes_total counter

go_memstats_alloc_bytes_total 3.7431448e+08

HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.

TYPE go_memstats_buck_hash_sys_bytes gauge

go_memstats_buck_hash_sys_bytes 1.453515e+06

HELP go_memstats_frees_total Total number of frees.

TYPE go_memstats_frees_total counter

go_memstats_frees_total 529392

HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.

TYPE go_memstats_gc_sys_bytes gauge

go_memstats_gc_sys_bytes 2.371584e+06

HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.

TYPE go_memstats_heap_alloc_bytes gauge

go_memstats_heap_alloc_bytes 2.47416e+06

HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.

TYPE go_memstats_heap_idle_bytes gauge

go_memstats_heap_idle_bytes 6.2644224e+07

HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.

TYPE go_memstats_heap_inuse_bytes gauge

go_memstats_heap_inuse_bytes 3.448832e+06

HELP go_memstats_heap_objects Number of allocated objects.

TYPE go_memstats_heap_objects gauge

go_memstats_heap_objects 3366

HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.

TYPE go_memstats_heap_released_bytes gauge

go_memstats_heap_released_bytes 958464

HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.

TYPE go_memstats_heap_sys_bytes gauge

go_memstats_heap_sys_bytes 6.6093056e+07

HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.

TYPE go_memstats_last_gc_time_seconds gauge

go_memstats_last_gc_time_seconds 1.5537113622319465e+09

HELP go_memstats_lookups_total Total number of pointer lookups.

TYPE go_memstats_lookups_total counter

go_memstats_lookups_total 0

HELP go_memstats_mallocs_total Total number of mallocs.

TYPE go_memstats_mallocs_total counter

go_memstats_mallocs_total 532758

HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.

TYPE go_memstats_mcache_inuse_bytes gauge

go_memstats_mcache_inuse_bytes 13824

HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.

TYPE go_memstats_mcache_sys_bytes gauge

go_memstats_mcache_sys_bytes 16384

HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.

TYPE go_memstats_mspan_inuse_bytes gauge

go_memstats_mspan_inuse_bytes 32832

HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.

TYPE go_memstats_mspan_sys_bytes gauge

go_memstats_mspan_sys_bytes 65536

HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.

TYPE go_memstats_next_gc_bytes gauge

go_memstats_next_gc_bytes 4.194304e+06

HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.

TYPE go_memstats_other_sys_bytes gauge

go_memstats_other_sys_bytes 1.792813e+06

HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.

TYPE go_memstats_stack_inuse_bytes gauge

go_memstats_stack_inuse_bytes 1.015808e+06

HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.

TYPE go_memstats_stack_sys_bytes gauge

go_memstats_stack_sys_bytes 1.015808e+06

HELP go_memstats_sys_bytes Number of bytes obtained from system.

TYPE go_memstats_sys_bytes gauge

go_memstats_sys_bytes 7.2808696e+07

HELP go_threads Number of OS threads created

TYPE go_threads gauge

go_threads 16

HELP http_request_duration_microseconds The HTTP request latencies in microseconds.

TYPE http_request_duration_microseconds summary

http_request_duration_microseconds{handler="prometheus",quantile="0.5"} 1100.171
http_request_duration_microseconds{handler="prometheus",quantile="0.9"} 1531.711
http_request_duration_microseconds{handler="prometheus",quantile="0.99"} 1537.143
http_request_duration_microseconds_sum{handler="prometheus"} 510585.29800000007
http_request_duration_microseconds_count{handler="prometheus"} 428

HELP http_request_size_bytes The HTTP request sizes in bytes.

TYPE http_request_size_bytes summary

http_request_size_bytes{handler="prometheus",quantile="0.5"} 213
http_request_size_bytes{handler="prometheus",quantile="0.9"} 213
http_request_size_bytes{handler="prometheus",quantile="0.99"} 213
http_request_size_bytes_sum{handler="prometheus"} 87718
http_request_size_bytes_count{handler="prometheus"} 428

HELP http_requests_total Total number of HTTP requests made.

TYPE http_requests_total counter

http_requests_total{code="200",handler="prometheus",method="get"} 428

HELP http_response_size_bytes The HTTP response sizes in bytes.

TYPE http_response_size_bytes summary

http_response_size_bytes{handler="prometheus",quantile="0.5"} 1450
http_response_size_bytes{handler="prometheus",quantile="0.9"} 1459
http_response_size_bytes{handler="prometheus",quantile="0.99"} 6725
http_response_size_bytes_sum{handler="prometheus"} 736369
http_response_size_bytes_count{handler="prometheus"} 428

HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.

TYPE process_cpu_seconds_total counter

process_cpu_seconds_total 1.07

HELP process_max_fds Maximum number of open file descriptors.

TYPE process_max_fds gauge

process_max_fds 1.048576e+06

HELP process_open_fds Number of open file descriptors.

TYPE process_open_fds gauge

process_open_fds 9

HELP process_resident_memory_bytes Resident memory size in bytes.

TYPE process_resident_memory_bytes gauge

process_resident_memory_bytes 1.9488768e+07

HELP process_start_time_seconds Start time of the process since unix epoch in seconds.

TYPE process_start_time_seconds gauge

process_start_time_seconds 1.5536334051e+09

HELP process_virtual_memory_bytes Virtual memory size in bytes.

TYPE process_virtual_memory_bytes gauge

process_virtual_memory_bytes 1.14618368e+08

The regex parsing the input string does not handle negative number like -23.42. It parses those numbers as positive values.