oliver006 / redis_exporter Goto Github PK

Latency monitoring is a relatively new feature introduced in Redis 2.8.13 that helps you troubleshoot latency problems. This tool logs latency spikes on your server, and the events that cause them. You must enable latency monitoring before you can use it however but I guess it would be nice metrics to have.
More details here: https://redis.io/topics/latency-monitor

Let me know if this is at all possible. I would love to work on it and see what we can come up with. Just getting my hands dirty to this whole open source thing, no big deal.

[root@Redis-master redis_exporter]# pwd
/root/redis_exporter

[root@Redis-master redis_exporter]# go get
go install: no install location for directory /root/redis_exporter outside GOPATH
For more details see: 'go help gopath'
[root@Redis-master redis_exporter]# ./redis_exporter -web.listen-address=10.21.20.241:9121 -redis.addr=redis://127.0.0.1:6379 -redis.password=xxx &
[1] 87963
INFO[0000] Redis Metrics Exporter <<< filled in by build >>> build date: <<< filled in by build >>> sha1: <<< filled in by build >>> Go: go1.8.3

INFO[0000] Providing metrics at 10.21.20.241:9121/metrics
INFO[0000] Connecting to redis hosts: []string{"redis://127.0.0.1:6379"}
INFO[0000] Using alias: []string{""}
[root@Redis-master redis_exporter]#

here am I start the redis_exporter right? and when i add a block to the scrape_configs of your prometheus.yml config file, i get the error in prometheus targets page:
http://10.21.20.241:9121/metrics
DOWN group="redis" instance="10.21.20.241:9121" 4.219s ago Get http://10.21.20.241:9121/metrics: dial tcp 10.21.20.241:9121: getsockopt: no route to host

here i don't know how to fix it? can you help me? thanks much.
[root@Redis-master redis_exporter]# ps -ef|grep redis
devdepl+ 31096 1 3 Aug03 ? 06:33:11 /usr/local/redis/bin/redis-server *:6379
root 62764 8869 0 Aug10 pts/2 00:00:00 ./redis_exporter -web.listen-address=10.21.20.241:9121 -redis.addr=redis://127.0.0.1:6379 -redis.password=DevCoRe657x
root 87431 99325 0 08:57 pts/1 00:00:00 grep --color=auto redis

Hi!

Nice work, we are using it every day!
For our docker setup I would like to know if it is possible to overwrite all flags with env variables. I could provide a pull request if you like.

With kind regards,
Nighthawk

Hello,

It seams that the key-check feature doesn't work with keys with ':' in the name, example "cntr:flood".

Max.

Hello, @oliver006 !

Thank you for you exporter! We are currently relying on it in our installation.

I've found out a strange behavior. Not sure if it is a feature or a bug.

We have multiple Redis servers and one redis-exporter that scrapes them all.
I want to make a designated key from each Redis to be visible to Prometheus. I've added -check-keys option. But it only shows the value from one server which it scrapes last.

Here is how to quickly reproduce the problem:

docker-compose.yml

services:
  redis1:
    image: redis:3.2.7
    container_name: redis1
    expose:
      - 6379
  redis2:
    image: redis:3.2.7
    container_name: redis2
    expose:
      - 6379
  redisexporter:
    container_name: redis_exporter
    image: oliver006/redis_exporter
    command:
      - '-check-keys=testkey'
      - '-redis.addr=redis://redis1:6379,redis://redis2:6379'
    expose:
      - 9121
    ports:
      - 127.0.0.1:9121:9121

Commands:

docker-compose -f docker-compose.yml up -d
docker exec -it redis1 redis-cli set testkey 1111
docker exec -it redis2 redis-cli set testkey 2222
curl -s localhost:9121/metrics | grep testkey

Thank you!

I tried to install new release of exporter to my server, but in metrics is a value with older version.
In your changelog is adding metric "up" to new release, but I haven't this value in my redis exporter.
Is that a new release of exporter?

redis_exporter_build_info{build_date="2016-12-02-21:09:17",commit_sha="a410b50d53985b76f117a6d9ee81ab052c8c1c91",golang_version="go1.7.3",version="v0.10.3"} 1

Thank you :)

Collecting metrics for config options like max_memory could be useful and should be easy to implement by parsing the output of CONFIG GET *.

https://circleci.com/docs/2.0/configuration-reference/

Once Go 1.9 is GA we should use sync.Map for e.metrics to simplify all the
lock -> set value -> unlock sequences.

Hi @oliver006,

I checked today the latest 0.11.2 version with promtool (Pass Prometheus metrics over stdin to lint them for consistency and correctness), and I think this could be fixed.

You can get promtool tool with,

GO15VENDOREXPERIMENT=1 go get github.com/prometheus/prometheus/cmd/promtool

# curl -s http://localhost.localdomain:9121/metrics | promtool check-metrics
redis_aof_current_rewrite_duration_sec: no help text
redis_aof_enabled: no help text
redis_aof_last_rewrite_duration_sec: no help text
redis_aof_rewrite_in_progress: no help text
redis_aof_rewrite_scheduled: no help text
redis_blocked_clients: no help text
redis_command_call_duration_seconds_count: non-histogram and non-summary metrics should not have "_count" suffix
http_request_duration_microseconds: use base unit "seconds" instead of "microseconds"
redis_client_longest_output_list: no help text
redis_command_call_duration_seconds_sum: non-histogram and non-summary metrics should not have "_sum" suffix
redis_commands_processed_total: no help text
redis_connected_clients: no help text
redis_connected_slaves: no help text
redis_connections_received_total: no help text
redis_expired_keys_total: no help text
redis_commands_processed_total: non-counter metrics should not have "_total" suffix
redis_connections_received_total: non-counter metrics should not have "_total" suffix
redis_evicted_keys_total: no help text
redis_expired_keys_total: non-counter metrics should not have "_total" suffix
redis_instantaneous_ops_per_sec: no help text
redis_keyspace_hits_total: no help text
redis_latest_fork_usec: no help text
redis_loading_dump_file: no help text
redis_memory_fragmentation_ratio: no help text
redis_memory_used_bytes: no help text
redis_rdb_changes_since_last_save: no help text
redis_evicted_keys_total: non-counter metrics should not have "_total" suffix
redis_keyspace_hits_total: non-counter metrics should not have "_total" suffix
redis_keyspace_misses_total: no help text
redis_master_repl_offset: no help text
redis_memory_used_lua_bytes: no help text
redis_memory_used_peak_bytes: no help text
redis_memory_used_rss_bytes: no help text
redis_net_input_bytes_total: no help text
redis_net_output_bytes_total: no help text
redis_process_id: no help text
redis_pubsub_channels: no help text
redis_pubsub_patterns: no help text
redis_rdb_current_bgsave_duration_sec: no help text
redis_rdb_last_bgsave_duration_sec: no help text
redis_rejected_connections_total: no help text
redis_uptime_in_seconds: no help text
redis_rejected_connections_total: non-counter metrics should not have "_total" suffix
redis_replication_backlog_bytes: no help text
redis_up: no help text
redis_used_cpu_sys: no help text
redis_used_cpu_sys_children: no help text
redis_used_cpu_user: no help text
redis_used_cpu_user_children: no help text
redis_keyspace_misses_total: non-counter metrics should not have "_total" suffix
redis_net_input_bytes_total: non-counter metrics should not have "_total" suffix
redis_net_output_bytes_total: non-counter metrics should not have "_total" suffix

The uptime & clents & memory usage： Can not get its value

The stack trace:

Error: Multiple Series Error
at b.setValues (http://192.168.1.2:4001/public/app/boot.c8e57b1c.js:29:8355)
at b.onDataReceived (http://192.168.1.2:4001/public/app/boot.c8e57b1c.js:29:6107)
at f.emit (http://192.168.1.2:4001/public/app/boot.c8e57b1c.js:61:19286)
at a.emit (http://192.168.1.2:4001/public/app/boot.c8e57b1c.js:61:21604)
at b.handleQueryResult (http://192.168.1.2:4001/public/app/boot.c8e57b1c.js:55:21101)
at i (http://192.168.1.2:4001/public/app/boot.c8e57b1c.js:51:24759)
at http://192.168.1.2:4001/public/app/boot.c8e57b1c.js:51:25181
at o.$eval (http://192.168.1.2:4001/public/app/boot.c8e57b1c.js:52:293)
at o.$digest (http://192.168.1.2:4001/public/app/boot.c8e57b1c.js:51:30809)
at o.$apply (http://192.168.1.2:4001/public/app/boot.c8e57b1c.js:52:576)

I found a bug about uptime when my time range was chosen to show 24 hours but my redis started just 2 minutes at this time, Uptime was N/A state and reported above the error. May be the clents & memory usage have been the same issue.

Hi!
thanks for redis_exporter!
In line with other exporters (e.g. memcached) it'd be nice to provide a gauge redis_up 0/1 whenever redis scraping fails or succeeds.

thanks!

Rather than passing the configuration flags as command line options could you add reading these from a configuration file? The primary concern is that when securing the RedisDB with a password, the password must then be passed as command line argument to the redis_exporter. Anyone who gains access to the machine could then do a simple ps command and observe the password. This violates various security requirements.

Hi, Im using old redis version 2.8 and there is no maxmemory in the INFO result. What about use "CONFIG GET maxmemory" on old redis version?

As redis has slow log metrics to show the slowest CMD, we should export these to prom. Thus, grafana could show the topK slow CMD, and makes redis ops feel happy.

Figure out if we can do this automatically via CircleCI, if not just do it manually but we should provide binaries.

I am trying to get this exported working against an Azure Redis Cache instance.
By default it uses SSL on port 6380.

I can get it working if I disable 'Allow access only via SSL' and connect to port 6379.
Is SSL support something you could add to the exporter or should I be running something else to handle the SSL connection?

Especially the master discovery part.

Hi, oliver006. Something puzzled me. I find redis_exporter do the scraping every 15 seconds(default). Is it too long ? I mean, there're probably many metrics changes within 15 seconds, we may miss the most part of them. Unless these redis metrics obtained by INFO command only go up(never go down), 15 seconds is workable.

However, as far as I know, there are some metrics that could either go up or go down, so you may not capture the major changes if setting the interval too long.

subj

CLUSTER INFO (http://redis.io/commands/cluster-info) has useful stats that could/should be exported.

howdy, after a bad period on our redis cluster the exporter stopper working as we started seeing:

invalid metric name "redis_cluster_stats_messages_auth-ack_received"

There's also

redis_cluster_stats_messages_auth-req_sent

Zip files are much easier to deal with on windows than tar.gz files.

For Redis_export where its log will be output? For docker way to pull up, whether you can set volumes for it log?

Redis 4 is stable, but this exporter only supports 2. 3. according to doc.

With used_memory in INFO memory, plus CONFIG GET maxmemory.

I noticed that the redis exporter has the ability to scrape multiple redis servers, and adds a addr label that partially duplicates the instance label. Per https://prometheus.io/docs/instrumenting/writing_exporters/#deployment there should be a 1-to-1 mapping between exporters and application instances.

The readme doesn't explain where flags should be specified. I'm using docker, how can I specify flags?

Is it normal that we have a PrintLn on the parseDBKeyspaceString method?

https://github.com/oliver006/redis_exporter/blob/master/exporter/redis.go#L205

Jul 27 08:21:45 r213-srv18 rkt[47236]: [82783.876068] redis_exporter[260]: 2016/07/27 08:21:45 740 740 <nil>
Jul 27 08:21:45 r213-srv18 rkt[47236]: [82783.876428] redis_exporter[260]: 2016/07/27 08:21:45 740 740 <nil>
Jul 27 08:21:45 r213-srv18 rkt[47236]: [82783.876589] redis_exporter[260]: 2016/07/27 08:21:45 157212478 1.57212478e+08 <nil>

Can you support Docker startup with environment variables? For example:

  Environment:
         REDIS_ADDR: 192.168.1.2: 4379
         REDIS_PASSWORD: aaaaaa

I want to create a docker image for redis_exporter based on vanilla alpine:3.5 that by default lacks the libraries required to run dynamically linked binaries such as the one you provide on the Releases tab. E.g. the https://github.com/prometheus/blackbox_exporter/ project does this and it would be very nice if you could, too :) That would allow creating docker images without having to locally compile the go program again.

Maybe http://blog.wrouesnel.com/articles/Totally%20static%20Go%20builds/ helps.

Dockerfile would probably look similar to https://github.com/discordianfish/nginx_exporter/blob/master/Dockerfile

[root@Redis-master redis_exporter]# pwd
/root/go/src/github.com/oliver006/redis_exporter
[root@Redis-master redis_exporter]# ll
total 7572
-rwxr-xr-x 1 root root 2540 Aug 8 05:42 build.sh
-rw-r----- 1 root root 2223 Aug 8 05:42 circle.yml
drwxr-x--- 2 root root 4096 Aug 8 05:42 contrib
-rw-r----- 1 root root 160 Aug 8 05:42 Dockerfile
drwxr-x--- 2 root root 4096 Aug 8 05:42 exporter
-rw-r----- 1 root root 1308 Aug 8 05:42 glide.lock
-rw-r----- 1 root root 366 Aug 8 05:42 glide.yaml
-rw-r----- 1 root root 1063 Aug 8 05:42 LICENSE
-rw-r----- 1 root root 5900 Aug 8 05:42 main.go
-rw-r----- 1 root root 4174 Aug 8 05:42 README.md
-rwxr-x--- 1 root root 7684197 Aug 11 10:14 redis_exporter
drwxr-x--- 3 root root 4096 Aug 11 10:11 src
[root@Redis-master redis_exporter]# ./redis_exporter -web.listen-address=10.21.20.241:9121 -redis.addr=redis://127.0.0.1:6379 -redis.password=password
INFO[0000] Redis Metrics Exporter <<< filled in by build >>> build date: <<< filled in by build >>> sha1: <<< filled in by build >>> Go: go1.8.3

INFO[0000] Providing metrics at 10.21.20.241:9121/metrics
INFO[0000] Connecting to redis hosts: []string{"redis://127.0.0.1:6379"}
INFO[0000] Using alias: []string{""}

At the same machine:

[root@Redis-master redis_exporter]# telnet 10.21.20.241 9121
Trying 10.21.20.241...
Connected to 10.21.20.241.
Escape character is '^]'.

HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Connection: close

400 Bad RequestConnection closed by foreign host.
[root@Redis-master redis_exporter]#

At other machine:
[root@SZX1000347429 ~]# telnet 10.21.20.241 9121
Trying 10.21.20.241...
telnet: connect to address 10.21.20.241: No route to host

Is the way that I start the redis_exporter not correct? how should i do? thanks much!
And prometheus error message is:
Get http://10.21.20.241:9121/metrics: dial tcp 10.21.20.241:9121: getsockopt: no route to host

Hi @oliver006,
I would like make a kind of feature request. Could it be possible having a Makefile here, as well as Build Information handle correctly like the most of Prometheus-Exporters? If you are interested about it, I would be making a PR soon. Thanks!

Hi @oliver006,

I think it would be nice to have the Dashboards in an specific directory, like /dashboards.

subj

Can you tell me the whole interface logic of this redis?

Through the command redis-cli 127.0.0.1 -p zzzaa001 connection redis is good. But the redis_exporter error is as follows:

[root@acserver ~]# docker run -e "REDIS_ADDR=127.0.0.1:6379" -e "REDIS_PASSWORD=zzzaa001" --name redis_exporter -p 9121:9121 oliver006/redis_exporter
time="2017-06-21T09:12:59Z" level=info msg="Redis Metrics Exporter v0.11.1    build date: 2017-05-24-14:10:15    sha1: d3af2b49709a2cab654d7c876497e791c6d9a082    Go: go1.8.1\n" 
time="2017-06-21T09:12:59Z" level=info msg="Providing metrics at :9121/metrics" 
time="2017-06-21T09:12:59Z" level=info msg="Connecting to redis hosts: []string{\"127.0.0.1:6379\"}" 
time="2017-06-21T09:12:59Z" level=info msg="Using alias: []string{\"\"}" 
time="2017-06-21T09:13:03Z" level=info msg="redis err: dial tcp 127.0.0.1:6379: getsockopt: connection refused" 
time="2017-06-21T09:13:08Z" level=info msg="redis err: dial tcp 127.0.0.1:6379: getsockopt: connection refused" 
time="2017-06-21T09:13:13Z" level=info msg="redis err: dial tcp 127.0.0.1:6379: getsockopt: connection refused" 

And my prometheus cofnig is:

  - job_name: redis_exporter_dev

    scrape_interval: 5s

    static_configs:
    - targets: ['192.168.1.2:9121']

  - job_name: redis_exporter_test

    scrape_interval: 5s

    static_configs:
    - targets: ['192.168.1.3:9121']

Use default redisaddr the error as below:

[root@acserver ~]# docker run -e "REDIS_PASSWORD=zzzaa001" --name redis_exporter -p 9121:9121 oliver006/redis_exporter
time="2017-06-21T09:32:22Z" level=info msg="Redis Metrics Exporter v0.11.1    build date: 2017-05-24-14:10:15    sha1: d3af2b49709a2cab654d7c876497e791c6d9a082    Go: go1.8.1\n" 
time="2017-06-21T09:32:22Z" level=info msg="Providing metrics at :9121/metrics" 
time="2017-06-21T09:32:22Z" level=info msg="Connecting to redis hosts: []string{\"redis://localhost:6379\"}" 
time="2017-06-21T09:32:22Z" level=info msg="Using alias: []string{\"\"}" 
time="2017-06-21T09:32:22Z" level=info msg="redis err: dial redis: unknown network redis"

hi,
Can you share the gafana template of redis ?
thanks!

Through the command redis-cli 127.0.0.1 -p zzzaa001 connection redis is good. But the redis_exporter error is as follows:

[root@acserver ~]# docker run -e "REDIS_ADDR=127.0.0.1:6379" -e "REDIS_PASSWORD=zzzaa001" --name redis_exporter -p 9121:9121 oliver006/redis_exporter
time="2017-06-21T09:12:59Z" level=info msg="Redis Metrics Exporter v0.11.1    build date: 2017-05-24-14:10:15    sha1: d3af2b49709a2cab654d7c876497e791c6d9a082    Go: go1.8.1\n" 
time="2017-06-21T09:12:59Z" level=info msg="Providing metrics at :9121/metrics" 
time="2017-06-21T09:12:59Z" level=info msg="Connecting to redis hosts: []string{\"127.0.0.1:6379\"}" 
time="2017-06-21T09:12:59Z" level=info msg="Using alias: []string{\"\"}" 
time="2017-06-21T09:13:03Z" level=info msg="redis err: dial tcp 127.0.0.1:6379: getsockopt: connection refused" 
time="2017-06-21T09:13:08Z" level=info msg="redis err: dial tcp 127.0.0.1:6379: getsockopt: connection refused" 
time="2017-06-21T09:13:13Z" level=info msg="redis err: dial tcp 127.0.0.1:6379: getsockopt: connection refused" 

And my prometheus cofnig is:

  - job_name: redis_exporter_dev

    scrape_interval: 5s

    static_configs:
    - targets: ['192.168.1.2:9121']

  - job_name: redis_exporter_test

    scrape_interval: 5s

    static_configs:
    - targets: ['192.168.1.3:9121']

Is there a special reason to have filtered the rdb and aof mettrics?

Running redis_exporter in alpine:edge with

/go/bin/redis_exporter -web.listen-address=9121 -redis.alias=control -debug

it gives the following logs:

INFO[0000] Redis Metrics Exporter <<< filled in by build >>>    build date: <<< filled in by build >>>    sha1: <<< filled in by build >>>
 
DEBU[0000] Enabling debug output                        
INFO[0000] Providing metrics at 9121/metrics            
INFO[0000] Connecting to redis hosts: []string{"redis://localhost:6379"} 
INFO[0000] Using alias: []string{"control"}             
FATA[0000] listen tcp: missing port in address 9121 

I'm running it inside a docker container alongside with redis instalation (that is running in the test above) and with go 1.7.4 and docker 17.03.1-ce-rc1.

This exporter is currently using direct instrumentation, which is strong discouraged for custom collectors. It should be switched to use ConstMetrics, and all the mutexes can then be removed.

hi my friend ,

i need your help to expose redis-cli info to /metrics in redis exporter so the grafana dashboard can collect them

thanks

Maybe I'm missing something but I don't see any Redis metrics around databases/keys/hit/miss

testing on various Redis versions from 2.4.x to 3.0.x and the only redis_ prefixed metrics I see are:

# HELP redis_exporter_last_scrape_duration_seconds The last scrape duration.
# TYPE redis_exporter_last_scrape_duration_seconds gauge
redis_exporter_last_scrape_duration_seconds 0.001755366
# HELP redis_exporter_last_scrape_error The last scrape error status.
# TYPE redis_exporter_last_scrape_error gauge
redis_exporter_last_scrape_error 1
# HELP redis_exporter_scrapes_total Current total redis scrapes.
# TYPE redis_exporter_scrapes_total counter
redis_exporter_scrapes_total 1

The latest docker image does not seem to be built correctly, the build_info series is not populated:

$ docker run -d -P oliver006/redis_exporter:v0.9.1
Unable to find image 'oliver006/redis_exporter:v0.9.1' locally
v0.9.1: Pulling from oliver006/redis_exporter
[...]
Digest: sha256:8c9585f48075dcf6888f4bca63813052c147d13a0fb9d7b0677c38b08ac13423
Status: Downloaded newer image for oliver006/redis_exporter:v0.9.1
$ curl localhost:32771/metrics | grep version
redis_exporter_build_info{build_date="<<< filled in by build >>>",commit_sha="<<< filled in by build >>>",golang_version="go1.6.3",version="<<< filled in by build >>>"} 1

issues like #51 could be detected more easily if there were better tests using more than one redis instance.

If provided with multiple redis hosts and one of them is inactive/dead via -redis.addr parameter - the exporter hangs even if other redis hosts are up and running.

Way to reproduce:
$./redis_exporter -redis.addr working_server1_IP:6379, working_server2_IP:6379,dead_server1_IP:6379

$wget http://localhost:9121/metrics