Converts JSON data from a http url to prometheus metrics using jsonpath

License: MIT License

Python 100.00%

prometheus-jsonpath-exporter's Introduction

prometheus-jsonpath-exporter

Converts json data from a http url into prometheus metrics using jsonpath

Config

exporter_port: 9158 # Port on which prometheus can call this exporter to get metrics
log_level: info
json_data_url: http://stubonweb.herokuapp.com/kong-cluster-status # Url to get json data used for fetching metric values
metric_name_prefix: kong_cluster # All metric names will be prefixed with this value
metrics:
- name: total_nodes # Final metric name will be kong_cluster_total_nodes
  description: Total number of nodes in kong cluster
  path: $.total
- name: alive_nodes # Final metric name will be kong_cluster_alive_nodes
  description: Number of live nodes in kong cluster
  path: count($.data[@.status is "alive"])

See the example below to understand how the json data and metrics will look for this config

Run

Using code (local)

# Ensure python 2.x and pip installed
pip install -r app/requirements.txt
python app/exporter.py example/config.yml

Using docker

docker run -p 9158:9158 -v $(pwd)/example/config.yml:/etc/prometheus-jsonpath-exporter/config.yml sunbird/prometheus-jsonpath-exporter /etc/prometheus-jsonpath-exporter/config.yml

JsonPath Syntax

This exporter uses objectpath python library. The syntax is documented here

Example

For the above config, if the configured json_data_url returns

{
  "data": [
    {
      "address": "x.x.x.15:7946",
      "status": "failed"
    },
    {
      "address": "x.x.x.19:7946",
      "status": "alive"
    },
    {
      "address": "x.x.x.12:7946",
      "status": "alive"
    }
  ],
  "total": 3
}

Metrics will available in http://localhost:9158

$ curl -s localhost:9158 | grep ^kong
kong_cluster_total_nodes 3.0
kong_cluster_alive_nodes 2.0

prometheus-jsonpath-exporter's People

Contributors

Stargazers

Watchers

prometheus-jsonpath-exporter's Issues

Prometheus exporter converting metrics using jsonpath

Support HTTP urls returning json data
Support jsonpath allwing filters based on condition
Support functions like count, sum, min, max

Error gettting values when number is between quotes (string)

Hi,

I'm trying to parse a json object that contains:
"NumberOfUsers":"30"

The number is between quotes. I can't change it, it an off the shelf app that generates this json.

In example.yaml I have the following metrics definition:

- name: NumberOfUsers
  description: Number of users
  path: $.NumberOfUsers

When I curl the endpoint it throws the following error:

2018-01-21 15:58:35,551 - root - DEBUG - metric_name: test_NumberOfUsers, value for '$.NumberOfUsers'
2018-01-21 15:58:35,551 - root - DEBUG - metric_name: test_NumberOfUsers, value for '$.NumberOfUsers' : 30
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/SocketServer.py", line 596, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/local/lib/python2.7/SocketServer.py", line 331, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/lib/python2.7/SocketServer.py", line 652, in __init__
    self.handle()
  File "/usr/local/lib/python2.7/BaseHTTPServer.py", line 340, in handle
    self.handle_one_request()
  File "/usr/local/lib/python2.7/BaseHTTPServer.py", line 328, in handle_one_request
    method()
  File "/usr/local/lib/python2.7/site-packages/prometheus_client/exposition.py", line 94, in do_GET
    output = generate_latest(registry)
  File "/usr/local/lib/python2.7/site-packages/prometheus_client/exposition.py", line 80, in generate_latest
    output.append('{0}{1} {2}\n'.format(name, labelstr, core._floatToGoString(value)))
  File "/usr/local/lib/python2.7/site-packages/prometheus_client/core.py", line 823, in _floatToGoString
    elif math.isnan(d):
TypeError: a float is required

multiple json sources

Hello

does this support the ability to setup multiple source urls ? I am looking for a solution that can easily cater for large number of json feeds and turn them into something that can be consumed by Prometheus.

The example here only shows a single url and was wondering if this scales ?

thanks.

Pete

the ask about config path

exporter_port: 9158 # Port on which prometheus can call this exporter to get metrics
log_level: info
json_data_url: http://stubonweb.herokuapp.com/kong-cluster-status # Url to get json data used for fetching metric values
metric_name_prefix: kong_cluster # All metric names will be prefixed with this value
metrics:

name: total_nodes # Final metric name will be kong_cluster_total_nodes
description: Total number of nodes in kong cluster
path: $.total
name: alive_nodes # Final metric name will be kong_cluster_alive_nodes
description: Number of live nodes in kong cluster
path: count($.data[@.status is "alive"])

hello，Can i ask a question，Can path use python syntax

Getting error "AttributeError: 'NoneType' object has no attribute 'get'

Hi,

While using the docker image for a deployment in kubernetes, I am getting the below error.

Traceback (most recent call last):
  File "/opt/prometheus-jsonpath-exporter/exporter.py", line 40, in <module>
    log_level = config.get('log_level', DEFAULT_LOG_LEVEL)
AttributeError: 'NoneType' object has no attribute 'get

Can someone help me...?

Error when pointing to an url in ip and port.

When I include an url with ip and port I get an error.

2018-06-08 14:48:32,622 - root - INFO - Starting server on port 9158

Traceback (most recent call last):
File "app/exporter.py", line 46, in
REGISTRY.register(JsonPathCollector(config))
File "/usr/local/lib/python2.7/dist-packages/prometheus_client/core.py", line 49, in register
names = self._get_names(collector)
File "/usr/local/lib/python2.7/dist-packages/prometheus_client/core.py", line 86, in _get_names
for metric in desc_func():
File "app/exporter.py", line 22, in collect
result = json.loads(urllib2.urlopen(config['json_data_url'], timeout=10).read())
File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 435, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 473, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 406:

I have tested the url and it works

project-sunbird / prometheus-jsonpath-exporter Goto Github PK