Giter Club home page Giter Club logo

mongodb-grafana's Introduction

MongoDB datasource for Grafana

Features

Allows MongoDB to be used as a data source for Grafana by providing a proxy to convert the Grafana Data source API into MongoDB aggregation queries

Current Version

  • Grafana plugin - 0.0.1
  • Server version - 0.0.5

Requirements

  • Grafana > 3.x.x
  • MongoDB > 3.4.x

Installation

Install the Grafana plugin components

  • Copy the whole mongodb-grafana dir into the Grafana plugins dir ( /usr/local/var/lib/grafana/plugins )
  • Restart the Grafana server. If installed via Homebrew, this will be brew services restart grafana

Install and Start the MongoDB proxy server

  • Open a command prompt in the mongodb-grafana directory
  • Run npm install to install the node.js dependencies
  • Run npm run server to start the REST API proxy to MongoDB. By default, the server listens on http://localhost:3333

Examples

Create a new data source of type MongoDB as shown below. The MongoDB details are :

Add Http Header:

  • mongodb_url : mongodb://rpiread:[email protected]:27017,rpi-sensor-data-shard-00-01-ifxxs.mongodb.net:27017,rpi-sensor-data-shard-00-02-ifxxs.mongodb.net:27017/test?ssl=true&replicaSet=rpi-sensor-data-shard-0&authSource=admin

Optional Parameters:

  • MongoDB Database - rpi

Sample Data Source

Then save the data source

Example 1 - Simple aggregate to rename fields

Import the dashboard in examples\RPI MongoDB - Atlas.json

This should show a graph of light sensor values from a Raspberry PI with an EnviroPHAT board feeding readings every minute into a MongoDB Atlas database.

Sample Dashboard

Clicking on the title of the graph allows you to see the aggregation query being run against the 'RPI Atlas' data source

Sample Query

The query here is

db.sensor_value.aggregate ( [ 
{ "$match" :    {   "sensor_type" : "$sensor",   "host_name" : "$host",   "ts" : { "$gte" : "$from", "$lte" : "$to" } }  },        
 {"$sort" : {"ts" : 1}},            
 {"$project" :   {  "name" : "value",   "value" : "$sensor_value",  "ts" : "$ts",  "_id" : 0} } ])

The API is expecting back documents with the following fields

  • name - Name of the series ( will be displayed on the graph)
  • value - The float value of the point
  • ts - The time of the point as a BSON date

These documents are then converted into the Grafana API

$from and $to are expanded by the plugin as BSON dates based on the range settings on the UI.

Template Variables

$sensor and $host are template variables that are filled in by Grafana based on the drop down. The sample template queries are shown below. They expect documents to be returned with a single _id field.

Sample Templates

Example 2 - Using $bucketAuto to push data point aggregation to the server

Grafana tells the backend server the date range along with the size of the buckets that should be used to calculate points. Therefore it's possible to use the MongoDB aggregation operator $bucketAuto to automatically bucket the data points into display points. To support this the backend provides the $dateBucketCount macro so that queries such as the one below can be written

db.sensor_value.aggregate( [ 
{  "$match" :  {  "sensor_type" : "$sensor", "host_name" : "$host" , "ts" : { "$gte" : "$from", "$lt" : "$to" }}},
{  "$bucketAuto" :  { "groupBy" : "$ts",  
                           "buckets" : "$dateBucketCount", 
                            "output" :  {  "maxValue" : { "$max" : "$sensor_value" }  }   }   },  
{  "$project" :  {  "name" : "value",  "value" : "$maxValue",  "ts" : "$_id.min",  "_id" : 0  }  }  ]  )

Note that _id field of the bucketAuto output contains the start and end of the bucket so we can use that as the ts value

The dashboard in examples\RPI MongoDB Bucket - Atlas.json shows this.

Example 3 - Using a Tabel Panel

Table Panel

Table panels are now supported with queries of the form

db.sensor_value.aggregate(
[
    {  "$match" :  {  "ts" : { "$gte" : "$from", "$lt" : "$to"  }}},
    {  "$group":    {  "_id":  { "sensor_name" : "$sensor_name",  "sensor_type" : "$sensor_type"  }, "cnt" : { "$sum" : 1 },  "ts" : { "$max" : "$ts" }  } }, 
    { "$project": {  "name" : { "$concat" : ["$_id.sensor_name",":","$_id.sensor_type" ]},  "value" : "$cnt",  "ts" : 1, "_id" : 0} } 
])

The dashboard in examples\Sensor Values Count - Atlas.json shows this.

Running the proxy as a service on a Mac

  • Install forever-mac
  • Copy server/mongodb-grafana-proxy.plist to ~/Library/LaunchAgents
  • run launchctl load mongodb-grafana-proxy from ~/Library/LaunchAgents

This launch ctrl plist runs the node script via forever. To check it's running, use forever list. Logs go into /usr/local/var/lib/grafana/plugins/mongodb-grafana/dist/server

Packing the plugin

  1. Run
    npm run build
  2. Create a ZIP archive of the dist directory.
    mv dist/ grafana-mongodb-datasource
    zip grafana-mongodb-datasource-<version>.zip grafana-mongodb-datasource -r
  3. Upload to artifactory

Pacing the proxy server

Build and Push the image

docker build -t docker-internal.artifactory.igentify.net/grafana-mongodb-datasource:<version> -f Docker/Dockerfile .
docker push docker-internal.artifactory.igentify.net/grafana-mongodb-datasource:<version>

Deploying the Proxy Service

Parameters:

  • API_TOKEN - token to access the service using the x-api-key header

mongodb-grafana's People

Contributors

ekirmayer avatar jamesosgood avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.