Giter Club home page Giter Club logo

logrus_influxdb's Introduction

InfluxDB Hook for Logrus

Feel free to create an issue or send me a pull request if you have any questions, bugs, or suggestions for this library.

Thank you for creating issues and pull requests!

Usage

import (
  "time"
  "github.com/Sirupsen/logrus"
  "github.com/Abramovic/logrus_influxdb"
)
func main() {
  log    := logrus.New()

  config := &logrus_influxdb.Config{
    Host: "localhost",
    Port: 8086,
    Database: "logrus",
    UseHTTPS: false,
    Precision: "ns",
    Tags: []string{"tag1", "tag2"},
    BatchInterval: (5 * time.Second),
    BatchCount: 200, // set to "0" to disable batching
  }

  /*
    Use nil if you want to use the default configurations

    hook, err := logrus_influxdb.NewInfluxDB(nil)
  */

  hook, err := logrus_influxdb.NewInfluxDB(config)
  if err == nil {
    log.Hooks.Add(hook)
  }  
}

With an existing InfluxDB Client

If you wish to initialize a InfluxDB Hook with an already initialized InfluxDB client, you can use the NewWithClientInfluxDBHook constructor:

import (
	"github.com/Abramovic/logrus_influxdb"
	"github.com/Sirupsen/logrus"
	client "github.com/influxdata/influxdb/client/v2"
)

func main() {
	log := logrus.New()

	// In this example we will use the default configurations
	config := &logrus_influxdb.Config{
		Tags: []string{"tag1", "tag2"}, // use the following tags
	}

	// Connect to InfluxDB using the standard client.
	influxClient, _ := client.NewHTTPClient(client.HTTPConfig{
		Addr: "http://localhost:8086",
	})

	hook, err := logrus_influxdb.NewInfluxDB(config, influxClient)
	if err == nil {
		log.Hooks.Add(hook)
	}
}

In syslog format for chronograf log viewer

If you wish to push your logs in syslog format so you can view all logs in the chronograf log viewer.

import (
  "time"
  "github.com/Sirupsen/logrus"
  "github.com/Abramovic/logrus_influxdb"
)

func main() {
  log    := logrus.New()

  config := &logrus_influxdb.Config{
    Host: "localhost",
    Port: 8086,
    Database: "syslog", // set to syslog to view in logviewer
    UseHTTPS: false,
    Precision: "ns",
    Tags: []string{"tag1", "tag2"},
    BatchInterval: (5 * time.Second),
    BatchCount: 200, // set to "0" to disable batching
    Syslog:        true, // enable syslog format or not
    Facility:      "user", // see https://en.wikipedia.org/wiki/Syslog#Facility
    FacilityCode:  1, // see https://en.wikipedia.org/wiki/Syslog#Facility
    AppName:       "cb-scheduler", // app_name to use
    Version:       "1.0", // version of app
  }

  /*
    Use nil if you want to use the default configurations

    hook, err := logrus_influxdb.NewInfluxDB(nil)
  */

  hook, err := logrus_influxdb.NewInfluxDB(config)
  if err == nil {
    log.Hooks.Add(hook)
  }  
}

Behind the scenes

Database Handling

When passing an empty string for the InfluxDB database name, we default to "logrus" as the database name.

When initializing the hook we attempt to first see if the database exists. If not, then we try to create it for your automagically.

Message Field

We will insert your message into InfluxDB with the field message so please make sure not to use that name with your Logrus fields or else it will be overwritten.

Special Fields

Some logrus fields have a special meaning in this hook, these are server_name, logger and http_request (taken from Sentry Hook).

  • server_name (also known as hostname) is the name of the server which is logging the event (hostname.example.com)
  • logger is the part of the application which is logging the event
  • http_request is the in-coming request (*http.Request)

logrus_influxdb's People

Contributors

abramovic avatar davidschrooten avatar franklinkim avatar heavyhorst avatar mavidser avatar rahul-208 avatar tekkamanendless avatar vincentserpoul avatar vlad-doru avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

logrus_influxdb's Issues

New Trace log level in logrus

Hello, I'm just opening this issue to let you know, the next release of logrus (v1.2.0) will add a new trace level named Trace below Debug.
Here is the PR for reference sirupsen/logrus#844
You may want to take into account this new level.

Any syntactically bad log statement will fail every other log message forever

I plan to submit a patch to address this tomorrow, but I wanted to report my findings today.

Short version of the story: the log buffer (batchP) is only cleared upon a successful Write to InfluxDB. This means that if a single bad statement is written, then the buffer will continue to fill and never be cleared, causing every log statement to attempt to send an ever-increasing buffer of logs to InfluxDB.

This can happen for silly, simple reasons:

  1. Someone logged WithField X of type float and you try to log WithField X of type integer. That will always fail until the InfluxDB database is cleared.
  2. You accidentally log a uint64 value, and the InfluxDB client supports encoding this as XXXu but the InfluxDB server does not (as of version 1.5.3) support parsing this value.

In both of these cases, the application that is logging will gradually slow down and die.

My proposed fix is to have writePoints always clear the buffer (batchP) and update lastBatchUpdate, and then just return an error if there happened to be one when writing.

Import path should be "github.com/sirupsen/logrus", not "github.com/Sirupsen/logrus"

Background

I find that github.com/sirupsen/logrus and github.com/Sirupsen/logrus coexist in this repo:
Gopkg.toml

[[constraint]]
  name = "github.com/Sirupsen/logrus"
  version = "1.4.2"

[[constraint]]
  branch = "master"
  name = "github.com/influxdata/influxdb1-client"

[[constraint]]
  name = "github.com/sirupsen/logrus"
  version = "1.4.2"

Some places used github.com/Sirupsen/logrus. Some places used github.com/sirupsen/logrus. But github.com/sirupsen/logrus and github.com/Sirupsen/logrus are the same repo.
As README of logrus said, github.com/Sirupsen/logrus is the old path, anything using it should be updated to import and require github.com/sirupsen/logrus.

Some environments experienced problems with the upper-case variant, so the lower-case was decided. Everything using logrus will need to use the lower-case: github.com/sirupsen/logrus. Any package that isn't, should be changed.

Solution

Replace all the import paths, change "github.com/Sirupsen/logrus" to "github.com/sirupsen/logrus".
Where did you import it: https://github.com/abramovic/logrus_influxdb/search?q=github.com%2FSirupsen%2Flogrus&unscoped_q=github.com%2FSirupsen%2Flogrus

New location of an InfluxDB client is required

Cannot run the hook when using InfluxDB 0.10.0, throws exception

Need to provide a new client import location: "github.com/influxdata/influxdb/client/v2"
instead of the current "github.com/influxdb/influxdb/client/v2"

time seems invalid in influxdb

Hi,

First of all, thanks for the module, good job 👍 .
But I have an issue I'm not sure why.
It actually writes to the influxdb but the time in influxdb indicates 1970, like it was close to timestamp 0.
Have you seen that behaviour before?

Tags are also created as fields

Apparently the configured tags are stored as tags and fields in InfluxDB.
Is this the intended behavior? I wouldn't expect them to be stored as fields. Or am I missing something?

config := &logrus_influxdb.Config{
	Tags:       []string{"service", "action"},
}

in InfluxDB:

> use logrus
Using database logrus

> show tag keys
name: logrus
tagKey
------
action
level
service

> show field keys
name: logrus
fieldKey fieldType
-------- ---------
action   string
err      string
message  string
service  string

> select * from logrus
name: logrus
time action action_1 err level message service service_1
---- ------ -------- --- ----- ------- ------- ---------
[...]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.