The gelf-transformer from makeros

GELF Transformer (ver. 2)

This project is based on Pino Gelf

GELF transformer is a tool which receives json formatted logs from the stdin and transforms them into GELF format GELF It can also use a custom mapping schema to fill the output log with more data.

Installation
Usage
Getting Started
Examples
GELF
Log Level Mapping
Acknowledgements

Installation

npm i -g gelf-transformer

Usage

Pipeline approach

If your application is pushing logs to the standard output then pipe them to gelf transformer.

node your-app.js | gelf-transformer log <options>

Getting Started

command `log`

gelf-transformer log --help

Switch	Description	Default	Notes
`-h`	Host	`127.0.0.1`	Graylog server host
`-p`	Port	`12201`	Graylog server port
`-m`	Maximum Chunk Size	`1420`
`-c`	Custom schema	`false`	You can provide a schema which will define which information from your original logs will be visible in the graylog formatted log
`-v`	Verbose mode	`false`	Output GELF to console
`-t`	Start sending logs to Graylog	`false`	It will start to send logs to the defined graylog server

Examples

Custom Fields

Given the log message (formatted as JSON for readability):

{
  "pid":16699,
  "hostname":"han",
  "name":"gelf-test-app",
  "level":30,
  "time":1481840140708,
  "msg":"request completed",
  "customField":"test",
  "res":{"statusCode":304},
  "responseTime":8,
  "req":{
    "method":"GET",
    "headers":{
      "host":"localhost:3000",
      "user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/602.2.14 (KHTML, like Gecko) Version/10.0.1 Safari/602.2.14"}
    },
  "v":1
}

Given custom schema json file (my_custom_schema.json):

{
  "title": "GELF Schema",
  "type": "object",
  "properties": {
    "_status_code": {
      "type": "integer",
      "source": "res.statusCode"
    },
    "_user_agent": {
      "type": "string",
      "source": "req.headers.user-agent"
    },
    "customField": {
      "type": "string"
    }
  }
}

And the usage:

node server.js | gelf-transformer log -v -c my_custom_schema.json

Gelf Transformer will show the following message to your Graylog server (formatted here as JSON for readability):

{
  "version":"1.1",
  "host":"han",
  "short_message":"request completed",
  "full_message":"request completed",
  "timestamp":1481840140.708,
  "level":6,
  "facility":"gelf-test-app",
  "_status_code":304,
  "_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/602.2.14 (KHTML, like Gecko) Version/10.0.1 Safari/602.2.14",
  "customField":"test"
}

GELF

Right now automatic mapping of fields is done as follows:

Output GELF	Input log	Notes
`version`	`-`	Hardcoded to 1.1 per GELF docs
`host`	`hostname`
`short_message`	`msg`	This message is truncated to 64 characters
`full_message`	`msg`	msg is not truncated
`timestamp`	`time`
`level`	`level`	Default level codes from Pino are mapped to SysLog levels¹
`facility`	`name`	deprecated

Log Level Mapping

Default behaviour

By default Gelf Transfomer will log level from a Pino format to syslog format:

Pino Log Level Value	Pino Log Level Name	SysLog Level
10	Trace	Debug
20	Debug	Debug
30	Info	Info
40	Warn	Warning
50	Error	Error
60	Fatal	Critical

Note: A log messages without a level map to SysLog Critical

Override log level from Schema

TBD

Acknowledgements

The implementation of Pino GELF is based in large part on pino-syslog and gelf-node.

makeros / gelf-transformer Goto Github PK

gelf-transformer's Introduction

GELF Transformer (ver. 2)

Contents

Installation

Usage

Pipeline approach

Getting Started

command log

Examples

Custom Fields

GELF

Log Level Mapping

Default behaviour

Override log level from Schema

Acknowledgements

gelf-transformer's People

Contributors

Stargazers

Watchers

gelf-transformer's Issues

Recommend Projects

Recommend Topics

Recommend Org

command `log`