Giter Club home page Giter Club logo

logro's Introduction

Logro

Deprecated

Almost all high performance log library use sync.Pool to reuse memory, so it may cause problem when just pass a pointer to logro.

Introduction

Logro is a non-blocking log rolling package with page cache control in Go. Inspired by lumberjack

Logro is built for high performance (The latency of per write is about 50 ns/op):

  • Non-blocking Write

    All write won't be blocked (Just pass a pointer then return). You will never have to worry about the log write stall impacting P999.

    When the IOPS is unusual high (may caused by bugs or unexpected behavior, e.g. 10 million/s), Logro will overwrite data on writes in lieu of blocking.

  • Write Combination

    All log data will be written to a user-space buffer first, then flush to the log file.

  • Sync in background

    Use sync in background avoiding write stall in user-facing.

    ps: OS can do the page cache flush by itself, but it may create a burst of write I/O when dirty pages hit a threshold.

  • Clean page cache

    It's meaningless to keep log files' data in page cache, so when the dirty pages are too many or we need reopen a new file, Logro will sync data to disk, then drop the page cache.

  • ...

Methods

  • WriteSyncCloser

    Logro implements such methods:

        Write(p []byte) (written int, err error)
        Sync() (err error)
        Close() (err error)
    

    Could satisfy most of log packages.

Rotation

e.g. The log file's name is a.log, the log files will be:

    a.log
    a-time.log
    a-time.log
    a-time.log
    a-time.log
    ....

Log shippers such as ELK's filebeat can set path to:

    a.log    

Control

Logro control rotation by file size only, it's simple and enough for the most cases. (Now we usually use log shippers to collect logs to databases, but not login machines and grep data)

Example

Stdlib Logger

    r, _ := New(&conf)
    log.New(r, "", log.Ldate)

Zap Logger

    r, _ := New(&conf)
    zapcore.AddSync(r)
    ...

Acknowledgments

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.