Giter Club home page Giter Club logo

tail-follow's Introduction

tail-follow

Stream a file in real time as it grows.

Synopsis

If you are familiar with UNIX's tail -f, you have a good idea what this module does. It provides a Readable stream interface to a file that emits new data appended to the file in real time. It can even do the equivalent of tail -F, to survive log rotation.

There are a lot of tail modules on npm, but this one differentiates itself by:

  • Being a Readable stream, so your data event listeners get Buffer objects instead of decoded strings split by lines.
  • Providing an API for retrieving positional data about where in file a chunk of data was read from.
  • Having a knob for controlling how much data to try to read from the file at a time, since the default of 16KB is too small for rapidly growing files.
  • Emitting a rename event when the underlying file is renamed or rotated.

API

var TailFollow = require("tail-follow")

Constructor TailFollow(path, [options])

Create new TailFollow instance, for the file at the given path. This is a Readable stream, so you should listen for the error event and data event. Or pipe() it to another stream.

The optional arguments object takes the same parameters as a Readable stream (encoding and highWaterMark), plus the following:

  • tailChunkSize: Sets the size in bytes of Buffer objects created when reading from the file. Too small of a value, you will spend too much CPU handling new chunks; too big of a value, and you will waste memory. The default is 16384.
  • surviveRotation: Set to true so that if the underlying file is renamed or deleted, data will continue to be read from a new file created at the same path. Think tail -F. The default is false.
  • objectMode: Setting to true enables object mode like on any other Readable stream, but has a few side effects. It enables position tracking so that you may call .positionForChunk() to determine where in the file a chunk of data was read from. If you have enabled decoding (via the encoding option), note that this will also cause your data events to emit String instances (as opposed to string primitives).
  • follow: Default true. Setting to false will cause the stream to close when it reaches the end of the file instead of continuing to follow it.
  • fileRenamePollingInterval: Default is null. Setting this to a number will cause TailFollow to poll for file rename events every given number of milliseconds instead of relying on fs.watch() rename events. Try this if your tail sometimes stops after a rotation.

Event rename

Emitted when the underlying file has been renamed. The first argument sent to the event handler is the old file path, the second is the new file path. Currently, this is limited to moves within the same directory, or within sibling directories of the file.

Method unfollow()

Stops following the file and ends the stream.

Method positionForChunk(chunk)

Returns the number of bytes from the beginning of the file that the chunk starts at. This must be the same object that was emitted to your data event handler. The stream must also be in objectMode. (See the documentation for the objectMode constructor option.)

Method setTailChunkSize(size)

See the documentation for the tailChunkSize constructor option above.

Method setSurviveRotation(bool)

See the documentation for the surviveRotation constructor option above.

Method setFileRenamePollingInterval(milliseconds)

See the documentation for the fileRenamePollingInterval constructor option above.

Property filePath

Convenience property to get the full path of the file that the instance is following.

Debugging

Run node with DEBUG=tail-follow.

Contributing

This module uses ES2015. To compile the source to ES5 (compatible with Node.js, io.js), run npm run compile.

Any changes in behavior need test coverage. To run the tests, run npm test.

tail-follow's People

Contributors

limulus avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

digiwano

tail-follow's Issues

Crash during file rotation

This likely means that we are sending null as the file descriptor to fs.close(). Chances are this means that _unfollow() was called during file rotation.

fs.js:465
  binding.close(fd, req);
          ^
TypeError: Bad argument
    at TypeError (native)
    at Object.fs.close (fs.js:465:11)
    at TailFollow._unfollow (/…/node_modules/live-index/node_modules/tail-follow/dist/es5/lib/TailFollow.js:322:23)
    at unfollowOldFileAndTailFromNewFile (/…/node_modules/live-index/node_modules/tail-follow/dist/es5/lib/TailFollow.js:224:20)
    at /…/node_modules/live-index/node_modules/tail-follow/dist/es5/lib/TailFollow.js:233:13
    at onList (/…/node_modules/live-index/node_modules/tail-follow/node_modules/fs-find/lib/index.js:186:5)
    at onCheck (/…/node_modules/live-index/node_modules/tail-follow/node_modules/fs-find/lib/index.js:163:16)
    at /…/node_modules/live-index/node_modules/tail-follow/node_modules/fs-find/lib/index.js:136:18
    at onCheck (/…/node_modules/live-index/node_modules/tail-follow/node_modules/fs-find/lib/index.js:163:16)
    at onStat (/…/node_modules/live-index/node_modules/tail-follow/node_modules/fs-find/lib/index.js:144:18)

Fall back to polling for rename events

It seems that sometimes rename events are not sent to the FSWatcher. (Seen on FreeBSD 10.2-PRERELEASE #1, potentially after a large number of changes to the file.) See if there is a way to fall back to polling for rename events.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.