Giter Club home page Giter Club logo

infinite-file-curtailer's Introduction

Infinite-File-Curtailer - a program that reads stdin and writes to a fixed size file.

Requires Linux 3.15+ and a filesystem that supports the FALLOC_FL_COLLAPSE_RANGE flag for fallocate. This is currently ext4 and XFS. Hopefully more in the future...


Usage

./curtail [-s size] <output file>

Options:

-verbose  enable all debug output (on stderr)
-quiet    disable all non-error output (on stderr)
-size     Maximum size of the output file (ie. 8192, 64K, 10M, 1G, etc) - default is 16K

Example

A typical usage scenario is to capture the output of a program in a file. Using curtail prevents the program from creating a runaway file that will eventually fill up the filesystem and cause system failure if not handled at the system level. In the example below, the file my_app_log.txt cannot exceed 2 megabytes in size.

./my_app | curtail -s 2M ./my_app_log.txt

Build instructions

Curtail uses autotools (must be installed on the local system). If not already installed, install the tools using the following commands with the appropriate package manager (apt, yum, etc) for your system:

sudo apt install libtool
sudo apt install automake
sudo apt install autoconf

To build curtail locally, please run the following commands from the base dir of the project:

libtoolize
aclocal
autoheader
autoconf
automake --add-missing
./configure
make

To install in the local system, run the following additional command with appropriate priviledges:

sudo make install

Library usage

Curtail can also be integrated directly into an application instead of used on the command line. Include the file curtail.h and link the application with -lcurtail. After successfully calling crtl_init, the program's stdout will be directed to the specified file until crtl_term is called.

Logrotate comparision

Curtail is not intended to be a replacement for logrotate. They are fundamentally different. Some of the notable differences are below:

logrotate mitigates runaway files... curtail prevents them.
logrotate requires sysadmin... curtail does not. logrotate creates multiple files... curtail only one.

infinite-file-curtailer's People

Contributors

dwolaver avatar kristakhare avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

infinite-file-curtailer's Issues

Block size

Add an option to retrieve the block size for a file and return an error if the filesystem type is not supported

Truncation size

Add an option to set the truncation size as a multiple of the block size

Provide compiled binaries or detailed build instructions

I found this through a post on Stack Exchange, the premise looks amazing, but I'm showing various errors when compiling the code. Here's the error:

crtl_private.h:43:94: error: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 4 has type ‘uint64_t {aka long unsigned int}’ [-Werror=format=]

Given my limited knowledge on how to compile, and even less knowledge on what the code actually does, I would greatly appreciate if you can just provide a binary that we can just use.

Option to limit data rate

Add a command line option to allow the user to set a maximum data rate over a specified period of time. This would serve two purposes:

  1. Prevent unnecessary disk writes in case of runaway output
  2. Preserve data prior to runaway event that may be useful to diagnose the condition

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.