Giter Club home page Giter Club logo

btdu's Introduction

btdu - sampling disk usage profiler for btrfs

Some btrfs features may make it difficult to estimate what disk space is being used for:

  • Subvolumes allow cheap copy-on-write snapshots of entire filesystem trees, with data that hasn't been modified being shared among snapshots
  • File and extent cloning allow creating cheap copies of files or parts thereof, with extents being stored only once
  • Compression transparently allows further reducing disk usage

For these reasons, classic disk usage analyzers such as ncdu cannot provide an accurate depiction of actual disk usage. (btrfs compression in particular is challenging to classic analyzers, and special tools must be used to query compressed usage.)

btdu is a sampling disk usage profiler for btrfs. It works according to the following algorithm:

  1. Pick a random point on the disk in use
  2. Find what is located at that point
  3. Add the path to the results
  4. Repeat the above steps indefinitely

Though it works by taking random samples, it is "eventually" accurate.

It differs from classic analyzers through the following properties:

  • btdu starts showing results instantly. Though wildly inaccurate at first, they become progressively more accurate the longer btdu is allowed to run.
  • btdu analyzes entire filesystems only. There is no way to analyze only a particular subdirectory or subvolume.
  • btdu counts extents used by multiple files only once. (The shortest path is used when placing the sample in the tree for visualization.)
  • By nature of its algorithm, btdu works correctly with compression and other btrfs filesystem features.
  • Because it queries raw filesystem metadata, btdu requires root privileges to run.

Use cases

  • Quickly summarize space usage

    btdu needs to collect only 100 samples to achieve a ~1% resolution, which means it can identify space hogs very quickly. This is useful if the disk is full and some space must be freed ASAP to get things back up and running.

  • Estimate snapshot size

    When an extent is in use by multiple files or snapshots, to decide where to place it in the browsable tree, btdu picks the path with the shortest length, or the lexicographically smaller path if the length is the same. An emergent effect of this property is that it can be used to estimate snapshot size, if your snapshots use a fixed-length lexicographically-ordered naming scheme (such as e.g. YYYY-MM-DD-HH-MM-SS): the size of snapshots displayed in btdu will thus indicate data that occurs in that snapshot or any later one, i.e. the amount of "new" data in that snapshot.

  • Estimate compressed data size

    If you use btrfs data compression (whether to save space / improve performance / conserve flash writes), btdu can be used to estimate how much real disk space compressed data uses.

  • Estimate unreachable extent size

    A feature unique to btdu is the ability to estimate the amount of space used by unreachable parts of extents, i.e. data in extents containing older versions of file content which has since been overwritten. This btrfs "dark matter" can be an easy to overlook space hog, which could be eliminated by rewriting or defragmentating affected files.

Installation

btdu can be installed in one of the following ways:

Building

  • Install a D compiler
  • Install Dub, if it wasn't included with your D compiler
  • Run dub build -b release

Usage

# btdu /path/to/filesystem/root

Note that the indicated path must be to the root subvolume (otherwise btdu will be unable to open other subvolumes for inode resolution). If in doubt, mount the filesystem to a new mountpoint with -o subvol=/,subvolid=5.

Run btdu --help for more usage information.

License

btdu is available under the GNU GPL v2. (The license is inherited from btrfs-progs.)

See Also

  • btsdu, the Btrfs Snapshot Disk Usage Analyzer

btdu's People

Contributors

cybershadow avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.