Sharness

Sharness is a portable shell library to write, run, and analyze automated tests for Unix programs. Since all tests output TAP, the Test Anything Protocol, they can be run with any TAP harness.

Each test is written as a shell script, for example:

#!/bin/sh

test_description="Show basic features of Sharness"

. ./sharness.sh

test_expect_success "Success is reported like this" "
    echo hello world | grep hello
"

test_expect_success "Commands are chained this way" "
    test x = 'x' &&
    test 2 -gt 1 &&
    echo success
"

return_42() {
    echo "Will return soon"
    return 42
}

test_expect_success "You can test for a specific exit code" "
    test_expect_code 42 return_42
"

test_expect_failure "We expect this to fail" "
    test 1 = 2
"

test_done

Running the above test script returns the following (TAP) output:

$ ./simple.t
ok 1 - Success is reported like this
ok 2 - Commands are chained this way
ok 3 - You can test for a specific exit code
not ok 4 - We expect this to fail # TODO known breakage
# still have 1 known breakage(s)
# passed all remaining 3 test(s)
1..4

Alternatively, you can run the test through prove(1):

$ prove simple.t
simple.t .. ok
All tests successful.
Files=1, Tests=4,  0 wallclock secs ( 0.02 usr +  0.00 sys =  0.02 CPU)
Result: PASS

Sharness was derived from the Git project - see README.git for the original documentation.

Installation

First, clone the Git repository:

$ git clone git://github.com/chriscool/sharness.git

Then choose an installation method that works best for you:

Per-project installation

If you like to add Sharness to the sources of a project you want to use it for, simply copy the files sharness.sh, aggregate-results.sh, and test/Makefile to a folder named test inside that project, and then set SHARNESS_TEST_SRCDIR to this folder somewhere, export it, and source $SHARNESS_TEST_SRCDIR/sharness.sh in your test files.

See for example how setting SHARNESS_TEST_SRCDIR is done in test/simple.t and in the install target of the Makefile.

The requirement to set SHARNESS_TEST_SRCDIR is new in current master. It used to be possible to only copy files and source sharness.sh, but chriscool#90 changed that.

Another way is to use Sharnessify.

Alternatively, you can also add Sharness as a Git submodule to your project.

In per-project installation, Sharness will optionally load extensions from sharness.d/*.sh if a sharness.d directory is found in the same directory as sharness.sh. This allows per-project extensions and enhancements to be added to the test library without requiring modification of sharness.sh.

Per-user installation

$ cd sharness
$ make install

This will install Sharness to $HOME/share/sharness, and its documentation and examples to $HOME/share/doc/sharness.

System-wide installation

$ cd sharness
# make install prefix=/usr/local

This will install Sharness to /usr/local/share/sharness, and its documentation and examples to /usr/local/share/doc/sharness.

Of course, you can change the prefix parameter to install Sharness to any other location.

Installation via Chef

If you want to install Sharness with Opscode Chef, the Sharness cookbook is for you.

Usage

The following files are essential to using Sharness:

• sharness.sh - core shell library providing test functionality, see separate API documentation. Meant to be sourced from test scripts, but not executed.
• aggregate-results.sh - helper script to aggregate test results
• test/Makefile - test driver. The default target runs the complete testsuite.

To learn how to write and run actual test scripts based on sharness.sh, please read README.git until I come up with more documentation myself.

Command-line options

The *.t test scripts have the following options (again, read README.git for details) :

• --debug, -d: helps debugging
• --immediate, -i: stop execution after the first failing test
• --long-tests, -l: run tests marked with prereq EXPENSIVE
• --interactive-tests: run tests marked with prereq INTERACTIVE
• --help, -h: show test description
• --verbose, -v: show additional debug output
• --quiet, -q: show less output
• --chain-lint/--no-chain-lint: check &&-chains in scripts
• --no-color: don't color the output
• --tee: also write output to a file
• --verbose-log: write output to a file, but not on stdout
• --root=<dir>: create trash directories in <dir> instead of current directory.

Projects using Sharness

See how Sharness is used in real-world projects:

• azuki
• cb2util
• dabba
• git-integration
• git-multimail
• git-spindle
• git-svn-fast-import
• go-ipfs
• go-multihash
• inotify-tools
• ipfs-update
• rdd.py
• Sharness itself
• tomdoc.sh

Furthermore, as Sharness was derived from Git, Git's test suite is worth examining as well, especially if you're interested in managing a big number of tests.

Alternatives

Here is a list of other shell testing libraries (sorted alphabetically):

• Bashtub
• Bats
• Cram
• rnt
• roundup
• shove
• shUnit2
• shspec
• testlib.sh
• ts

License

Sharness is licensed under the terms of the GNU General Public License version 2 or higher. See file COPYING for full license text.

Contributing

Contributions are welcome, see file CONTRIBUTING for details.

Authors

Sharness was created in April 2011 and maintained until June 2016 by Mathias Lafeldt. The library is derived from the Git project's test-lib.sh. It is currently maintained by Christian Couder, thanks to sponsorship from Protocol Labs.

See Github's "contributors" page for a list of developers.

A complete list of authors should include Git contributors to test-lib.sh too.