side by side diff tool with colorization and regular expression support for filtering

1. Introduction
2. Using It
3. Colors
4. Command Line Options
5. Installation
6. Modification
7. About
8. TODO

Command line tool that does a side by side diff of two text files with regular expression filtering and ANSI terminal colorization.

It is useful for analyzing text files that have patterns like timestamps that can easily be filtered out.

The tool is very similar to sdiff, wdiff or diff. You specify two files or substitutions and it outputs the differences between them using ANSI terminal colorization to highlight the differences.

You can specify replacement patterns to mask or filter differences that you don't care about, like line numbers or time stamps. It is very similar to using process substitution in sdiff but the regular expression patterns are defined by go. Doing it with sdiff would look something like this (with no colorization).

$ sdiff \
    <(cat test/td03.txt | sed -E -e 's@[0-9][0-9]:[0-9][0-9]:[0-9][0-9]@HH:MM:SS@g') \
    <(cat test/td04.txt | sed -E -e 's@[0-9][0-9]:[0-9][0-9]:[0-9][0-9]@HH:MM:SS@g' )

The default colorization is very simple, a light grey background is used to highlight differences. A background color was chosen so that it would be easy to see extra spaces. You can customize the colors.

Here is an example that shows two files whose content only differ by a time stamp.

$ bin/Darwin-x86_64/csdiff test/td03.txt test/td04.txt

You can see the differences in the vertical, light grey stripes.

Here is an example that shows how to use the replacement option (-r) to ignore the timestamp differences. As you can see, the go regular expressions are a bit more concise than the previous sed example.

$ bin/Darwin-x86_64/csdiff \
    -r '\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}' 'yyyy-mm-dd HH:MM:SS' \
    test/td03.txt test/td04.txt

Now there are no differences because they were masked.

You have the option of customizing the output colors based on the type of data by specifying colormaps for different types of output.

Here is an example of how that is done.

$ bin/Darwin-x86_64/csdiff -c cd=bold,fgRed \
                           -c cm=bold,fgBlue \
                           -c s=bold,fgMagenta \
                           -c lm=bold,fgGreen \
                           -c llo=bold,fgCyan \
                           -c rlo=bold,fgCyan \
                           test/td01.txt test/td02.txt

Contrast that with the default below and you can see the differences.

$ bin/Darwin-x86_64/csdiff test/td01.txt test/td02.txt

As you can see, there are many more colors in the first example.

Color is specified by defining a color map for colorizable entities. The syntax looks like this.

   -c cd=bold,fgRed
   ^  ^ ^^    ^
   |  | ||    +--- Another colorization value or text modifier, you can specify as many as you like,
   |  | |+-------- A colorization value or text modifier (described below).
   |  | +--------- Equals sign that separates the tag from the attributes.
   |  +----------- Tag (described below)
   +-------------- Option that specifies a color map.

Note that you can have multiple color maps on the same line by separating them with a semi-colon as show below.

   -c 'cd=bold,fgRed;cm=bold,fgBlue'

These tags describe the data that can be colored.

These are the symbols that csdiff inserts between the lines. They cannot be changed.

These are the values that you can specify to color text. It is a subset of the ANSI terminal colors that should work everywhere. They are not case sensitive.

1. fgDefault
2. fgBlack
3. fgBlue
4. fgCyan
5. fgGreen
6. fgMagenta
7. fgRed
8. fgYellow
9. fgWhite
10. fgLightBlue
11. fgLightCyan
12. fgLightGreen
13. fgLightGrey
14. fgLightMagenta
15. fgLightRed
16. fgLightYellow
17. fgDarkGrey

These are the values that you can specify for the background. It is a subset of the ANSI terminal colors that should work everywhere. They are not case sensitive.

1. bgDefault
2. bgBlack
3. bgBlue
4. bgCyan
5. bgGreen
6. bgMagenta
7. bgRed
8. bgYellow
9. bgLightBlue
10. bgLightCyan
11. bgLightGreen
12. bgLightGrey
13. bgLightMagenta
14. bgLightRed
15. bgLightYellow
16. bgDarkGrey

These are the values that you can specify to change how the text is displayed. They are not case sensitive.

You can specify a much wider range of colors using the ANSI 256 color mode colors using: fg256[N] and bg256[N] where N is a number between in the range [0..255]. To see all of the colors available use the --256 option.

$ csdiff --256

The fg256[N] and bg256[N] color options are only available in version 0.5.x or later.

The command line options are best accessed by looking at the inline help because they may change over time but these are the command line options available in 0.4.x.

Just download the tar or zip image for your system and extract the executable. You can use it directly or copy it.

$ curl -s -k -L -O https://github.com/jlinoff/csdiff/releases/download/v0.5.0/csdiff-linux_amd64.tar.gz
$ tar ztvf csdiff-linux_amd64.tar.gz 
-rwxr-xr-x  1 jlinoff 1784424920 2331734 Jun  9 13:52 csdiff

$ curl -s -k -L -O https://github.com/jlinoff/csdiff/releases/download/v0.5.0/csdiff-darwin_amd64.zip
$ unzip -l csdiff-darwin_amd64.zip
Archive:  rel/csdiff-darwin_amd64.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
  2317280  06-09-2017 13:52   csdiff
---------                     -------
  2317280                     1 file

Here is how you check out the source to modify it. If you make changes or implement bug fixes that you think might be useful, please let me know so that I can incorporate them.

$ git clone https://github.com/jlinoff/csdiff.git

Note that I used the following Dockerfile to create the linux image because I was working on a Mac and wanted to be able to test on linux. To simply build on linux, you can use the built-in go capability for cross compilation using GOOS and GOARCH.

# This docker file creates a go compilation container that can be used
# to cross-compile go for linux on any platform that supports docker.
#
#   $ cd csdiff
#   $ docker run -it --rm -v $(pwd):/opt/go/project goco make
#
# To create it. Note that gover is an ARG that defines the version of
# go that you want to build. This example shows how to build it for go-1.8.3.
#
#   $ docker build --build-arg gover=1.8.3 -f Dockerfile -t goco:1.8.3 -t goco:latest .
FROM centos:latest

RUN yum clean all && yum update -y && yum install -y git make

ARG gover
ENV GO_VERSION=$gover
ENV GOROOT=/opt/go/latest
ENV GOPATH=/opt/go/project
ENV GO_PROG=/opt/go/latest/bin/go

# Setup the volume.
RUN mkdir -p ${GOPATH}
VOLUME ${GOPATH}

# Install go in /opt/go
RUN mkdir -p /opt/go/${GO_VERSION}/dl && \
    cd /opt/go/${GO_VERSION}/dl && \
    curl -k -O -L https://storage.googleapis.com/golang/go${GO_VERSION}.linux-amd64.tar.gz && \
    cd /opt/go/${GO_VERSION} && \
    tar zxf dl/go${GO_VERSION}.linux-amd64.tar.gz && \
    ln -s /opt/go/${GO_VERSION}/go /opt/go/latest && \
    ${GO_PROG} version
    
# Install golint
RUN cd /opt/go/${GO_VERSION}/dl && \
    GOPATH=/opt/go/${GO_VERSION}/dl ${GOROOT}/bin/go get -u github.com/golang/lint/golint && \
    cp bin/* ${GOROOT}/bin

# Wrapper for the go command that makes it
# natural for the user to run something like
# docker run -it --rm -v $(pwd):/opt/go/project goco go build myprog.go
RUN /bin/echo '#!/bin/bash'                           > /opt/go/goco.sh && \
    /bin/echo 'export PATH="${GOROOT}/bin:${PATH}"'  >> /opt/go/goco.sh && \
    /bin/echo 'export GOOS=linux'                    >> /opt/go/goco.sh && \
    /bin/echo 'export GOARCH=amd64'                  >> /opt/go/goco.sh && \
    /bin/echo 'cd /opt/go/project'                   >> /opt/go/goco.sh && \
    /bin/echo '$*'                                   >> /opt/go/goco.sh && \
    chmod a+rx /opt/go/goco.sh && \
    /opt/go/goco.sh go version

# Run in go environment.
ENTRYPOINT ["/opt/go/goco.sh"]
CMD ["/opt/go/latest/bin/go", "version"]

This is how you use the goco docker image to compile csdiff for linux.

$ cd csdiff
$ docker run -it --rm -v $(pwd):/opt/go/project goco make

The binary will appear in bin/Linux-x86_64/csdiff.

I originally developed this because I was working on a Mac and FileMerge did not provide line numbers but then added regular expression filtering which also made it useful on linux.

This tool was written in go-1.8.3 using the atom and emacs editors on Mac OSX 10.12.5.

It has been tested on Mac OS X 10.12.5, CentOS 7 and CentOS 6.

Suggestions and improvements are greatly appreciated.

1. Get go cross-compilation working for windows. The terminal size syscall in termcolors does not work.