side by side diff tool with colorization and regular expression support for filtering
Command line tool that does a side by side diff of two text files with regular expression filtering and ANSI terminal colorization.
It is useful for analyzing text files that have patterns like timestamps that can easily be filtered out.
The tool is very similar to sdiff, wdiff or diff. You specify two files or substitutions and it outputs the differences between them using ANSI terminal colorization to highlight the differences.
You can specify replacement patterns to mask or filter differences that you don't care about, like line numbers or time stamps. It is very similar to using process substitution in sdiff but the regular expression patterns are defined by go. Doing it with sdiff would look something like this (with no colorization).
$ sdiff \
<(cat test/td03.txt | sed -E -e 's@[0-9][0-9]:[0-9][0-9]:[0-9][0-9]@HH:MM:SS@g') \
<(cat test/td04.txt | sed -E -e 's@[0-9][0-9]:[0-9][0-9]:[0-9][0-9]@HH:MM:SS@g' )
The default colorization is very simple, a light grey background is used to highlight differences. A background color was chosen so that it would be easy to see extra spaces. You can customize the colors.
Here is an example that shows two files whose content only differ by a time stamp.
$ bin/Darwin-x86_64/csdiff test/td03.txt test/td04.txt
You can see the differences in the vertical, light grey stripes.
Here is an example that shows how to use the replacement option (-r) to ignore the timestamp differences. As you can see, the go regular expressions are a bit more concise than the previous sed
example.
$ bin/Darwin-x86_64/csdiff \
-r '\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}' 'yyyy-mm-dd HH:MM:SS' \
test/td03.txt test/td04.txt
Now there are no differences because they were masked.
You have the option of customizing the output colors based on the type of data by specifying colormaps for different types of output.
Here is an example of how that is done.
$ bin/Darwin-x86_64/csdiff -c cd=bold,fgRed \
-c cm=bold,fgBlue \
-c s=bold,fgMagenta \
-c lm=bold,fgGreen \
-c llo=bold,fgCyan \
-c rlo=bold,fgCyan \
test/td01.txt test/td02.txt
Contrast that with the default below and you can see the differences.
$ bin/Darwin-x86_64/csdiff test/td01.txt test/td02.txt
As you can see, there are many more colors in the first example.
Color is specified by defining a color map for colorizable entities. The syntax looks like this.
-c cd=bold,fgRed
^ ^ ^^ ^
| | || +--- Another colorization value or text modifier, you can specify as many as you like,
| | |+-------- A colorization value or text modifier (described below).
| | +--------- Equals sign that separates the tag from the attributes.
| +----------- Tag (described below)
+-------------- Option that specifies a color map.
Note that you can have multiple color maps on the same line by separating them with a semi-colon as show below.
-c 'cd=bold,fgRed;cm=bold,fgBlue'
These tags describe the data that can be colored.
Entity | Abbreviation | Description |
---|---|---|
CharsMatch | cm | Color of characters that match on both lines when there are differences. |
CharsDiff | cd | Color of characters that differ on both lines. |
LineMatch | lm | Color of characters when both lines match. |
LeftLineOnly | llo | Color of characters when there is no right line. |
RightLineOnly | rlo | Color of characters when there is no left line. |
Symbol | sym | Color of the sdiff symbol in the middle. The symbol is |, <, > or nothing. |
These are the symbols that csdiff inserts between the lines. They cannot be changed.
Symbol | Description |
---|---|
| | There are differences between the two lines. |
< | There is no matching right line, the left line was inserted. |
> | There is no matching left line, the right line was inserted. |
There is no difference between the lines. |
These are the values that you can specify to color text. It is a subset of the ANSI terminal colors that should work everywhere. They are not case sensitive.
- fgDefault
- fgBlack
- fgBlue
- fgCyan
- fgGreen
- fgMagenta
- fgRed
- fgYellow
- fgWhite
- fgLightBlue
- fgLightCyan
- fgLightGreen
- fgLightGrey
- fgLightMagenta
- fgLightRed
- fgLightYellow
- fgDarkGrey
These are the values that you can specify for the background. It is a subset of the ANSI terminal colors that should work everywhere. They are not case sensitive.
- bgDefault
- bgBlack
- bgBlue
- bgCyan
- bgGreen
- bgMagenta
- bgRed
- bgYellow
- bgLightBlue
- bgLightCyan
- bgLightGreen
- bgLightGrey
- bgLightMagenta
- bgLightRed
- bgLightYellow
- bgDarkGrey
These are the values that you can specify to change how the text is displayed. They are not case sensitive.
Tag | Description |
---|---|
bold | Make the text bold. |
dim | Make the text dim. |
underline | Underline the text. |
blink | Blink. |
reverse | Reverse the foreground and background. |
reset | Reset the colors and modifiers. |
You can specify a much wider range of colors using the ANSI 256 color mode colors using: fg256[N]
and bg256[N]
where N is a number between in the range [0..255]. To see all of the colors available use the --256
option.
$ csdiff --256
The
fg256[N]
andbg256[N]
color options are only available in version 0.5.x or later.
The command line options are best accessed by looking at the inline help because they may change over time but these are the command line options available in 0.4.x.
Long Option | Short Option | Brief Description |
---|---|---|
--256 | NONE | Print the 256 color ANSI color map values. |
--color-map COLOR_MAP | --c COLOR_MAP | Specify a color map for a tag. |
--clear | NONE | Clear the default color map. |
--config FILE | NONE | Specify a color map config file. |
--help | -h | Inline help. |
--diff | -d | Do a traditional diff. Useful for very long lines. |
--no-color | -n | Turn off colorization. Used for testing. |
--replace PATT REP | -r PATT REP | Specify a pattern to replace. Can be specified multiple times. |
--suppress | -s | Suppress common lines. |
--version | -V | Print the program version and exit. |
--width NUM | -w NUM | The width of the output. The default is the width of the terminal. |
Just download the tar or zip image for your system and extract the executable. You can use it directly or copy it.
$ curl -s -k -L -O https://github.com/jlinoff/csdiff/releases/download/v0.5.0/csdiff-linux_amd64.tar.gz
$ tar ztvf csdiff-linux_amd64.tar.gz
-rwxr-xr-x 1 jlinoff 1784424920 2331734 Jun 9 13:52 csdiff
$ curl -s -k -L -O https://github.com/jlinoff/csdiff/releases/download/v0.5.0/csdiff-darwin_amd64.zip
$ unzip -l csdiff-darwin_amd64.zip
Archive: rel/csdiff-darwin_amd64.zip
Length Date Time Name
--------- ---------- ----- ----
2317280 06-09-2017 13:52 csdiff
--------- -------
2317280 1 file
Here is how you check out the source to modify it. If you make changes or implement bug fixes that you think might be useful, please let me know so that I can incorporate them.
$ git clone https://github.com/jlinoff/csdiff.git
Note that I used the following Dockerfile to create the linux image because I was working on a Mac and wanted to be able to test on linux. To simply build on linux, you can use the built-in go capability for cross compilation using GOOS and GOARCH.
# This docker file creates a go compilation container that can be used
# to cross-compile go for linux on any platform that supports docker.
#
# $ cd csdiff
# $ docker run -it --rm -v $(pwd):/opt/go/project goco make
#
# To create it. Note that gover is an ARG that defines the version of
# go that you want to build. This example shows how to build it for go-1.8.3.
#
# $ docker build --build-arg gover=1.8.3 -f Dockerfile -t goco:1.8.3 -t goco:latest .
FROM centos:latest
RUN yum clean all && yum update -y && yum install -y git make
ARG gover
ENV GO_VERSION=$gover
ENV GOROOT=/opt/go/latest
ENV GOPATH=/opt/go/project
ENV GO_PROG=/opt/go/latest/bin/go
# Setup the volume.
RUN mkdir -p ${GOPATH}
VOLUME ${GOPATH}
# Install go in /opt/go
RUN mkdir -p /opt/go/${GO_VERSION}/dl && \
cd /opt/go/${GO_VERSION}/dl && \
curl -k -O -L https://storage.googleapis.com/golang/go${GO_VERSION}.linux-amd64.tar.gz && \
cd /opt/go/${GO_VERSION} && \
tar zxf dl/go${GO_VERSION}.linux-amd64.tar.gz && \
ln -s /opt/go/${GO_VERSION}/go /opt/go/latest && \
${GO_PROG} version
# Install golint
RUN cd /opt/go/${GO_VERSION}/dl && \
GOPATH=/opt/go/${GO_VERSION}/dl ${GOROOT}/bin/go get -u github.com/golang/lint/golint && \
cp bin/* ${GOROOT}/bin
# Wrapper for the go command that makes it
# natural for the user to run something like
# docker run -it --rm -v $(pwd):/opt/go/project goco go build myprog.go
RUN /bin/echo '#!/bin/bash' > /opt/go/goco.sh && \
/bin/echo 'export PATH="${GOROOT}/bin:${PATH}"' >> /opt/go/goco.sh && \
/bin/echo 'export GOOS=linux' >> /opt/go/goco.sh && \
/bin/echo 'export GOARCH=amd64' >> /opt/go/goco.sh && \
/bin/echo 'cd /opt/go/project' >> /opt/go/goco.sh && \
/bin/echo '$*' >> /opt/go/goco.sh && \
chmod a+rx /opt/go/goco.sh && \
/opt/go/goco.sh go version
# Run in go environment.
ENTRYPOINT ["/opt/go/goco.sh"]
CMD ["/opt/go/latest/bin/go", "version"]
This is how you use the goco docker image to compile csdiff for linux.
$ cd csdiff
$ docker run -it --rm -v $(pwd):/opt/go/project goco make
The binary will appear in bin/Linux-x86_64/csdiff.
I originally developed this because I was working on a Mac and FileMerge did not provide line numbers but then added regular expression filtering which also made it useful on linux.
This tool was written in go-1.8.3 using the atom and emacs editors on Mac OSX 10.12.5.
It has been tested on Mac OS X 10.12.5, CentOS 7 and CentOS 6.
Suggestions and improvements are greatly appreciated.
- Get go cross-compilation working for windows. The terminal size syscall in termcolors does not work.