jiazhen-rong / biofast Goto Github PK

Biofast is a small benchmark for evaluating the performance of programming languages and implementations on a few common tasks in the field of Bioinformatics. It currently includes two benchmarks: FASTQ parsing and interval query. Please see also the companion blog post.

We ran the test on a CentOS 7 server with two EPYC 7301 CPUs and 1TB memory. The system comes with gcc-4.8.5, python-3.7.6, nim-1.2.0, julia-1.4.1, go-1.14.3, luajit-322db02 and k8-0.2.5. Relatively small libraries are included in the lib directory directory.

We tried to avoid other active processes when test programs were running. Timing in this page was obtained with hyperfine, which reports CPU time averaged in at least ten rounds. Peak memory was often measured only once as hyperfine doesn't report memory usage.

Full results can be found in the fqcnt and bedcov directories, respectively. This README only shows one implementation per language. We exclude those binding to C libraries and try to select the one implementing a similar algorithm to the C version.

In this benchmark, we parse a 4-line FASTQ file consisting of 5,682,010 records and report the number of records and the total length of sequences and quality. The input file is M_abscessus_HiSeq.fq in biofast-data-v1.tar.gz from the download page. In the table below, "t_gzip" gives the CPU time in seconds for gzip'd input and "t_plain" gives the time for raw input without compression.

• For the full table, see the fqcnt directory.

In this benchmark, we load one BED file into memory. We stream another BED file and compute coverage of each interval using the cgranges algorithm (see the C++ header for algorithm details). The output all programs should be identical "bedtools coverage". In the table below, "t" stands for CPU time in seconds and "M" for peak memory in mega-bytes. Subscripts "g2r" and "r2g" correspond to the following two command lines, respectively:

bedcov ex-rna.bed ex-anno.bed  # g2r
bedcov ex-anno.bed ex-rna.bed  # r2g

Both input BED files can be found in biofast-data-v1.tar.gz from the download page.

• For the full table, see the bedcov directory.

• Crystal, Nim and Julia use a long contiguous memory block for loaded intervals on each chromosome. Javascript and LuaJIT instead use a list of separate objects.

• Bounds check is completely disabled for Nim and partially disabled for Crystal. Bounds check might be partially disabled in Julia, too. Not sure.

• For "g2r", sorting takes significant time. C uses a radix sort. Other languages rely on their standard libraries.