ofanoyi / ea-utils Goto Github PK

What steps will reproduce the problem?
1. copy fwd.fastq in one dir
2. copy rvs.fastq in the same dir
3. fastq-join fwd.fastq rvs.fastq -o c1

What is the expected output? What do you see instead?
I expect three joined reads. Everything goes right if I process each of the 
separately!

But I see just an error:
*** glibc detected *** fastq-join: realloc(): invalid next size: 
0x0000000000ad4e30 ***
======= Backtrace: =========
/lib/libc.so.6(+0x774b6)[0x7f6e8e4ff4b6]
/lib/libc.so.6(+0x7db96)[0x7f6e8e505b96]
/lib/libc.so.6(realloc+0xf0)[0x7f6e8e505eb0]
fastq-join[0x401065]
fastq-join[0x401ce5]
/lib/libc.so.6(__libc_start_main+0xfe)[0x7f6e8e4a6d8e]
fastq-join[0x400c79]
======= Memory map: ========
00400000-00403000 r-xp 00000000 08:05 9579591                            
/usr/local/bin/fastq-join
00602000-00603000 r--p 00002000 08:05 9579591                            
/usr/local/bin/fastq-join
00603000-00604000 rw-p 00003000 08:05 9579591                            
/usr/local/bin/fastq-join
00ad4000-00af5000 rw-p 00000000 00:00 0                                  [heap]
7f6e88000000-7f6e88021000 rw-p 00000000 00:00 0 
7f6e88021000-7f6e8c000000 ---p 00000000 00:00 0 
7f6e8e488000-7f6e8e602000 r-xp 00000000 08:05 4458098                    
/lib/libc-2.12.1.so
7f6e8e602000-7f6e8e801000 ---p 0017a000 08:05 4458098                    
/lib/libc-2.12.1.so
7f6e8e801000-7f6e8e805000 r--p 00179000 08:05 4458098                    
/lib/libc-2.12.1.so
7f6e8e805000-7f6e8e806000 rw-p 0017d000 08:05 4458098                    
/lib/libc-2.12.1.so
7f6e8e806000-7f6e8e80b000 rw-p 00000000 00:00 0 
7f6e8e80b000-7f6e8e820000 r-xp 00000000 08:05 4456504                    
/lib/libgcc_s.so.1
7f6e8e820000-7f6e8ea1f000 ---p 00015000 08:05 4456504                    
/lib/libgcc_s.so.1
7f6e8ea1f000-7f6e8ea20000 r--p 00014000 08:05 4456504                    
/lib/libgcc_s.so.1
7f6e8ea20000-7f6e8ea21000 rw-p 00015000 08:05 4456504                    
/lib/libgcc_s.so.1
7f6e8ea21000-7f6e8eaa3000 r-xp 00000000 08:05 4461329                    
/lib/libm-2.12.1.so
7f6e8eaa3000-7f6e8eca2000 ---p 00082000 08:05 4461329                    
/lib/libm-2.12.1.so
7f6e8eca2000-7f6e8eca3000 r--p 00081000 08:05 4461329                    
/lib/libm-2.12.1.so
7f6e8eca3000-7f6e8eca4000 rw-p 00082000 08:05 4461329                    
/lib/libm-2.12.1.so
7f6e8eca4000-7f6e8ed8c000 r-xp 00000000 08:05 1313209                    
/usr/lib/libstdc++.so.6.0.14
7f6e8ed8c000-7f6e8ef8b000 ---p 000e8000 08:05 1313209                    
/usr/lib/libstdc++.so.6.0.14
7f6e8ef8b000-7f6e8ef93000 r--p 000e7000 08:05 1313209                    
/usr/lib/libstdc++.so.6.0.14
7f6e8ef93000-7f6e8ef95000 rw-p 000ef000 08:05 1313209                    
/usr/lib/libstdc++.so.6.0.14
7f6e8ef95000-7f6e8efaa000 rw-p 00000000 00:00 0 
7f6e8efaa000-7f6e8efca000 r-xp 00000000 08:05 4461330                    
/lib/ld-2.12.1.so
7f6e8f1a6000-7f6e8f1ab000 rw-p 00000000 00:00 0 
7f6e8f1c5000-7f6e8f1ca000 rw-p 00000000 00:00 0 
7f6e8f1ca000-7f6e8f1cb000 r--p 00020000 08:05 4461330                    
/lib/ld-2.12.1.so
7f6e8f1cb000-7f6e8f1cc000 rw-p 00021000 08:05 4461330                    
/lib/ld-2.12.1.so
7f6e8f1cc000-7f6e8f1cd000 rw-p 00000000 00:00 0 
7fff309b5000-7fff309d6000 rw-p 00000000 00:00 0                          [stack]
7fff309ff000-7fff30a00000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  
[vsyscall]


What version of the product are you using? On what operating system?
ea-utils.1.1.1-237.tar.gz, but also tried older version or those precompiled 
from CentOs rpm (same problem).

Please provide any additional information below.

Apparently it cannot work on OS X 10.7 (nor on 10.6) because of getopt function 
which always returns -1.

I suggest to apply this patch to use getopt_long:


$ diff -u fastq-mcf.c fastq-mcf.c.new 
--- fastq-mcf.c 2011-11-09 21:33:42.000000000 +0100
+++ fastq-mcf.c.new 2011-12-20 14:30:53.000000000 +0100
@@ -29,6 +29,7 @@
 #include <ctype.h>
 #include <stdio.h>
 #include <stdlib.h>
+#include <getopt.h>
 #include <unistd.h>
 #include <string.h>
 #include <errno.h>
@@ -87,7 +88,7 @@
 int debug=0;
 int warncount = 0;
 int main (int argc, char **argv) {
-   char c;
+   int c;
    bool eol;
    int nmin = 1, nkeep = 15, nmax=0;
    float minpct = 0.25;
@@ -116,7 +117,7 @@
    int e_n = 0;
    bool skipb = 0;

-   while ( (c = getopt (argc, argv, "-nf0uUVSRdbehp:o:l:s:m:t:k:x:P:q:L:C:w:")) 
!= -1) {
+   while ( (c = getopt_long (argc, argv, 
"-nf0uUVSRdbehp:o:l:s:m:t:k:x:P:q:L:C:w:", NULL, NULL)) != -1) {
        switch (c) {
        case '\1': 
            if (!afil) 

What steps will reproduce the problem?
1. Use GATCGGAAGAGCGGTTCAGCAGGAATGCCGAGATCGGAAG as adapter1
2. Use GATCGGAAGAGCTCGTCAGCAGGAATGCCGAGATCGGAAG as adapter2
3. Issue fastq-mcf -n <adapter>.fa <fastq>

What is the expected output? What do you see instead?
Should show that there are two adapters as input.  You will see that it read 
the first adapter and discarded the second adapter.

What version of the product are you using? On what operating system?
ea-utils.1.0.3-145.tar.gz on Fedora 14 / 2.6.35.13-92.fc14.x86_64

Please provide any additional information below.

What steps will reproduce the problem?

1. any attempt to clip adapters

What is the expected output? What do you see instead?

expect them to be clipped

Please use labels and text to provide additional information.

typo in barcode-skip code caused all reads to be skipped... fixed

I just downloaded ea-utils.1.1.0-190.tar.gz onto my Mac (OS 10.6) and tried to 
compile it, but got this error.

$ sudo make install
grep: ea-utils.spex: No such file or directory
g++ -O3 -o fastq-clipper fastq-clipper.c
fastq-clipper.c: In function 'int main(int, char**)':
fastq-clipper.c:138: error: 'getline' was not declared in this scope
make: *** [fastq-clipper] Error 1

I'm pretty sure the getline function is GNU specific, which restricts the use 
of ea-utils to Linux. I wouldn't imagine it to be too difficult to use 
something more portable (fgets perhaps) that would also allow the code to run 
on other Unix flavors (OS X, Solaris, BSD, etc).

...unless you're choosing only to support Linux...

I just ran fastq-multx tool to demultiplex barcodes from the fastq file and got 
this error. 

Error opening file 'GFI-410.fastq': Value too large for defined data type

I am using ea-utilsv1.1.258 on Ubuntu 8.04.4 LTS/Linux 2.6.24-25-generic i686

This is the command I used:
fastq-multx -B barcode-list.txt GFI-410.fastq -o barcode.%.fq -b

My fastq file consist of about 170 million reads and of size 41 GB size. My 
barcode list consist of 56 lines of different barcodes 

Is there a size limit for the fastq file to be used / should I use different 
version. Any help would be much appreciated. 

Thank you