lbcb-sci / ra Goto Github PK

View Code? Open in Web Editor NEW

This project forked from rvaser/ra

38.0 38.0 4.0 1.29 MB

Assembler for raw de novo genome assembly of long uncorrected reads.

License: MIT License

CMake 7.80% Python 92.20%

ra's People

Contributors

Stargazers

Watchers

Forkers

pythseq dayedepps pseudogene hinanawi-tanshi

ra's Issues

Combining Pacbio and nanopore

Hello,
how do we feed the 2 types of reads to Ra, for a single assembly?

thank you

Hello,
I have been running ra on a ~5 Gb plant genome, highly heterozygous.
The input file is about 157 Gb of ONT reads (>QV7, above 10 kb), N50 24 kb. After about 5 days, the job dies while still at the minimap stage and the only message I have is:

[M::worker_pipeline::479382.631*42.28] mapped 30111 sequences
[M::worker_pipeline::479526.304*42.27] mapped 29912 sequences
[ERROR] failed to write the results

furthermore, the .paf file and the working dir are gone.
Is there a way to run separately (or even outside of ra) the alignment step?
I will test raven as well, but a colleague told me that it died as well (I don't know more details about this though).
Thanks

Question about Racon Rounds

Hello, I have a naive question about Ra.
How many rounds of Racon does it implement during the whole process of assembly construction ?

Thank you for your time

resume job ?

Hi,

My job failed due to cluster limit.
Is it possible to resume a job ?

I was at this stage :

[M::main] Version: 2.14-r892-dirty
[M::main] CMD: ra/vendor/minimap2/minimap2 -t 53 -x ava-ont Tpa_nanopore.fastq Tpa_nanopore.fastq
[M::main] Real time: 269685.254 sec; CPU: 11681533.570 sec; Peak RSS: 57.985 GB
[Ra::run] preconstruction stage
[rala::Graph::initialize] loaded sequences

Release for bioconda?

Hi!

can you add a release version so that we can add this to bioconda-recipes? That would be fab, especially as we can then install it easily using conda install -c bioconda ...

Rala seems stuck

Hello,
Rala seems somewhat stuck, it has been "running" for 2 days, the threads are in S status. The ram usage is quite high however. Is it normal behaviour? Here was my command

./ra/build/bin/ra -t 48 -x pb allLongReads.noduplicateheader.reformat.fa reads.fq.gz > Ra.assembly.fa
The expected genome size is 120 Mb

And this is the size of the files in the "working directory"

ls -lh
total 323G
-rw-rw-r-- 1 lege lege    0 Aug 19 04:04 layout.fasta
-rw-rw-r-- 1 lege lege 308G Aug 18 21:32 overlaps.paf
-rw-rw-r-- 1 lege lege 810M Aug 19 03:54 preconstruction.fasta
-rw-rw-r-- 1 lege lege  15G Aug 19 04:04 sensitive_overlaps.paf

Thank you

EDIT: a colleague pointed to me there was 250 Go left on the drive, can it be Rala is waiting to reserve enough disk space to write everything at once?

Cannot find minimap2

Hello,

I am having trouble to run Ra on a cluster. I tried on two different clusters, and get this:
[Ra::run] preconstruction overlap stage [Ra::run] error: unable to run minimap2!
But minimap2 is in my PATH. How can I make sure it finds minimap2?

How to add Illumina paired end raw reads?

sivico26 commented on May 20:
The command line I used to run Ra was the following:
ra -t $threads -x ont $ont_reads reads/ngs.fastq.gz > assembly.fasta
where $threads was set to 24 and $ont_reads, was the path to ont-reads and ngs.fastq.gz are Illumina reads.

BUT it is not clear WHAT is ngs.fastq.gz file? Paired-end raw reads?

My question is how to use Illumina paired-end raw reads?

I am going to use Illumina paired-end raw reads (DRR_1.fastq DRR_2.fastq) to polish my scaffolds.
Is this command correct?
ra -t $threads -x ont ont_reads.fastq DRR_1.fastq DRR_2.fastq > assembly.fasta`

RA in Docker

Have you ever tried packaging RA in Docker? I am getting errors indicating that files cannot be written when building the image. Eg. error: unable to write file test_data/Chromosome_2890043_3890042_0/mutated_94_perc.fasta

estimation of footprint to assemble a large plant genome

Hello,
we would like to use Ra to assemble a ~5 Gb plant genome (the genome is actually 2.5Gb in size, but it is highly-heterozygous, so we want to distinguish the two alleles in separate contigs). I have about 45x (of 5 Gb) of ONT data (N50 15 kb, QV >7) and we wonder if there is a way to predict the size of the minimap2 file and optimize the alignment step, since it is the most expensive.
For example, to simplify the output, adding/tweaking the minimap2 options: -X -p -N
and to be more sensitive: -A -B -O -E -z.
Do you think there is room in that part to increase specificity of alignments and decrease footprint and computation time?

Lastly, can you estimate the memory usage for an assembly where we have up to 36 CPUs (hyperthreading to 72) and max 500 GB RAM available? Will the polishing/graph construction steps be more memory demanding than the alignment step?
Thanks,
Dario

Problems with compiling ...

Could you help me further? I tried different compilers with no success:

cmake -DCMAKE_BUILD_TYPE=Release .. && make
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /home/prog/gcc/gcc-7.4.0/bin/gcc
-- Check for working C compiler: /home/prog/gcc/gcc-7.4.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/prog/gcc/gcc-7.4.0/bin/g++
-- Check for working CXX compiler: /home/prog/gcc/gcc-7.4.0/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of off64_t
-- Check size of off64_t - done
-- Looking for fseeko
-- Looking for fseeko - found
-- Looking for unistd.h
-- Looking for unistd.h - found
Setting warning flags
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ms257013/bin/ra/build
Scanning dependencies of target minimap2
[  0%] Built target minimap2
Scanning dependencies of target thread_pool
[  2%] Building CXX object vendor/rala/vendor/thread_pool/CMakeFiles/thread_pool.dir/src/thread_pool.cpp.o
[  5%] Linking CXX static library lib/libthread_pool.a
[  5%] Built target thread_pool
Scanning dependencies of target zlibstatic
[  7%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/adler32.o
[ 10%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/compress.o
[ 13%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/crc32.o
[ 15%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/deflate.o
[ 18%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/gzclose.o
[ 21%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/gzlib.o
[ 23%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/gzread.o
[ 26%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/gzwrite.o
[ 28%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/inflate.o
[ 31%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/infback.o
[ 34%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/inftrees.o
[ 36%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/inffast.o
[ 39%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/trees.o
[ 42%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/uncompr.o
[ 44%] Building C object vendor/rala/vendor/bioparser/vendor/zlib/CMakeFiles/zlibstatic.dir/zutil.o
[ 47%] Linking C static library ../../lib/libz.a
[ 47%] Built target zlibstatic
Scanning dependencies of target rala
[ 50%] Building CXX object vendor/rala/CMakeFiles/rala.dir/src/graph.cpp.o
[ 52%] Building CXX object vendor/rala/CMakeFiles/rala.dir/src/main.cpp.o
[ 55%] Building CXX object vendor/rala/CMakeFiles/rala.dir/src/overlap.cpp.o
[ 57%] Building CXX object vendor/rala/CMakeFiles/rala.dir/src/pile.cpp.o
[ 60%] Building CXX object vendor/rala/CMakeFiles/rala.dir/src/sequence.cpp.o
[ 63%] Building CXX object vendor/rala/CMakeFiles/rala.dir/src/timer.cpp.o
[ 65%] Linking CXX executable bin/rala
[ 65%] Built target rala
Scanning dependencies of target edlib_static
[ 68%] Building CXX object vendor/racon/vendor/edlib/CMakeFiles/edlib_static.dir/edlib/src/edlib.cpp.o
[ 71%] Linking CXX static library ../../../../lib/libedlib_static.a
[ 71%] Built target edlib_static
Scanning dependencies of target spoa
[ 73%] Building CXX object vendor/racon/vendor/spoa/CMakeFiles/spoa.dir/src/alignment_engine.cpp.o
[ 76%] Building CXX object vendor/racon/vendor/spoa/CMakeFiles/spoa.dir/src/graph.cpp.o
/tmp/cc3avReL.s: Assembler messages:
/tmp/cc3avReL.s:3615: Error: no such instruction: `vpbroadcastd %xmm0,%ymm0'
/tmp/cc3avReL.s:3760: Error: no such instruction: `vpbroadcastd %xmm0,%ymm0'
/tmp/cc3avReL.s:3887: Error: no such instruction: `vpbroadcastd %xmm0,%ymm0'
/tmp/cc3avReL.s:4055: Error: no such instruction: `vpbroadcastd %xmm0,%ymm0'
/tmp/cc3avReL.s:5249: Error: no such instruction: `vpbroadcastd %xmm0,%ymm0'
/tmp/cc3avReL.s:5394: Error: no such instruction: `vpbroadcastd %xmm0,%ymm0'
/tmp/cc3avReL.s:5521: Error: no such instruction: `vpbroadcastd %xmm0,%ymm0'
/tmp/cc3avReL.s:5689: Error: no such instruction: `vpbroadcastd %xmm0,%ymm0'
/tmp/cc3avReL.s:6709: Error: no such instruction: `shlx %rax,%rdi,%rax'
/tmp/cc3avReL.s:6722: Error: no such instruction: `shlx %rax,%rdi,%rax'
/tmp/cc3avReL.s:6731: Error: no such instruction: `shlx %r11,%rdi,%r11'
/tmp/cc3avReL.s:7065: Error: no such instruction: `shlx %r10,%rax,%r14'
/tmp/cc3avReL.s:7071: Error: no such instruction: `shlx %r9,%rax,%rax'
/tmp/cc3avReL.s:7137: Error: no such instruction: `vpbroadcastq %xmm1,%ymm0'
/tmp/cc3avReL.s:7201: Error: no such instruction: `shlx %rbx,%r8,%rsi'
/tmp/cc3avReL.s:7289: Error: no such instruction: `shlx %rcx,%r9,%rsi'
/tmp/cc3avReL.s:7327: Error: no such instruction: `shlx %rcx,%r9,%r8'
/tmp/cc3avReL.s:7335: Error: no such instruction: `shlx %rcx,%r9,%r8'
/tmp/cc3avReL.s:7404: Error: no such instruction: `shlx %rax,%r10,%rsi'
/tmp/cc3avReL.s:7478: Error: no such instruction: `vpbroadcastq %xmm2,%ymm0'
/tmp/cc3avReL.s:7545: Error: no such instruction: `shlx %rax,%rdi,%rsi'
/tmp/cc3avReL.s:7573: Error: no such instruction: `shlx %rax,%rsi,%rdx'
/tmp/cc3avReL.s:7582: Error: no such instruction: `shlx %rax,%rsi,%rdx'
/tmp/cc3avReL.s:7618: Error: no such instruction: `shlx %r13,%rdi,%rax'
/tmp/cc3avReL.s:7619: Error: no such instruction: `shlx %rbx,%rdi,%r8'
/tmp/cc3avReL.s:7649: Error: no such instruction: `shlx %rax,%rsi,%rcx'
/tmp/cc3avReL.s:7657: Error: no such instruction: `shlx %rax,%rsi,%rcx'
/tmp/cc3avReL.s:7691: Error: no such instruction: `shlx %rbx,%r8,%rcx'
/tmp/cc3avReL.s:7700: Error: no such instruction: `shlx %rbx,%r8,%rcx'
/tmp/cc3avReL.s:7730: Error: no such instruction: `shlx %rbx,%r8,%rsi'
/tmp/cc3avReL.s:7738: Error: no such instruction: `shlx %rbx,%r8,%rsi'
/tmp/cc3avReL.s:7761: Error: no such instruction: `shlx %rax,%rsi,%rcx'
/tmp/cc3avReL.s:7770: Error: no such instruction: `shlx %rax,%rsi,%rcx'
/tmp/cc3avReL.s:7791: Error: no such instruction: `shlx %rax,%rsi,%rcx'
/tmp/cc3avReL.s:8126: Error: no such instruction: `shlx %r9,%rbx,%r12'
/tmp/cc3avReL.s:8222: Error: no such instruction: `shlx %rbx,%rsi,%r12'
/tmp/cc3avReL.s:8535: Error: no such instruction: `shlx %rax,%r14,%r8'
/tmp/cc3avReL.s:8545: Error: no such instruction: `shlx %rax,%r14,%rdi'
/tmp/cc3avReL.s:8612: Error: no such instruction: `shlx %rax,%r14,%rax'
/tmp/cc3avReL.s:9529: Error: no such instruction: `vpbroadcastd %xmm0,%ymm0'
/tmp/cc3avReL.s:9992: Error: no such instruction: `vpbroadcastd %xmm0,%ymm0'
/tmp/cc3avReL.s:10527: Error: no such instruction: `shlx %r12,%rdx,%rdx'
/tmp/cc3avReL.s:10606: Error: no such instruction: `shlx %r11,%r10,%r9'
make[2]: *** [vendor/racon/vendor/spoa/CMakeFiles/spoa.dir/src/graph.cpp.o] Error 1
make[1]: *** [vendor/racon/vendor/spoa/CMakeFiles/spoa.dir/all] Error 2
make: *** [all] Error 2

build > export CXX=/home/prog/gcc/gcc-5.4.0/bin/g++
 build > export CC=/home/prog/gcc/gcc-5.4.0/bin/gcc
 build > cmake -DCMAKE_BUILD_TYPE=Release .. && make
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /home/prog/gcc/gcc-5.4.0/bin/gcc
-- Check for working C compiler: /home/prog/gcc/gcc-5.4.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/prog/gcc/gcc-5.4.0/bin/g++
-- Check for working CXX compiler: /home/prog/gcc/gcc-5.4.0/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of off64_t
-- Check size of off64_t - done
-- Looking for fseeko
-- Looking for fseeko - found
-- Looking for unistd.h
-- Looking for unistd.h - found
Setting warning flags
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ms257013/bin/ra/build
Scanning dependencies of target minimap2
[  0%] Built target minimap2
Scanning dependencies of target thread_pool
[  2%] Building CXX object vendor/rala/vendor/thread_pool/CMakeFiles/thread_pool.dir/src/thread_pool.cpp.o
/tmp/cc9xYZfp.s: Assembler messages:
/tmp/cc9xYZfp.s:2106: Error: no such instruction: `vinserti128 $0x1,%xmm2,%ymm1,%ymm1'
/tmp/cc9xYZfp.s:2110: Error: no such instruction: `vinserti128 $0x1,%xmm1,%ymm0,%ymm0'
make[2]: *** [vendor/rala/vendor/thread_pool/CMakeFiles/thread_pool.dir/src/thread_pool.cpp.o] Error 1
make[1]: *** [vendor/rala/vendor/thread_pool/CMakeFiles/thread_pool.dir/all] Error 2
make: *** [all] Error 2

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.