bioinfologics / w2rap Goto Github PK

View Code? Open in Web Editor NEW

22.0 22.0 8.0 35.04 MB

WGS (Wheat) Robust Assembly Pipeline

Python 65.46% CMake 3.68% C++ 30.86%

w2rap's People

Contributors

Stargazers

Watchers

Forkers

jianzuoyi httphangithub garethlin sunyoungbio villegar yuzhenpeng genostack

w2rap's Issues

singularity container or bioconda package for w2rap

Hi,
I found a singularity container for w2rap made by @remiolsen . Is it correct that this container is still missing the required dependencies?

Does anyone have plans to get w2rap running inside singularity container or to create a bioconda package?

Thank you in advance.

Michal

memory limits

Hi,

I repeatedly get memory overflow errors when setting -m option.
As it is a "soft" memory limit, as written in the options, is there an absolute way to limit memory requirements of w2rap?
I use a node of 256 Gb RAM and set -m 200

Thank you,
Michel

lmp_processing Error

Good day Jon,

I am trying to look into my 8kb mate-pair library, but while running the lmp_processing script, I get the following:

In the error message I get:

terminate called after throwing an instance of 'std::regex_error'
what(): regex_error
./run.sh: line 6: 6888 Aborted dedup_fastq -i MP_8000_100_S2L2.extendedFrags.fastq -o MP_8000_100_S2L2.extendedFrags.dedup > MP_8000_100_S2L2_dedup.log
cat: MP_8000_100_S2L2.extendedFrags.dedup.fastq: No such file or directory
cat: MP_8000_100_S2L2.extendedFrags.dedup_rc.fastq: No such file or directory
Traceback (most recent call last):
File "/apps/chpc/bio/w2rap/scripts/lmp_processing", line 219, in <module>
print get_flash_stats(lib["prefix"]) 
File "/apps/chpc/bio/w2rap/scripts/lmp_processing", line 76, in get_flash_stats
return "{0}:\n{1}\n{2}\n{3}\n{4}\n\nAfter de-duplication: {5}\n".format(lib_prefix, f_fields[1], f_fields[2], f_fields[3], f_fields[4], lines[5])
IndexError: list index out of range

In the output message:

#### w2rap LMP processing ####

FLASH found: /apps/chpc/bio/w2rap/bin/flash
dedup_fastq found: /apps/chpc/bio/w2rap/bin/dedup_fastq
Nextclip found: /apps/chpc/bio/w2rap/bin/nextclip
Number of libraries to process: 1
/home/astander2/lustre/00_GenomeRooibos/01_Data/02_RawData/MP_8000_100_S2L2_R1.fastq /home/astander2/lustre/00_GenomeRooibos/01_Data/02_RawData/MP_8000_100_S2L2_R2.fastq

Running FLASh and de-duplicating combined reads...

The log file in FLASH:

[FLASH] Starting FLASH v1.2.11
[FLASH] Fast Length Adjustment of SHort reads
[FLASH] 
[FLASH] Input files:
[FLASH] /home/astander2/lustre/00_GenomeRooibos/01_Data/02_RawData/MP_8000_100_S2L2_R1.fastq
[FLASH] /home/astander2/lustre/00_GenomeRooibos/01_Data/02_RawData/MP_8000_100_S2L2_R2.fastq
[FLASH] 
[FLASH] Output files:
[FLASH] ./MP_8000_100_S2L2.extendedFrags.fastq
[FLASH] ./MP_8000_100_S2L2.notCombined_1.fastq
[FLASH] ./MP_8000_100_S2L2.notCombined_2.fastq
[FLASH] ./MP_8000_100_S2L2.hist
[FLASH] ./MP_8000_100_S2L2.histogram
[FLASH] 
[FLASH] Parameters:
[FLASH] Min overlap: 10
[FLASH] Max overlap: 100
[FLASH] Max mismatch density: 0.250000
[FLASH] Allow "outie" pairs: false
[FLASH] Cap mismatch quals: false
[FLASH] Combiner threads: 32
[FLASH] Input format: FASTQ, phred_offset=33
[FLASH] Output format: FASTQ, phred_offset=33
[FLASH] 
[FLASH] Starting reader and writer threads
[FLASH] Starting 32 combiner threads
[FLASH] Processed 25000 read pairs
[FLASH] Processed 50000 read pairs

....
[FLASH] Processed 68900000 read pairs
[FLASH] Processed 68914397 read pairs
[FLASH] 
[FLASH] Read combination statistics:
[FLASH] Total pairs: 68914397
[FLASH] Combined pairs: 7515731
[FLASH] Uncombined pairs: 61398666
[FLASH] Percent combined: 10.91%
[FLASH] 
[FLASH] Writing histogram files.
[FLASH] 
[FLASH] FLASH v1.2.11 complete!
[FLASH] 130.723 seconds elapsed

Do you have any idea what is causing this?

My jobscript:

#!/bin/bash
#PBS -l select=1:ncpus=32:mpiprocs=32:mem=300GB
#PBS -l walltime=48:00:00
#PBS -q bigmem
#PBS -W group_list=bigmemq
#PBS -P CBBI1133
#PBS -o /home/astander2/lustre/00_GenomeRooibos/00_SubsetRuns/07_w2rap/01_MP8_SameLength/lmp_processing.out
#PBS -e /home/astander2/lustre/00_GenomeRooibos/00_SubsetRuns/07_w2rap/01_MP8_SameLength/lmp_processing.err
#PBS -N lmp_processing
#PBS -M [email protected]

module load gcc/6.1.0
module load chpc/python/3.5.2_gcc-6.2.0

cd /home/astander2/lustre/00_GenomeRooibos/00_SubsetRuns/07_w2rap/01_MP8_SameLength

export PATH=/apps/chpc/bio/w2rap/bin:$PATH
/apps/chpc/bio/w2rap/scripts/lmp_processing ListOfReads 32

Kind regards,
Allison

kmer questions

Hi,
Do you think it would be a good idea:

to use any of khmer recipes and
to use kmergenie in order to determine -K parameter.

Thank you in advance

Michal

scaffolder

Hi all,
Have you ever tried to use BESST scaffolder and what do you think about it?

Thank you in advance.

Michal

error at step5

Dear Jon,

I reached step5 but get a core dump.

Sterr:

/var/spool/slurmd/job08977/slurm_script: line 19: 103982 Aborted                 (core dumped) /home/mmoser/w2rap-contigger/bin/w2rap-contigger -t 8 -m 100 -r /data/users/mmoser/Illumina/Peaxi_pe01kb_UIL2013_P1.fq,/data/users/mmoser/Illumina/Peaxi_pe01kb_UIL2013_P2.fq -o contigs -p pax_k200 --disk_batches 20

By looking at stout, it is unclear to me if some parameter setting might resolve the problem.

Stout:

--== Step 5: Assembling gaps ==--
Fri Apr 20 22:58:35 2018: inverting paths
Fri Apr 20 22:59:05 2018: Finding unsatisfied path clusters
Fri Apr 20 23:00:05 2018: Merging 8909194 clusters
Fri Apr 20 23:10:28 2018: 1192450 non-inverted clusters
Fri Apr 20 23:13:36 2018: processing 1192450 blobs
ForceAssertGt(MemAvailable(),0ul) at /home/mmoser/w2rap-contigger/src/kmers/naif_kmer/NaifKmerizer.h:560 failed in function
void naif_kmerize(KERNEL_t*, size_t, size_t, size_t) [with KERNEL_t = PreCorrector<Kmer29H, MasterVec<SerfVec<unsigned char> > >; size_t = long unsigned int]
with values arg1 = 0 and arg2 = 0
ForceAssertGt(MemAvailable(),0ul) at /home/mmoser/w2rap-contigger/src/kmers/naif_kmer/NaifKmerizer.h:560 failed in function
void naif_kmerize(KERNEL_t*, size_t, size_t, size_t) [with KERNEL_t = PreCorrector<Kmer29H, MasterVec<SerfVec<unsigned char> > >; size_t = long unsigned int]
with values arg1 = 0 and arg2 = 0

Additionally, when looking into the assembly-folder, i got an pax_k200.first.frags.dist.png.FAIL -file telling me some execs are missing on the cluster i run w2rap:

Could not generate frags.dist.png because the following executables were not found:      
pstopnm,pnmcrop,pnmpad,pnmtopng.

But i assume this would not cause the assembly to terminate, right?

Could you help me to get arround this error?

Thank you,
Michel

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.