bioinfologics / w2rap Goto Github PK
View Code? Open in Web Editor NEWWGS (Wheat) Robust Assembly Pipeline
WGS (Wheat) Robust Assembly Pipeline
Hi,
I found a singularity container for w2rap made by @remiolsen . Is it correct that this container is still missing the required dependencies?
Does anyone have plans to get w2rap running inside singularity container or to create a bioconda package?
Thank you in advance.
Michal
Hi,
I repeatedly get memory overflow errors when setting -m option.
As it is a "soft" memory limit, as written in the options, is there an absolute way to limit memory requirements of w2rap?
I use a node of 256 Gb RAM and set -m 200
Thank you,
Michel
Good day Jon,
I am trying to look into my 8kb mate-pair library, but while running the lmp_processing script, I get the following:
In the error message I get:
terminate called after throwing an instance of 'std::regex_error'
what(): regex_error
./run.sh: line 6: 6888 Aborted dedup_fastq -i MP_8000_100_S2L2.extendedFrags.fastq -o MP_8000_100_S2L2.extendedFrags.dedup > MP_8000_100_S2L2_dedup.log
cat: MP_8000_100_S2L2.extendedFrags.dedup.fastq: No such file or directory
cat: MP_8000_100_S2L2.extendedFrags.dedup_rc.fastq: No such file or directory
Traceback (most recent call last):
File "/apps/chpc/bio/w2rap/scripts/lmp_processing", line 219, in <module>
print get_flash_stats(lib["prefix"])
File "/apps/chpc/bio/w2rap/scripts/lmp_processing", line 76, in get_flash_stats
return "{0}:\n{1}\n{2}\n{3}\n{4}\n\nAfter de-duplication: {5}\n".format(lib_prefix, f_fields[1], f_fields[2], f_fields[3], f_fields[4], lines[5])
IndexError: list index out of range
In the output message:
#### w2rap LMP processing ####
FLASH found: /apps/chpc/bio/w2rap/bin/flash
dedup_fastq found: /apps/chpc/bio/w2rap/bin/dedup_fastq
Nextclip found: /apps/chpc/bio/w2rap/bin/nextclip
Number of libraries to process: 1
/home/astander2/lustre/00_GenomeRooibos/01_Data/02_RawData/MP_8000_100_S2L2_R1.fastq /home/astander2/lustre/00_GenomeRooibos/01_Data/02_RawData/MP_8000_100_S2L2_R2.fastq
Running FLASh and de-duplicating combined reads...
The log file in FLASH:
[FLASH] Starting FLASH v1.2.11
[FLASH] Fast Length Adjustment of SHort reads
[FLASH]
[FLASH] Input files:
[FLASH] /home/astander2/lustre/00_GenomeRooibos/01_Data/02_RawData/MP_8000_100_S2L2_R1.fastq
[FLASH] /home/astander2/lustre/00_GenomeRooibos/01_Data/02_RawData/MP_8000_100_S2L2_R2.fastq
[FLASH]
[FLASH] Output files:
[FLASH] ./MP_8000_100_S2L2.extendedFrags.fastq
[FLASH] ./MP_8000_100_S2L2.notCombined_1.fastq
[FLASH] ./MP_8000_100_S2L2.notCombined_2.fastq
[FLASH] ./MP_8000_100_S2L2.hist
[FLASH] ./MP_8000_100_S2L2.histogram
[FLASH]
[FLASH] Parameters:
[FLASH] Min overlap: 10
[FLASH] Max overlap: 100
[FLASH] Max mismatch density: 0.250000
[FLASH] Allow "outie" pairs: false
[FLASH] Cap mismatch quals: false
[FLASH] Combiner threads: 32
[FLASH] Input format: FASTQ, phred_offset=33
[FLASH] Output format: FASTQ, phred_offset=33
[FLASH]
[FLASH] Starting reader and writer threads
[FLASH] Starting 32 combiner threads
[FLASH] Processed 25000 read pairs
[FLASH] Processed 50000 read pairs
....
[FLASH] Processed 68900000 read pairs
[FLASH] Processed 68914397 read pairs
[FLASH]
[FLASH] Read combination statistics:
[FLASH] Total pairs: 68914397
[FLASH] Combined pairs: 7515731
[FLASH] Uncombined pairs: 61398666
[FLASH] Percent combined: 10.91%
[FLASH]
[FLASH] Writing histogram files.
[FLASH]
[FLASH] FLASH v1.2.11 complete!
[FLASH] 130.723 seconds elapsed
Do you have any idea what is causing this?
My jobscript:
#!/bin/bash
#PBS -l select=1:ncpus=32:mpiprocs=32:mem=300GB
#PBS -l walltime=48:00:00
#PBS -q bigmem
#PBS -W group_list=bigmemq
#PBS -P CBBI1133
#PBS -o /home/astander2/lustre/00_GenomeRooibos/00_SubsetRuns/07_w2rap/01_MP8_SameLength/lmp_processing.out
#PBS -e /home/astander2/lustre/00_GenomeRooibos/00_SubsetRuns/07_w2rap/01_MP8_SameLength/lmp_processing.err
#PBS -N lmp_processing
#PBS -M [email protected]
module load gcc/6.1.0
module load chpc/python/3.5.2_gcc-6.2.0
cd /home/astander2/lustre/00_GenomeRooibos/00_SubsetRuns/07_w2rap/01_MP8_SameLength
export PATH=/apps/chpc/bio/w2rap/bin:$PATH
/apps/chpc/bio/w2rap/scripts/lmp_processing ListOfReads 32
Kind regards,
Allison
Hi all,
Have you ever tried to use BESST scaffolder and what do you think about it?
Thank you in advance.
Michal
Dear Jon,
I reached step5 but get a core dump.
Sterr:
/var/spool/slurmd/job08977/slurm_script: line 19: 103982 Aborted (core dumped) /home/mmoser/w2rap-contigger/bin/w2rap-contigger -t 8 -m 100 -r /data/users/mmoser/Illumina/Peaxi_pe01kb_UIL2013_P1.fq,/data/users/mmoser/Illumina/Peaxi_pe01kb_UIL2013_P2.fq -o contigs -p pax_k200 --disk_batches 20
By looking at stout, it is unclear to me if some parameter setting might resolve the problem.
Stout:
--== Step 5: Assembling gaps ==--
Fri Apr 20 22:58:35 2018: inverting paths
Fri Apr 20 22:59:05 2018: Finding unsatisfied path clusters
Fri Apr 20 23:00:05 2018: Merging 8909194 clusters
Fri Apr 20 23:10:28 2018: 1192450 non-inverted clusters
Fri Apr 20 23:13:36 2018: processing 1192450 blobs
ForceAssertGt(MemAvailable(),0ul) at /home/mmoser/w2rap-contigger/src/kmers/naif_kmer/NaifKmerizer.h:560 failed in function
void naif_kmerize(KERNEL_t*, size_t, size_t, size_t) [with KERNEL_t = PreCorrector<Kmer29H, MasterVec<SerfVec<unsigned char> > >; size_t = long unsigned int]
with values arg1 = 0 and arg2 = 0
ForceAssertGt(MemAvailable(),0ul) at /home/mmoser/w2rap-contigger/src/kmers/naif_kmer/NaifKmerizer.h:560 failed in function
void naif_kmerize(KERNEL_t*, size_t, size_t, size_t) [with KERNEL_t = PreCorrector<Kmer29H, MasterVec<SerfVec<unsigned char> > >; size_t = long unsigned int]
with values arg1 = 0 and arg2 = 0
Additionally, when looking into the assembly-folder, i got an pax_k200.first.frags.dist.png.FAIL
-file telling me some execs are missing on the cluster i run w2rap:
Could not generate frags.dist.png because the following executables were not found:
pstopnm,pnmcrop,pnmpad,pnmtopng.
But i assume this would not cause the assembly to terminate, right?
Could you help me to get arround this error?
Thank you,
Michel
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.