Comments (6)
Hi @Hecate08 ,
I have to admit that there is no need to extract reads from BAM file. Unfortunately, this is how RSEM is programmed. At the first 10 EM iterations, RSEM need to load these files for estimating sequencing error parameters.
However, I am surprised that the indexing step took that long time. In theory, this step should be as fast as parsing and writing the BAM file. If your BAM file is not as large as several TB, the only reason I can guess is if the server ran out of disk space. Can you check this possibility?
Thanks,
Bo
from rsem.
Hello,
The server has 20 threads and about 60GB memory. For this job I gave rsem 5 threads and in sum 12GB memory. So this should be ok.
I tried rsem now with a smaller sample and with the rsem star alignment. Below are the temp files. It took about 5 hours to build the fastq-files for a bam-file of 3GB. The whole STAR-alignment step tooks 2 hours. What can be the problem that it takes so long?
-rw-rw-r-- 1 lneums jclab 3001763491 Apr 27 09:23 JZ-3.FCC8R8HACXX_L6_IGCCAAT.bam
-rw-rw-r-- 1 lneums jclab 7090 Apr 27 09:23 JZ-3.FCC8R8HACXX_L6_IGCCAATLog.progress.out
-rw-rw-r-- 1 lneums jclab 52225 Apr 27 09:23 JZ-3.FCC8R8HACXX_L6_IGCCAATLog.out
-rw------- 1 jchien jclab 7107497 Apr 27 09:24 JZ-3.FCC8R8HACXX_L6_IGCCAATSJ.out.tab
-rw------- 1 jchien jclab 1867 Apr 27 09:24 JZ-3.FCC8R8HACXX_L6_IGCCAATLog.final.out
-rw------- 1 jchien jclab 0 Apr 27 09:24 JZ-3.FCC8R8HACXX_L6_IGCCAAT.omit
-rw------- 1 jchien jclab 457810116 Apr 27 14:36 JZ-3.FCC8R8HACXX_L6_IGCCAAT_un_2.fq
-rw------- 1 jchien jclab 457810116 Apr 27 14:36 JZ-3.FCC8R8HACXX_L6_IGCCAAT_un_1.fq
-rw------- 1 jchien jclab 277061770 Apr 27 14:36 JZ-3.FCC8R8HACXX_L6_IGCCAAT.dat
-rw------- 1 jchien jclab 2997789796 Apr 27 14:36 JZ-3.FCC8R8HACXX_L6_IGCCAAT_alignable_2.fq
-rw------- 1 jchien jclab 2997789796 Apr 27 14:36 JZ-3.FCC8R8HACXX_L6_IGCCAAT_alignable_1.fq
-rw------- 1 jchien jclab 6032032 Apr 27 14:39 JZ-3.FCC8R8HACXX_L6_IGCCAAT_alignable_1.fq.ridx
-rw------- 1 jchien jclab 6032032 Apr 27 14:41 JZ-3.FCC8R8HACXX_L6_IGCCAAT_alignable_2.fq.ridx
-rw------- 1 jchien jclab 31 Apr 27 14:41 JZ-3.FCC8R8HACXX_L6_IGCCAAT.mparams
-rw------- 1 jchien jclab 739293536 Apr 27 15:53 JZ-3.FCC8R8HACXX_L6_IGCCAAT.ofg
-rw------- 1 jchien jclab 3704242 Apr 27 15:53 JZ-3.FCC8R8HACXX_L6_IGCCAAT.iso_res
-rw------- 1 jchien jclab 3169194 Apr 27 15:53 JZ-3.FCC8R8HACXX_L6_IGCCAAT.gene_res
Thanks
Hecate08
from rsem.
How much time does it take to finish RSEM once you have extracted read files?
from rsem.
@Hecate08 That's so weird. Is it possible that in your server, it is very slow to write things to the disk?
P.S., for the space, I refer to disk space instead of memory.
from rsem.
Sorry, there are 6 TB of diskspace on this server.
The speed of writing to the disk may be the problem. Is there a way to specify a local temp folder instead of writing to the file distribution?
Thanks
from rsem.
@Hecate08, yes, see option --temporary-folder
from rsem.
Related Issues (20)
- how to simulate single-end reads?
- The SAM/BAM file declares less than one reference sequence! HOT 2
- genomeSAsparseD option for RSEM Star?
- Problem implementing a reverse-strand specific analysis HOT 2
- about gffcompare
- RSEM deal with multimapping? HOT 1
- Pearl error
- error when using convert-sam-for-rsem to a RSEM compatible BAM file.
- rsem-prepare-reference arguments
- Using STAR BAMs from genomic alignment HOT 2
- rsem-parse-alignments and rsem-build-read-index
- Gtf file
- cleaned.fq.gz: command not found HOT 3
- Two mates are aligned to two different transcripts HOT 1
- rsem-generate-ngvector command not found HOT 1
- bam only containing aligned reads produced different expression profile
- Not sure if rsem is using --num-threads option...
- fatal error: buffer size for SJ output is too small Solution: increase input parameter --limitOutSJcollapsed HOT 3
- Single-end fragment length mean and SD
- rsem-extract-reference-transcripts fails with "Error Message: Strand is neither '+' nor '-'!"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rsem.