Comments (18)
from chipseq_pipeline.
Thank you so much, Jin. Yes, it is still running. I thought it gets into a dead loop because it takes so long this time. I will keep looking how it goes and let you know. Cheers!
from chipseq_pipeline.
I've had the same issue. I've let it spin for 3 days before to no avail.
from chipseq_pipeline.
Check your job status with sacct -j 1610998
and squeue -j 1610998
. Also check your output directory.
$ ls -l out/qc/*/*
$ ls -l out/align/*/*
from chipseq_pipeline.
$ ls -l out20180516/qc//
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 390 May 18 04:49 out20180516/qc/rep1/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.2.flagstat.qc
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 391 May 18 04:51 out20180516/qc/rep2/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.3.flagstat.qc
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 322 May 18 05:05 out20180516/qc/rep3/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.dup.qc
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 390 May 17 07:20 out20180516/qc/rep3/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.flagstat.qc
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 322 May 18 05:06 out20180516/qc/rep4/Sample_Hues6_P37_DMSO_H3K27ac_GACCGTTG.R1.dup.qc
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 391 May 17 07:22 out20180516/qc/rep4/Sample_Hues6_P37_DMSO_H3K27ac_GACCGTTG.R1.flagstat.qc
$ ls -l out20180516/align//
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 736038176 May 18 04:49 out20180516/align/rep1/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.2.bam
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 3481536 May 18 04:49 out20180516/align/rep1/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.2.bam.bai
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 511004826 May 18 05:07 out20180516/align/rep1/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.2.filt.bam
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 6000520 May 18 05:07 out20180516/align/rep1/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.2.filt.bam.bai
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 559273988 May 18 04:50 out20180516/align/rep2/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.3.bam
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 3979872 May 18 04:51 out20180516/align/rep2/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.3.bam.bai
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 354197738 May 18 05:07 out20180516/align/rep2/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.3.filt.bam
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 5869784 May 18 05:07 out20180516/align/rep2/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.3.filt.bam.bai
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 428556883 May 17 07:20 out20180516/align/rep3/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.bam
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 4201584 May 17 07:20 out20180516/align/rep3/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.bam.bai
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 250005527 May 18 05:05 out20180516/align/rep3/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.dupmark.bam
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 248908658 May 18 04:44 out20180516/align/rep3/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.filt.bam
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 5807680 May 18 04:44 out20180516/align/rep3/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.filt.bam.bai
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 400974621 May 17 07:21 out20180516/align/rep4/Sample_Hues6_P37_DMSO_H3K27ac_GACCGTTG.R1.bam
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 4311208 May 17 07:22 out20180516/align/rep4/Sample_Hues6_P37_DMSO_H3K27ac_GACCGTTG.R1.bam.bai
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 241749196 May 18 05:06 out20180516/align/rep4/Sample_Hues6_P37_DMSO_H3K27ac_GACCGTTG.R1.dupmark.bam
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 240300507 May 18 04:44 out20180516/align/rep4/Sample_Hues6_P37_DMSO_H3K27ac_GACCGTTG.R1.filt.bam
-rw-r--r-- 1 liangma1 scgpm-informatics_chetty 5804072 May 18 04:44 out20180516/align/rep4/Sample_Hues6_P37_DMSO_H3K27ac_GACCGTTG.R1.filt.bam.bai
from chipseq_pipeline.
Check job status, exit code or fail reason first. Please learn about some SLURM command for this (sacct, sinfo, squeue).
If your FASTQS are huge, check if you gave enough memory for mapping tasks (bwa and dedup). Try with higher memory settings (-mem_bwa 30G
or -mem_dedup 30G
according to the stage where pipeline failed). They are 12G by default.
SLURM is very strict about memory usage.
from chipseq_pipeline.
$ sacct -j 1611181
JobID JobName Partition Account AllocCPUS State ExitCode
1611181 sbatch batch chettys 2 COMPLETED 0:0
1611181.bat+ batch chettys 2 COMPLETED 0:0
1611181.ext+ extern chettys 2 COMPLETED 0:0
$ squeue -j 1611181
slurm_load_jobs error: Invalid job id specified
from chipseq_pipeline.
Thank you, Jin. I've killed current job and run it with increased memory setting:
bds_scr h3k27ac_noncellcycle_dmso20180518 ~/TF_chipseq_pipeline/chipseq.bds -se -species hg19 -dup_marker sambamba -nth 4 -type histone -mem_bwa 30G -mem_dedup 30G -mem_macs2 30G -out_dir out20180516 -fastq1 Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.2.fastq.gz -fastq2 Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.3.fastq.gz -fastq3 Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.fastq.gz -fastq4 Sample_Hues6_P37_DMSO_H3K27ac_GACCGTTG.R1.fastq.gz -ctl_fastq1 /labs/chettys/liangma1/test/chipseq/wce/wce_for_dmso2.gz -ctl_fastq2 /labs/chettys/liangma1/test/chipseq/wce/wce_for_dmso3.gz
from chipseq_pipeline.
Hi Jin,
I have 2 jobs is running. but I guess the jobs get into dead loop again, could you take a look?
h3k27ac_noncellcycle_dmso20180518.BDS.log
SRR3554631.BDS.log
from chipseq_pipeline.
from chipseq_pipeline.
Thanks, Jin.
The status below shows all of them are pending, but they are actually keep printing on screen. So what to do to get it normal?
For job h3k27ac_noncellcycle_dmso
##############
21:53:03.741 ExecutionerClusterGeneric 'ClusterGeneric[33]': Tasks [ClusterGeneric[33]] Pending: 0 Running: 4 Done: 0 Failed: 0
| PID | Task state | Task name | Dependencies | Task definition |
| ------- | ----------------- | ------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| 1658231 | running (RUNNING) | markdup_bam_sambamba rep2 | | sambamba markdup -t 1 --hash-table-size=17592186044416; --overflow-list-size=20000000 --io-buffer-size=256 /labs/chettys/liangma1/test/chipseq/h3k27ac_noncellcycle_dmso/out20180516/align/rep2/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.3.filt.bam /labs/chettys/liangma1/test/chipseq/h3k27ac_noncel |
| 1658233 | running (RUNNING) | bwa_aln ctl2 | | bwa aln -q 5 -l 32 -k 2 -t 1 /ifs/scratch/leepc12/pipeline_genome_data/hg19/bwa_index/male.hg19.fa /labs/chettys/liangma1/test/chipseq/wce/wce_for_dmso3.gz > /labs/chettys/liangma1/test/chipseq/h3k27ac_noncellcycle_dmso/out20180516/align/ctl2/wce_for_dmso3.sai; TASKTIME=$[$(date +%s)-${STARTTIME}]; |
| 1658230 | running (RUNNING) | markdup_bam_sambamba rep1 | | sambamba markdup -t 1 --hash-table-size=17592186044416; --overflow-list-size=20000000 --io-buffer-size=256 /labs/chettys/liangma1/test/chipseq/h3k27ac_noncellcycle_dmso/out20180516/align/rep1/Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.2.filt.bam /labs/chettys/liangma1/test/chipseq/h3k27ac_noncel |
| 1658232 | running (RUNNING) | bwa_aln ctl1 | | bwa aln -q 5 -l 32 -k 2 -t 1 /ifs/scratch/leepc12/pipeline_genome_data/hg19/bwa_index/male.hg19.fa /labs/chettys/liangma1/test/chipseq/wce/wce_for_dmso2.gz > /labs/chettys/liangma1/test/chipseq/h3k27ac_noncellcycle_dmso/out20180516/align/ctl1/wce_for_dmso2.sai; TASKTIME=$[$(date +%s)-${STARTTIME}]; |
^C
[liangma1@sgiuv20-rcf-111-32 h3k27ac_noncellcycle_dmso]$ sacct -j 1658231
JobID JobName Partition Account AllocCPUS State ExitCode
1658231 sbatch batch default 1 PENDING 0:0
[liangma1@sgiuv20-rcf-111-32 h3k27ac_noncellcycle_dmso]$ sacct -j 1658233
JobID JobName Partition Account AllocCPUS State ExitCode
1658233 sbatch batch default 1 PENDING 0:0
[liangma1@sgiuv20-rcf-111-32 h3k27ac_noncellcycle_dmso]$ sacct -j 1658230
JobID JobName Partition Account AllocCPUS State ExitCode
1658230 sbatch batch default 1 PENDING 0:0
[liangma1@sgiuv20-rcf-111-32 h3k27ac_noncellcycle_dmso]$ sacct -j 1658232
JobID JobName Partition Account AllocCPUS State ExitCode
1658232 sbatch batch default 1 PENDING 0:0
[liangma1@sgiuv20-rcf-111-32 h3k27ac_noncellcycle_dmso]$
[liangma1@sgiuv20-rcf-111-32 h3k27ac_noncellcycle_dmso]$ squeue -j 1658231
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
1658231 batch sbatch liangma1 PD 0:00 1 (AccountNotAllowed)
[liangma1@sgiuv20-rcf-111-32 h3k27ac_noncellcycle_dmso]$ squeue -j 1658233
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
1658233 batch sbatch liangma1 PD 0:00 1 (AccountNotAllowed)
[liangma1@sgiuv20-rcf-111-32 h3k27ac_noncellcycle_dmso]$ squeue -j 1658230
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
1658230 batch sbatch liangma1 PD 0:00 1 (AccountNotAllowed)
[liangma1@sgiuv20-rcf-111-32 h3k27ac_noncellcycle_dmso]$ squeue -j 1658232
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
1658232 batch sbatch liangma1 PD 0:00 1 (AccountNotAllowed)
For job SRR3554631
##########
[liangma1@sgiuv20-rcf-111-32 test]$ sacct -j 1659097
JobID JobName Partition Account AllocCPUS State ExitCode
1659097 sbatch batch default 1 PENDING 0:0
[liangma1@sgiuv20-rcf-111-32 test]$ squeue -j 1659097
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
1659097 batch sbatch liangma1 PD 0:00 1 (AccountNotAllowed)
from chipseq_pipeline.
from chipseq_pipeline.
Thanks. Yes, I tried to add '--account=chettys', but still having this error. Maybe the position I added is not right?
bds_scr SRR3554631 ~/TF_chipseq_pipeline/chipseq.bds --account=chettys -pe -species hg19 -dup_marker sambamba -nth 4 -type histone -mem_bwa 30G -mem_dedup 30G -mem_macs2 30G -out_dir out -fastq1_1 SRR3554631_1.fastq.gz -fastq1_2 SRR3554631_2.fastq.gz
$ sacct -j 1687688
JobID JobName Partition Account AllocCPUS State ExitCode
1687688 sbatch batch default 1 PENDING 0:0
[liangma1@sgiuv20-rcf-111-32 test]$ squeue -j 1687688
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
1687688 batch sbatch liangma1 PD 0:00 1 (AccountNotAllowed)
from chipseq_pipeline.
from chipseq_pipeline.
Got you. I will change it to '-q' and try it again.
from chipseq_pipeline.
Hi Jin, where should I put -q chettys
in the command? I used the command below but a fetal error popped up, said "Error message : 'sbatch: error: Unable to open file chettys", I've attached the log.
$ bds_scr h3k27ac_noncellcycle_dmso20180519.2 ~/TF_chipseq_pipeline/chipseq.bds -q chettys -se -species hg19 -dup_marker sambamba -nth 4 -type histone -mem_bwa 30G -mem_dedup 30G -mem_macs2 30G -out_dir out20180519.2 -fastq1 Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.2.fastq.gz -fastq2 Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.3.fastq.gz -fastq3 Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.fastq.gz -fastq4 Sample_Hues6_P37_DMSO_H3K27ac_GACCGTTG.R1.fastq.gz -ctl_fastq1 /labs/chettys/liangma1/test/chipseq/wce/wce_for_dmso2.gz -ctl_fastq2 /labs/chettys/liangma1/test/chipseq/wce/wce_for_dmso3.gz
$ bds_scr h3k27ac_noncellcycle_dmso20180519.2 -q chettys ~/TF_chipseq_pipeline/chipseq.bds -se -species hg19 -dup_marker sambamba -nth 4 -type histone -mem_bwa 30G -mem_dedup 30G -mem_macs2 30G -out_dir out20180519.2 -fastq1 Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.2.fastq.gz -fastq2 Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.3.fastq.gz -fastq3 Sample_Hues6_P35_DMSO_H3K27ac_R1_2_TGTCGGAT.R1.fastq.gz -fastq4 Sample_Hues6_P37_DMSO_H3K27ac_GACCGTTG.R1.fastq.gz -ctl_fastq1 /labs/chettys/liangma1/test/chipseq/wce/wce_for_dmso2.gz -ctl_fastq2 /labs/chettys/liangma1/test/chipseq/wce/wce_for_dmso3.gz
h3k27ac_noncellcycle_dmso20180519.2.BDS.log
from chipseq_pipeline.
from chipseq_pipeline.
Thank you, Jin. I've updated BDS and use '-q' as you suggested, and the job status looks good
$ squeue -j 1688140
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
1688140 batch sbatch liangma1 R 1:58 1 sgisummit-rcf-111-08
from chipseq_pipeline.
Related Issues (20)
- aquas_chipseq pipline SPP error HOT 6
- Trimmed data-sets HOT 1
- Read length HOT 2
- shuf: end of file HOT 8
- "-q [SLURM_ACCOUNT_NAME]" interpreted as file HOT 1
- ERROR:root:--extsize must >= 1! HOT 2
- Fatal error: chipseq.bds, line 1354, pos 2. Task/s failed. HOT 4
- [fread] Unexpected end of file (bwa_sai2sam_pe_core) HOT 6
- job submission command HOT 2
- [Error] two same tag align file in (z)cat and shuf in pipe HOT 1
- Missing `-c` for `macs2 peakcall` ? HOT 1
- pipeline deprecated? HOT 2
- Can't download hg19 genome data HOT 2
- Some reps get ERROR:root:--extsize must >= 1! HOT 2
- Documentation for macs2? HOT 2
- Citing the AQUAS Pipeline HOT 1
- ERROR:root:--extsize must >= 1! and chipseq.bds can not get all replicates go through HOT 25
- mm10 HOT 1
- JSD error: "RuntimeError: module compiled against API version 0xc but this version of numpy is 0xa" HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chipseq_pipeline.