Comments (6)
katsikora
commented on July 29, 2024
1
there is also this thing that I do not know if it is relevant:
/scratch/tgeorgom/mamba/snakepipes_devel/lib/python3.11/site-packages/snakePipes/workflows/ChIP-seq/internals.snakefile:145: UserWarning: 16 out of 16 have a matching control warnings.warn( str(len(chip_samples_w_ctrl)) + " out of " + str(len(chip_samples)) + " have a matching control ")
That's not related. It's just a warning message if control samples were found in the chipdict yaml file or not. Saves a lot of headache with some cutNx experiments.
from snakepipes.
katsikora
commented on July 29, 2024
Hi Theodoros,
I'm sorry to hear that plot fingerpring fails for you. Why do you think it should take more than 6 hours? Ideally, it should be done within minutes. Perhaps increasing the memory resource would help it run faster on your system?
The zombie jobs are not unusual with anything written in python.
Best wishes,
Katarzyna
from snakepipes.
sunta3iouxos
commented on July 29, 2024
Hi Katarzyna,
this is the error I get. Notice that the provided memory is: mem_mb=115345, disk_mb=115345
---- This analysis has been done using snakePipes version 2.8.1 ----
Sample sheet found and header is ok!
/scratch/tgeorgom/mamba/snakepipes_devel/lib/python3.11/site-packages/snakePipes/workflows/ChIP-seq/internals.snakefile:145: UserWarning: 16 out of 16 have a matching control
warnings.warn( str(len(chip_samples_w_ctrl)) + " out of " + str(len(chip_samples)) + " have a matching control ")
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 24
Rules claiming more threads will be scaled down.
Provided resources: mem_mb=115345, disk_mb=115345
Select jobs to execute...
[Tue Apr 16 09:32:35 2024]
rule plotFingerprint:
input: filtered_bam/A006200376_218434_S7_L001.filtered.bam, filtered_bam/A006200376_218437_S8_L001.filtered.bam, filtered_bam/A006200376_218439_S9_L001.filtered.bam, filtered_bam/A006200376_218442_S10_L001.filtered.bam, filtered_bam/A006200376_218444_S11_L001.filtered.bam, filtered_bam/A006200376_218446_S12_L001.filtered.bam, filtered_bam/A006200376_218448_S13_L001.filtered.bam, filtered_bam/A006200376_218450_S14_L001.filtered.bam, filtered_bam/A006200376_218452_S15_L001.filtered.bam, filtered_bam/A006200376_218454_S16_L001.filtered.bam, filtered_bam/A006200376_218456_S17_L001.filtered.bam, filtered_bam/A006200376_218458_S18_L001.filtered.bam, filtered_bam/A006200376_218460_S19_L001.filtered.bam, filtered_bam/A006200376_218462_S20_L001.filtered.bam, filtered_bam/A006200376_218464_S21_L001.filtered.bam, filtered_bam/A006200376_218466_S22_L001.filtered.bam, filtered_bam/A006200376_218468_S23_L001.filtered.bam, filtered_bam/A006200376_218470_S24_L001.filtered.bam, filtered_bam/A006200376_218472_S25_L001.filtered.bam, filtered_bam/A006200376_218474_S26_L001.filtered.bam, filtered_bam/A006200376_218434_S7_L001.filtered.bam.bai, filtered_bam/A006200376_218437_S8_L001.filtered.bam.bai, filtered_bam/A006200376_218439_S9_L001.filtered.bam.bai, filtered_bam/A006200376_218442_S10_L001.filtered.bam.bai, filtered_bam/A006200376_218444_S11_L001.filtered.bam.bai, filtered_bam/A006200376_218446_S12_L001.filtered.bam.bai, filtered_bam/A006200376_218448_S13_L001.filtered.bam.bai, filtered_bam/A006200376_218450_S14_L001.filtered.bam.bai, filtered_bam/A006200376_218452_S15_L001.filtered.bam.bai, filtered_bam/A006200376_218454_S16_L001.filtered.bam.bai, filtered_bam/A006200376_218456_S17_L001.filtered.bam.bai, filtered_bam/A006200376_218458_S18_L001.filtered.bam.bai, filtered_bam/A006200376_218460_S19_L001.filtered.bam.bai, filtered_bam/A006200376_218462_S20_L001.filtered.bam.bai, filtered_bam/A006200376_218464_S21_L001.filtered.bam.bai, filtered_bam/A006200376_218466_S22_L001.filtered.bam.bai, filtered_bam/A006200376_218468_S23_L001.filtered.bam.bai, filtered_bam/A006200376_218470_S24_L001.filtered.bam.bai, filtered_bam/A006200376_218472_S25_L001.filtered.bam.bai, filtered_bam/A006200376_218474_S26_L001.filtered.bam.bai
output: deepTools_ChIP/plotFingerprint/plotFingerprint.metrics.txt
log: deepTools_ChIP/logs/plotFingerprint.out, deepTools_ChIP/logs/plotFingerprint.err
jobid: 0
benchmark: deepTools_ChIP/.benchmark/plotFingerprint.benchmark
reason: Missing output files: deepTools_ChIP/plotFingerprint/plotFingerprint.metrics.txt, deepTools_ChIP/.benchmark/plotFingerprint.benchmark
threads: 24
resources: mem_mb=115345, disk_mb=115345, tmpdir=/scratch/tgeorgom_temp
Activating conda environment: ../../mamba/snakepipes_devel/envs/c21237e8c0954c1620210ed0f7ba2e63_
slurmstepd: error: *** JOB 18773077 ON cheops11804 CANCELLED AT 2024-04-16T15:31:45 DUE TO TIME LIMIT ***from snakepipes.
sunta3iouxos
commented on July 29, 2024
there is also this thing that I do not know if it is relevant:
/scratch/tgeorgom/mamba/snakepipes_devel/lib/python3.11/site-packages/snakePipes/workflows/ChIP-seq/internals.snakefile:145: UserWarning: 16 out of 16 have a matching control
warnings.warn( str(len(chip_samples_w_ctrl)) + " out of " + str(len(chip_samples)) + " have a matching control ")from snakepipes.
katsikora
commented on July 29, 2024
345
Hi,
am I reading this correctly, that rule plotFingerprint is given 100Gb memory? That's a lot. I believe, by default, it should get 1Gb.
Did you modify the memory resource yourself?
Best,
Katarzyna
from snakepipes.
sunta3iouxos
commented on July 29, 2024
345Hi,
am I reading this correctly, that rule plotFingerprint is given 100Gb memory? That's a lot. I believe, by default, it should get 1Gb. Did you modify the memory resource yourself?
Best,
Katarzyna
regarding the memory, I increased it just in case.
This what I got from another run of the same bam files, BUT in this case I used mm38 instead of mm39.
s h:m:s max_rss max_vms max_uss max_pss io_in io_out mean_load cpu_time
3611.4827 1:00:11 4608.45 88808.61 2212.22 2299.83 0.00 0.00 1649.02 357.03
/scratch/tgeorgom/PAV11/bamGRnCm38/deepTools_ChIP/.benchmark/plotFingerprint.benchmark (END)
with the following settings from lib/python3.11/site-packages/snakePipes/shared/cluster.yaml:
plotFingerprint:
memory: 3G
and from lib/python3.11/site-packages/snakePipes/shared/defaults.yaml
max_thread: 24
from snakepipes.
Related Issues (20)
- add allelic-counting mode to mRNA seq
- fix SEACR with control
- error in ChIP-seq HOT 2
- update the snakePipes installed from the github
- Error in mRNA-seq allelic-mapping when using custom genome
- mRNA-seq: sambamba markdup fails because "too many open files"
- [Question] where to find the exact command issued? HOT 2
- [Request] looping inputs for multiple ChIP-seq comparisons HOT 1
- allow user to specify mincount for filtering windows for DB with CSAW
- snakePipes createEnvs fails HOT 1
- add support for custom model formula to mRNAseq and other workflows HOT 2
- DNA-mapping allelic mode - NMaskindex / SNPfile combination
- error in createIndices: module 'pulp' has no attribute 'list_solvers'. Did you mean: 'listSolvers'? HOT 3
- snakePipes envInfo: AttributeError: 'NoneType' object has no attribute 'encode' HOT 4
- conda and "strict channel priorities" HOT 2
- allow for custom bed to be used as input to CSAW for DB
- fix nearest Gene annotation on CSAW_DBR
- Genrich -e option addition
- add peak qc for SEACR as for MACS2 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from snakepipes.