khuang28jhu / bs3 Goto Github PK

Hi,

I installed BS3 according to the instructions, and when I tried checking if it worked, I got an error while trying to run bs3-builder:
~/app/bs3$ ./bs3-build.py -h
Traceback (most recent call last):
File "/home/juaguila/app/bs3/./bs3-build.py", line 5, in
from bs_index.wg_build import *
File "/home/juaguila/app/bs3/bs_index/wg_build.py", line 49
chrom_num += 1
TabError: inconsistent use of tabs and spaces in indentation

Is there any step besides - git clone https://github.com/khuang28jhu/bs3 - ??

Thanks;

Hi,
While trying to build an index for bs3 the bs3-build.py failed with the error message:

ERROR: Cannot file program snap for execution.

I was calling the bs3-build.py from another folder and added the bs3 folder (and the snap folder) to my PATH before.
I am not whether this is supported, because simply adding the snap executable to the bs3 folder where I checked out the bs3 git repo, was not enough.
I had to change line 65 of bs3-build.py to:

65 builder_exec = 'snap'

and disable the code in lines 74,75 to make it work. I think it would be good to update the script(s) to allow calling bs3 from anywhere.

Thanks,
Marcel

Hi,
I ran ./bs3/bs3-align.py -i FASTQ_FILE -g FASTA_FILE but bs3 was not running and no error was reported. I cloned bs3 today (Apr 10). Any idea about this issue? Please let me know if you need more information. Thanks!

Yupeng

When I tried to index the genome reference the following error appears:

Traceback (most recent call last):
File "bs3-build.py", line 5, in
from bs_index.wg_build import *
File "bs_seeker3/bs_index/wg_build.py", line 49
chrom_num += 1
^
TabError: inconsistent use of tabs and spaces in indentation

Hi,

I am trying to index this genome:
ftp://ftp.ensembl.org/pub/release-91/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz

I am using this command line:
./bs3-build -f /mnt/data/Genomes/hg38/Homo_sapiens.GRCh38.dna.primary_assembly.fa -s 20 -L 8

I get this error:

Bias computation: 6000000000 / 6199695936
Bias computation: 6100000000 / 6199695936
Computed bias table in 20s
Allocating memory for hash tables...terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)
[Done] Last: 0:02:50.279478     Total: 0:04:45.361098
Build Whole genome preprocessing
--- 285.360810041 seconds ---

Also tried:
s:25, L:5 --> error
s:25, L:8 --> error

System specs:
threads: 64
RAM: 728 Gb
Any ideas of what could be wrong?

Thanks,
Sol

Hi, I ran bs3 with your test data without any problem. But when I'm trying to index hg38, indexing done without any error ([Done] Build Whole genome preprocessing) while align gave "Unable to open file 'reference_genome/hg38.fa_snap/W_C2T/GenomeIndex". I checked the hg38.fa_snap folder and there is no file named as GenomeIndex. So I'm guessing there is something wrong with index. Is there anyway to check what's the problem?

I used bs3-call_methylation with alignment from test data, but got error message as follow,

 BS-Seeker3 beta

[bam_sort] Use -T PREFIX / -o FILE to specify temporary and final output files
Usage: samtools sort [options...] [in.bam]
Options:
-l INT Set compression level, from 0 (uncompressed) to 9 (best)
-m INT Set maximum memory per thread; suffix K/M/G recognized [768M]
-n Sort by read name
-t TAG Sort by value of TAG. Uses position as secondary index (or read name if -n is set)
-o FILE Write final output to FILE rather than standard output
-T PREFIX Write temporary files to PREFIX.nnnn.bam
--input-fmt-option OPT[=VAL]
Specify a single input file format option in the form
of OPTION or OPTION=VALUE
-O, --output-fmt FORMAT[,OPT[=VAL]]...
Specify output format (SAM, BAM, CRAM)
--output-fmt-option OPT[=VAL]
Specify a single output file format option in the form
of OPTION or OPTION=VALUE
--reference FILE
Reference sequence FASTA FILE [null]
-@, --threads INT
Number of additional threads to use [0]
[E::bgzf_flush] File write failed (wrong size)
samtools view: writing to standard output failed: Broken pipe
[E::bgzf_close] File write failed
samtools view: error closing standard output: -1
[2019-01-31 16:53:54] indexing sorted alignments
[E::hts_open_format] Failed to open file output.bam
samtools index: failed to open "output.bam": No such file or directory
[2019-01-31 16:53:54] calculating methylation levels
[E::hts_open_format] Failed to open file output.bam
Traceback (most recent call last):
File "bs3-call_methylation.py", line 128, in
sorted_input = pysam.Samfile('output.bam', 'rb')
File "pysam/libcalignmentfile.pyx", line 736, in pysam.libcalignmentfile.AlignmentFile.cinit
File "pysam/libcalignmentfile.pyx", line 935, in pysam.libcalignmentfile.AlignmentFile._open
IOError: [Errno 2] could not open alignment file output.bam: No such file or directory

command used
./bs3-align -1 test_data/pair1.fq -2 test_data/pair2.fq -o WGBC -f sam -g reference_genome/genome.fa -d reference_genome/

error I got:

File "bs3-align.py", line 149
print "[Error] Cannot open input file : %s" % read_file
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("[Error] Cannot open input file : %s" % read_file)?

Hi @khuang28jhu ,

In the README, you said the required version of SNAP is version 1.0beta.23. However, the current release of SNAP is only 1.0beta.18.

Best,

Kai

Hello!

Im trying to use BS-seeker3 for my experiment. Unfortunately, I am getting an error "Loading index from directory... Killed" and also no output is produced despite successful (?) alignment. I know that others have raised this issue, but I figured my case may be a bit different.

SOME RELEVANT INFORMATION:

• ps3_build seemed to work well, without any problems.
• My reads are paired, around 75 bp of length.
• I am working on Singularity-Ubuntu:18.04
• Seed size is 16.
• I have about 200GB RAM available
• Command that I used:

python ./bs3-align.py --snap-t 10 -1 ../read1.fq_trimmed.fq.gz -2 ../read2.fq_trimmed.fq.gz -g mm10ucsc.fa -f sam -o ./FullBS3test.sam -d reference_genome/

THE OUTPUT:

     BS-Seeker3 beta

Welcome to SNAP version 1.0dev.97.

Loading index from directory... Welcome to SNAP version 1.0dev.97.

Loading index from directory... Welcome to SNAP version 1.0dev.97.

Loading index from directory... Welcome to SNAP version 1.0dev.97.

Loading index from directory... Welcome to SNAP version 1.0dev.97.

Loading index from directory... Welcome to SNAP version 1.0dev.97.

Loading index from directory... Welcome to SNAP version 1.0dev.97.

Loading index from directory... Welcome to SNAP version 1.0dev.97.

Loading index from directory... Killed
Killed
625s.  1166842752 bases, seed size 16
436s.  1166842752 bases, seed size 16
562s.  1166842752 bases, seed size 16
688s.  1166842752 bases, seed size 16
Aligning.
Aligning.
Aligning.
Aligning.
Killed
619s.  1166842752 bases, seed size 16
668s.  1166842752 bases, seed size 16
Aligning.
Aligning.
Killed
619s.  1166842752 bases, seed size 16
668s.  1166842752 bases, seed size 16
Aligning.
Aligning.
Killed
Total Reads    Aligned, MAPQ >= 10    Aligned, MAPQ < 10     Unaligned              Too Short/Too Many Ns  %Pairs    Reads/s   Time in Aligner (s)
4,800,256      2,927,452 (60.99%)     543,465 (11.32%)       582,544 (12.14%)       746,795 (15.56%)       29.40%    1,414     3,394
(None, None)
Total Reads    Aligned, MAPQ >= 10    Aligned, MAPQ < 10     Unaligned              Too Short/Too Many Ns  %Pairs    Reads/s   Time in Aligner (s)
6,400,000      3,905,781 (61.03%)     722,389 (11.29%)       778,360 (12.16%)       993,470 (15.52%)       29.39%    1,550     4,126
Total Reads    Aligned, MAPQ >= 10    Aligned, MAPQ < 10     Unaligned              Too Short/Too Many Ns  %Pairs    Reads/s   Time in Aligner (s)
6,400,000      3,887,135 (60.74%)     725,278 (11.33%)       776,962 (12.14%)       1,010,625 (15.79%)     29.07%    1,549     4,132
Total Reads    Aligned, MAPQ >= 10    Aligned, MAPQ < 10     Unaligned              Too Short/Too Many Ns  %Pairs    Reads/s   Time in Aligner (s)
6,400,000      3,902,420 (60.98%)     724,141 (11.31%)       778,564 (12.17%)       994,875 (15.54%)       29.51%    1,634     3,916
(None, None)
(None, None)
(None, None)
(None, None)
(None, None)
(None, None)
(None, None)

        BS-seeker3 Result

        Final Alignment Report
================================================

Number of reads in total: 49600256
Number of unique-hits reads (before post-filtering): 0
Number of reads mapped after post-filtering 0

Alignment Time: 5107.08876896secs

        Final Cytosine Report
================================================

Total Number of Cytosines: 0
Total Number of Cs in CpG context: 0
Total Number of Cs in CHG context: 0
Total Number of Cs in CHH context: 0

Rate of Methylation
 mCG  0.000%
 mCHG  0.000%
 mCHH  0.000%

I will be thankful for response!

Best
Adrian

Hi~ boy
i want to use this soft . run align afte built index, but can not any message..
just like the below:
#######################
python bs3-align.py
-1 VP_Liu_C1_RDML00187_BHCLCYCCXY_L1_1.clean.fq.gz
-2 VP_Liu_C1_RDML00187_BHCLCYCCXY_L1_2.clean.fq.gz
-g ucsc.hg19.fasta

 BS-Seeker3 beta

#######################
just got the message---BS-Seeker3 beta
what can i do for it ????????????????????????????

I tried to align fastq-files with BS-Seeker3 but no 1 read was mapped.

$ ./bs3-align -1 B213-036-pool_S6_L001_R1_001.fastq -2 B213-036-pool_S6_L001_R2_001.fastq -o B213-036 -f sam -g reference_genome/genome.fa -d reference_genome/

     BS-Seeker3 beta

Welcome to SNAP version 1.0.0.

Loading index from directory... 0s.  60856842 bases, seed size 20
Aligning.
Total Reads    Aligned, MAPQ >= 10    Aligned, MAPQ < 10     Unaligned              Too Short/Too Many Ns  %Pairs    Reads/s   Time in Aligner (s)
398,926        0 (0.00%)              0 (0.00%)              398,926 (100.00%)      0 (0.00%)              0.00%     60,979    7                   
(None, None)


	BS-seeker3 Result

	Final Alignment Report
================================================

Number of reads in total: 398926
Number of unique-hits reads (before post-filtering): 0.0
Number of reads mapped after post-filtering 0.0

Alignment Time: 9.74573802948secs

	Final Cytosine Report
================================================

Total Number of Cytosines: 0.0
Total Number of Cs in CpG context: 0.0
Total Number of Cs in CHG context: 0.0
Total Number of Cs in CHH context: 0.0

Rate of Methylation
 mCG  0.000%
 mCHG  0.000%
 mCHH  0.000%



$ 

Hi, @khuang28jhu

when snap aligning reads, everthing seems ok, as below:

as you can see, the align ratio for reads input is high, but the sam file is empty, there is no aligned reads, so the methylation information is empty too:

Honestly speaking, I'm very pretty interested in your new algorithm. But at now your scripts are not robust enough for use. Since I cloned the scripts and added the PATH to .bashrc, I'd been tortured by directory issues. I need to fix them one by one, then finally when I did QC for lambda, I gave up. Look at the code below as an example, it really drives me crazy:

align_cmd = 'python bs3-align.py -i ' + options.input  + '  -o lamdba_unconversion -g ' +   options.genome
call_methyl_cmd = 'python bs3-call_methylation.py -i lamdba_unconversion -o lamda_unconversion  --db bs_align/reference_genomes/lamdba.fa_snap/'
print_graph_cmd =  'python bs3-methyl_display.py -u y -m lamda_unconversion.CGmap.gz'

1. First, I cannot run the script under the installation directory, so it is impossible to find the script bs3-align.py under the current directory. This is a universal problem in bs3;
2. The output temp file uses one fixed name called "lambda_unconversion" to save the aligned result. But what if I running in batches? They would clash others ruthlessly. Isn't it common sense?
3. Even if I bypass these two problems, STILL I can't get my aligned result using align_cmd. It just output:
   BS-Seeker3 beta
   And the aligned script, bs3-align.py , is too complicated for me to fix it. I've lost all my patience and here I do wish you could optimize your software and make it more user-friendly!

Thx!!!!

yours,
Gran

hi
I used bs3 and every things goes well as follow. But at the end sam file is empty. I found out some other people faced to the same problem. I think that the main code has some problem. Let me know can it be my fault.

python '/home/bs3/bs3-align.py' -1 '/home/Data/R1_1.fq.gz' -2 '/home/Data/R1_2.fq.gz' -g '/home/ARS-UCD1/bt_ref_ARS-UCD1.2.fa' -d '/media/reference_genome' -o aex

 BS-Seeker3 beta

Welcome to SNAP version 1.0beta.23.

Loading index from directory... 59s. 961853050 bases, seed size 20
Aligning.
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns %Pairs Reads/s Time in Aligner (s)
400,000 325,572 (81.39%) 36,032 (9.01%) 27,723 (6.93%) 10,673 (2.67%) 45.33% 78,724 5
(None, None)

BS-seeker3 Result

Final Alignment Report

================================================

Number of reads in total: 400000
Number of unique-hits reads (before post-filtering): 0
Number of reads mapped after post-filtering 0

Alignment Time: 79.7409200668secs

Final Cytosine Report

================================================

Total Number of Cytosines: 0
Total Number of Cs in CpG context: 0
Total Number of Cs in CHG context: 0
Total Number of Cs in CHH context: 0

Rate of Methylation
mCG 0.000%
mCHG 0.000%
mCHH 0.000%

i have a question:
when i run bs3-align.py, a bug will open

python: can't open file 'bs_align/bs_single_end3.py': [Errno 2] No such file or directory
python: can't open file 'bs_align/bs_single_end3.py': [Errno 2] No such file or directory
python: can't open file 'bs_align/bs_single_end3.py': [Errno 2] No such file or directory
rm: cannot remove `temp-WGBS-_sam-': No such file or directory

I'm attempting to align a .fq file, containing simulated reads with bisulfite conversion. The file was produced using the tools wgsim [1], for creating the actual reads, and fastx-mutation-tools [2], for adding bisulfite conversion. The reference genome is the human genome (v37) [3]. The reads are 100 base-pairs long. The file to be aligned contains 1000002 reads. The index built by bs3 is 130GB big. The result logs of the attempted alignment are as follows:

Welcome to SNAP version 2.0.3.

Loading index from directory... 17s. 6,203,947,478 bases, seed size 20.
Aligning.
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns Reads/s Time in Aligner (s)
1,000,002 930,826 (93.08%) 62,075 (6.21%) 7,101 (0.71%) 0 (0.00%) 83,570 12
corrupted size vs. prev_size

The latter "Final Alignment Report" reports only 0 unique-hits reads, 0 reads mapped after post-filtering, etc.

The server I'm running bs3 on has 125GB of effective RAM. Therefore, I assume that this is the issue why bs3 doesn't work. Interestingly enough however, it works with a small file of reads, like 2000 reads in total. So even though, the available RAM on my server is smaller than the index size, the alignment process works. Therefore, I would like to ask how much RAM is needed for bs3 to work with a big file of reads like the one mentioned in beginning of this post?

[1] https://github.com/lh3/wgsim
[2] https://github.com/nicolaprezza/fastx-mutate-tools
[3] https://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.gz

The application output sam format only, the option for bam file exist but wasn't implemented.
-f bam doesn't work.

The sam file has an error too, Any buddy understand this line.... Look like the same read map to two chromosome and written to the same line....

NS500599:122:HYY5WBGX2:3:12410:21586:16399 0 4 83813929 70 93M * 0 0 TTTATGAGGGAGGTATTNS500599:121:HYVLGBGX2:4:13606:12721:18500 0 11 126043545 70 112M * 0 0 TGTTAAGGAGTTTTAATTTTATTTTTAGGTGATATGTTGTTATTGGAGGATTTATTTGGGAAGTGTTTAGGTTATATGTGTATTTTAGAAAGATTTTTTTGATTTTATTGGA EEEEEE/EEEEEEEEEEEEEEEEEEAEEAAAEEEEEEEEE/EEEEEEEAE/EEEEEEAEEEEEEEEEEEE<AAEEAEEEAEEEAEE<AEEEEEEEEEEEEEEE<EEAEEA<A PG:Z:SNAP NM:i:0 RG:Z:FASTQ PL:Z:Illumina PU:Z:pu LB:Z:lb SM:Z:sm

I am having problems configuring it and building a docker container for it, probably due to the antient python2, maybe you can consider moving to python3?
I am also confused why bseeker2 was updated more recently than bseeker3?

SNAP can take unaligned SAM/BAM files as input, which is nice to handle metadata tags that have been added prior to sequencing. It would be good if BS-Seeker3 would expose this option of SNAP.

Hi,
I am analyzing WGBS data using bs-seeker3. While building the index of the genome, I get "ERROR: Cannot file program ./snap for execution." Then I checked that SNAP is not install, then I install SNAP(version 2006-07-28), however, I still get the same error message. I am writing to seeking help, How could I fix this error? I have changed the python to version 2.7.5. Here my code "/ifs1/Grp6/furuiying/bs3/bs3-build.py -f /ifs1/Grp6/furuiying/Genome-bs3/Ciona_intestinalis.KH.dna.toplevel.fa”
Best regards,
Ruiying Fu

I simply cannot seem to resolve this. Does anyone know how to trouble shoot this issue?
I have tried increasing the number of thread and cores but still nothing unfortunately.

I doubt this is an issue with the snap aligner. Here is the problem, 250G ram, 56 core Linux machine. Out of memory. I did change the temp directory in bashrc to fix if is related to to the temp space, that didn't fix. Changed to single core, still run out of memory.

`bs3-align -i $SAMDIR/full.21421L_S4_R1_001_trimmed.fq.gz -o $SAMDIR/full.21421L_S4_R1_001.bam -f bam -d $GHOME -g hg38_r87.fa --snap-t 10 --snap--b --snap-so

 BS-Seeker3 beta

Welcome to SNAP version 1.0beta.18.

Loading index from directory... Welcome to SNAP version 1.0beta.18.

Loading index from directory... Welcome to SNAP version 1.0beta.18.

Loading index from directory... Welcome to SNAP version 1.0beta.18.

Loading index from directory... Welcome to SNAP version 1.0beta.18.

Loading index from directory... Welcome to SNAP version 1.0beta.18.

Loading index from directory... Welcome to SNAP version 1.0beta.18.

Loading index from directory... Welcome to SNAP version 1.0beta.18.

Loading index from directory... Welcome to SNAP version 1.0beta.18.

Loading index from directory... Welcome to SNAP version 1.0beta.18.

Loading index from directory... mmap: Cannot allocate memory
SNAP exited with exit code 1 from line 394 of file SNAPLib/BigAlloc.cpp
`

when i run the ./bs3-call_methylation using test data, there have a question.
please tell me why? thanks
Samtools-htslib-API: bam_index_build2() not yet implemented
Traceback (most recent call last):
File "/TJNAS01/CANCER/RD/TEST/chendongdong/bs3/bs3-call_methylation.py", line 163, in
for col in sorted_input.pileup():
File "pysam/calignmentfile.pyx", line 1087, in pysam.calignmentfile.AlignmentFile.pileup (pysam/calignmentfile.c:12475)
ValueError: no index available for pileup

The command ran was:

python bs3-build.py -f /file/location/hg38.fa

The error received:

File "bs3-build.py", line 5, in
from bs_index.wg_build import *
File "/gpfs/home/jmm498/software/bs3-dev1/bs_index/wg_build.py", line 49
chrom_num += 1
^
TabError: inconsistent use of tabs and spaces in indentation

The reference genome was recently downloaded and there are no issues utilizing it to create an index with other software packages.

Thank you for your help in advance!

Thanks for the wonderful tool to deal with methylation datasets.

I wonder whether bs3 is suitable for PBAT library? I noticed that bs2 can handle PBAT datasets. I wonder whether bs3 is also OK with PBAT datasets?

Looking forward to your reply!

Best wishes,

Zorro
20190619

Hi,
I built the genome without any issues and am now trying to align some reads. Unfortunately, bs3-align is just printing the version and exiting. Is there a verbose option to see what is happening?

Here are my commands:
bs3-build -f Mmul_8.0.1_chrom_only.fa --rrbs --cut-site=MspI

bs3-align -g Mmul_8.0.1_chrom_only.fa -1 LID_100316.R1.fq.gz -2 LID_100316.R2.fq.gz -m 0.08 -o $sample.bs3 --rrbs -c C-CGG

This 2nd command returns:
BS-Seeker3 beta

Any idea what is going on?

Hi,
these days i analysis methylseq data.
and i count bases(C/T, G/A) of target region.

but some reads are vanished in output bam file.
so i guess, these reads mapped multiple region.

i want to extract multiple region that some reads are mapped to.

what can i do?

Best regards.
Jeongmin

Hi,
I did bs3-build as "./bs3-build -f ./reference_genome/Sscrofa11.fa --aligner=snap". This command went well.
However, I can not find the index file in "./reference_genome/Sscrofa11.fa_snap/W_G2A and W_C2T".

Thus "./bs3-align.py" said that
"Loading index from directory... Unable to open file './reference_genome/Sscrofa11.fa_snap/W_C2T/GenomeIndex' for read.
Index load failed, aborting."

"Loading index from directory... Unable to open file './reference_genome/Sscrofa11.fa_snap/W_G2A/GenomeIndex' for read.
Index load failed, aborting."
So How should I do to solve this problem.
Look forward to hearing from you soon.

Hi,
I am running bs3-align.py and I am wondering if one can set the number of threads to use? Currently it seems that 2 SNAP threads are running. I could not find an option in the readme or option output of bs3-align.py.

Thanks for the help,
Marcel

hi I am trying to use BS-seeker 3 but non of my reads are getting aligned. Here is the command and screen output

./bs3-align -1 14083.FCHTYN2BBXX_L5_R1_IAAGAGG.fastq.gz -2 14083.FCHTYN2BBXX_L5_R2_IAAGAGG.fastq.gz -g GCA_000001405.15_GRCh38_no_alt_analysis_set.fa -d $PWD/reference_genome/ -o 14083 -f bam

 BS-Seeker3 beta

Welcome to SNAP version 1.0beta.23.

Loading index from directory... 30s. 1905073286 bases, seed size 20
Aligning.
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns %Pairs Reads/s Time in Aligner (s)
6,400,000 0 (0.00%) 0 (0.00%) 0 (0.00%) 6,400,000 (100.00%) 0.00% 3,426,124 2
Welcome to SNAP version 1.0beta.23.

Loading index from directory... 29s. 1905073286 bases, seed size 20
Aligning.
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns %Pairs Reads/s Time in Aligner (s)
6,400,000 0 (0.00%) 0 (0.00%) 0 (0.00%) 6,400,000 (100.00%) 0.00% 3,418,803 2
Welcome to SNAP version 1.0beta.23.

Loading index from directory... 29s. 1905073286 bases, seed size 20
Aligning.
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns %Pairs Reads/s Time in Aligner (s)
6,400,000 0 (0.00%) 0 (0.00%) 0 (0.00%) 6,400,000 (100.00%) 0.00% 3,154,263 2
Welcome to SNAP version 1.0beta.23.

Loading index from directory... 28s. 1905073286 bases, seed size 20
Aligning.
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns %Pairs Reads/s Time in Aligner (s)
6,400,000 0 (0.00%) 0 (0.00%) 0 (0.00%) 6,400,000 (100.00%) 0.00% 3,373,748 2
Welcome to SNAP version 1.0beta.23.

Loading index from directory... 29s. 1905073286 bases, seed size 20
Aligning.
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns %Pairs Reads/s Time in Aligner (s)
6,400,000 0 (0.00%) 0 (0.00%) 0 (0.00%) 6,400,000 (100.00%) 0.00% 3,324,675 2
Welcome to SNAP version 1.0beta.23.

Loading index from directory... 29s. 1905073286 bases, seed size 20
Aligning.
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns %Pairs Reads/s Time in Aligner (s)
6,400,000 0 (0.00%) 0 (0.00%) 0 (0.00%) 6,400,000 (100.00%) 0.00% 3,422,459 2
Welcome to SNAP version 1.0beta.23.

Loading index from directory... 27s. 1905073286 bases, seed size 20
Aligning.
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns %Pairs Reads/s Time in Aligner (s)
6,400,000 0 (0.00%) 0 (0.00%) 0 (0.00%) 6,400,000 (100.00%) 0.00% 3,497,267 2
Welcome to SNAP version 1.0beta.23.

Loading index from directory... Welcome to SNAP version 1.0beta.23.

Loading index from directory... 28s. 1905073286 bases, seed size 20
Aligning.
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns %Pairs Reads/s Time in Aligner (s)
6,400,000 0 (0.00%) 0 (0.00%) 0 (0.00%) 6,400,000 (100.00%) 0.00% 3,300,670 2
27s. 1905073286 bases, seed size 20
Aligning.
Total Reads Aligned, MAPQ >= 10 Aligned, MAPQ < 10 Unaligned Too Short/Too Many Ns %Pairs Reads/s Time in Aligner (s)
2,183,328 0 (0.00%) 0 (0.00%) 0 (0.00%) 2,183,328 (100.00%) 0.00% 2,166,000 1
(None, None)
(None, None)
(None, None)
(None, None)
(None, None)
(None, None)
(None, None)
(None, None)
(None, None)

BS-seeker3 Result

Final Alignment Report

================================================

Number of reads in total: 53383328
Number of unique-hits reads (before post-filtering): 0.0
Number of reads mapped after post-filtering 0.0

Alignment Time: 460.614155054secs

Final Cytosine Report

================================================

Total Number of Cytosines: 0.0
Total Number of Cs in CpG context: 0.0
Total Number of Cs in CHG context: 0.0
Total Number of Cs in CHH context: 0.0

Rate of Methylation
mCG 0.000%
mCHG 0.000%
mCHH 0.000%

After I installed and placed snap in the same directory as suggested, I got this error

ERROR: Cannot file program ./snap for execution.

I have a server with 256GB of RAM, and two ~127GB FASTQ files. It seems that once you start creating Trimed files, they're entirely in-core, and eventually exhaust system memory.

There's also sort of an unbound limit to the number of snap processes you'll spawn, and snap (even with this -map flag) will devour 64GB+ per hg38 index load.

I can of course get access to larger servers, but you might want to issue a warning of some kind, or implement out of core resource access.

Thanks for your work here!

I am getting the following error when running the paired-end alignment command provided in your markup file. It's c;learly a SNAP error but I want to know if there's a way to bypass or increase memory allocation.
Thank you

command Used:
./bs3-align -1 test_data/pair1.fq -2 test_data/pair2.fq -o WGBC -f sam -g reference_genome/genome.fa -d reference_genome/

Error:

BS-Seeker3 beta

Welcome to SNAP version 1.0beta.18.

Loading index from directory... 0s. 60856842 bases, seed size 20
Aligning.
BigAllocator: allocating too much memory, 6329296 > 6329252
SNAP exited with exit code 1 from line 489 of file SNAPLib/BigAlloc.cpp
(None, None)

    BS-seeker3 Result

    Final Alignment Report

================================================

Number of reads in total: 20000
Number of unique-hits reads (before post-filtering): 0.0
Number of reads mapped after post-filtering 0.0

Alignment Time: 0.88127207756secs

    Final Cytosine Report

================================================

Total Number of Cytosines: 0.0
Total Number of Cs in CpG context: 0.0
Total Number of Cs in CHG context: 0.0
Total Number of Cs in CHH context: 0.0

Rate of Methylation
mCG 0.000%
mCHG 0.000%
mCHH 0.000%

line 22-28 are commented out with one line not commented out giving this message.
python bs3-build.py -h
File "bs3-build.py", line 23
"Use this options with conjuction of -r [--rrbs]")
^
IndentationError: unexpected indent