Comments (7)
chhylp123
commented on May 26, 2024
What's the size of your dataset? If it is not too large, I guess it should be fine.
Dear hifiasm team
I'm wondering how much memory the package needs for a read dataset of 30x coverage from a single human chromosome?
from hifiasm.
sinamajidian
commented on May 26, 2024
Thanks for fast reply. The fq.gz file is about one Gigabyte. It seems that I can not run it on my own desktop with 16 GB ram.
from hifiasm.
chhylp123
commented on May 26, 2024
Could you give me the log information? I think one Gigabyte should be fine...
from hifiasm.
sinamajidian
commented on May 26, 2024
After 7 hours the terminal is empty. It seems that there is no log. Is it producing log in a file? The desktop becomes very slow. The program only uses 1% of one cpu and almost all of the memory.
I've also tried with small example of simulated read dataset (70 Mbyte) from small reference genome. It finished but the gfa files are empty. The following is the log for the small example. Note the last line of the log.
[M::ha_analyze_count] lowest: count[3] = 1547
[M::ha_analyze_count] highest: count[4] = 12872
[M::ha_hist_line] 2: ****************************************************************************************************> 46760342
[M::ha_hist_line] 3: ************ 1547
[M::ha_hist_line] 4: **************************************************************************************************** 12872
[M::ha_hist_line] 5: ** 268
[M::ha_hist_line] 6: ************ 1512
[M::ha_hist_line] 7: * 118
[M::ha_hist_line] 8: **** 544
[M::ha_hist_line] 9: * 90
[M::ha_hist_line] 10: ** 241
[M::ha_hist_line] rest: ***** 624
[M::ha_analyze_count] left: none
[M::ha_analyze_count] right: count[6] = 1512
[M::ha_ft_gen] peak_hom: 6; peak_het: 4
[M::ha_ft_gen::9480.725*[email protected]] ==> filtered out 2 k-mers occurring 30 or more times
[M::ha_opt_update_cov] updated max_n_chain to 100
[M::ha_pt_gen::9493.968*0.03] ==> counted 2007574 distinct minimizer k-mers
[M::ha_pt_gen] count[4095] = 0 (for sanity check)
[M::ha_analyze_count] lowest: count[1] = 243557
[M::ha_analyze_count] highest: count[2] = 1763522
[M::ha_hist_line] 1: ************** 243557
[M::ha_hist_line] 2: **************************************************************************************************** 1763522
[M::ha_hist_line] rest: 495
[M::ha_analyze_count] left: none
[M::ha_analyze_count] right: none
[M::ha_pt_gen] peak_hom: 2; peak_het: -1
[M::ha_pt_gen::9503.243*0.03] ==> indexed 3529370 positions
[M::ha_assemble::9514.903*[email protected]] ==> corrected reads for round 1
[M::ha_assemble] # bases: 149998500; # corrected bases: 0; # recorrected bases: 0
[M::ha_assemble] size of buffer: 0.005GB
[M::ha_pt_gen::9521.783*0.03] ==> counted 2007574 distinct minimizer k-mers
[M::ha_pt_gen] count[4095] = 0 (for sanity check)
[M::ha_analyze_count] lowest: count[1] = 243557
[M::ha_analyze_count] highest: count[2] = 1763522
[M::ha_hist_line] 1: ************** 243557
[M::ha_hist_line] 2: **************************************************************************************************** 1763522
[M::ha_hist_line] rest: 495
[M::ha_analyze_count] left: none
[M::ha_analyze_count] right: none
[M::ha_pt_gen] peak_hom: 2; peak_het: -1
[M::ha_pt_gen::9528.960*0.03] ==> indexed 3529370 positions
[M::ha_assemble::9540.653*[email protected]] ==> corrected reads for round 2
[M::ha_assemble] # bases: 149998500; # corrected bases: 0; # recorrected bases: 0
[M::ha_assemble] size of buffer: 0.005GB
[M::ha_pt_gen::9547.512*0.03] ==> counted 2007574 distinct minimizer k-mers
[M::ha_pt_gen] count[4095] = 0 (for sanity check)
[M::ha_analyze_count] lowest: count[1] = 243557
[M::ha_analyze_count] highest: count[2] = 1763522
[M::ha_hist_line] 1: ************** 243557
[M::ha_hist_line] 2: **************************************************************************************************** 1763522
[M::ha_hist_line] rest: 495
[M::ha_analyze_count] left: none
[M::ha_analyze_count] right: none
[M::ha_pt_gen] peak_hom: 2; peak_het: -1
[M::ha_pt_gen::9554.677*0.03] ==> indexed 3529370 positions
[M::ha_assemble::9566.112*[email protected]] ==> corrected reads for round 3
[M::ha_assemble] # bases: 149998500; # corrected bases: 0; # recorrected bases: 0
[M::ha_assemble] size of buffer: 0.005GB
[M::ha_pt_gen::9572.965*0.04] ==> counted 2007574 distinct minimizer k-mers
[M::ha_pt_gen] count[4095] = 0 (for sanity check)
[M::ha_analyze_count] lowest: count[1] = 243557
[M::ha_analyze_count] highest: count[2] = 1763522
[M::ha_hist_line] 1: ************** 243557
[M::ha_hist_line] 2: **************************************************************************************************** 1763522
[M::ha_hist_line] rest: 495
[M::ha_analyze_count] left: none
[M::ha_analyze_count] right: none
[M::ha_pt_gen] peak_hom: 2; peak_het: -1
[M::ha_pt_gen::9580.119*0.04] ==> indexed 3529370 positions
[M::ha_assemble::9588.611*[email protected]] ==> found overlaps for the final round
[M::ha_print_ovlp_stat] # overlaps: 50264
[M::ha_print_ovlp_stat] # strong overlaps: 0
[M::ha_print_ovlp_stat] # weak overlaps: 50264
[M::ha_print_ovlp_stat] # exact overlaps: 8820
[M::ha_print_ovlp_stat] # inexact overlaps: 41444
[M::ha_print_ovlp_stat] # overlaps without large indels: 50264
[M::ha_print_ovlp_stat] # reverse overlaps: 0
Writing reads to disk...
Reads has been written.
Writing ma_hit_ts to disk...
ma_hit_ts has been written.
Writing ma_hit_ts to disk...
ma_hit_ts has been written.
Writing raw unitig GFA to disk...
Writing processed unitig GFA to disk...
Writing primary contig GFA to disk...
Writing alternate contig GFA to disk...
[M::main] Version: 0.5-dirty-r246
[M::main] CMD: /home/ssm/Documents/software/hifiasm/hifiasm -o asm/out reads/pacbio.fastq.gz
[M::main] Real time: 9588.930 sec; CPU: 353.453 sec; Peak RSS: 15.123 GB
from hifiasm.
chhylp123
commented on May 26, 2024
I see. For 70MB file, its coverage might be just 1 or 2, which is too small for assembler. Thus, hifiasm outputted an empty gfa. As for the 1G file, hifiasm looks like out of memory. Maybe you can try v0.3 (https://github.com/chhylp123/hifiasm/releases/tag/v0.3), its memory requirement should be lower.
from hifiasm.
lh3
commented on May 26, 2024
For small datasets, apply -f0 to disable bloom filter, which takes 16GB by default. Your coverage is too low for hifiasm to work by the way.
from hifiasm.
sinamajidian
commented on May 26, 2024
Thanks for the information.
from hifiasm.
Related Issues (20)
- ONT read integration HOT 8
- Un-balanced paternal/maternal haplotigs from trio binning (Revio HG002 HiFi child + NGS parental) HOT 2
- hifiasm's output with Hi-C Integrated Assembly model works badly in juicer and 3ddna pipeline HOT 2
- Missing of large segment in primary contigs HOT 3
- Trio binning - 1 mother 2 possible fathers HOT 2
- Duplicated sequence "h1tg000001l" when `--n-hap 4 ` HOT 6
- impact of k-mer size and b value on assembly phasing HOT 2
- parental data recommendation HOT 3
- weird k-mer peak HOT 1
- Does it make sense to direct assemble a allopolyploid combined with hic data and with --n-hap=2? HOT 1
- Segmentation fault at trio mode HOT 5
- Process fail - Core dump HOT 9
- Assembly fail with small regions (« ERROR: failed to read the graph ») HOT 4
- Assertion `a[p->aidx].pidx == (uint32_t)k' failed for only one sample HOT 2
- Assertion error != (unsigned int)-1 HOT 6
- Aborted run HOT 1
- Can Hifiasm distinguish duplicated chromosomes with different coverage in the genome assembly? HOT 4
- Assertion error: void filter_short_ulalignments HOT 2
- Accuracy in regions with only ONT support? HOT 3
- Can I see the contig information of hap1 and hap2 matched with each other in the hifiasm result? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hifiasm.