pmelsted / bfcounter Goto Github PK
View Code? Open in Web Editor NEWLicense: GNU General Public License v3.0
License: GNU General Public License v3.0
BFCounter is a memory efficient program for counting k-mers from sequencing files. Installation ============ Just run make to compile. % make The default maximum k-mer size supported is 31 (8 bytes of memory per k-mer), to modify this either replace MAX_KMER_SIZE in Makefile with an appropriate number or directly to make with % make MAX_KMER_SIZE=64 In this case it the maximum k-mer size allowed is 63, and each k-mer will use 16 bytes of memory. Running ======= To run the program use % ./BFCounter To list available commands % ./BFCounter count % ./BFCounter dump will provide more complete help. Notes ===== * BFCounter dump will print a text file with each k-mer and the number of occurrances of the k-mer. Only counts of 2 and greater will be printed, there is no limit on the number of occurrances it can handle. For each k-mer only the lexicographically smaller of the k-mer and the reverse complement of the k-mer, will be printed and the counts will be the sum of both values. * BFCounter was developed on x86-64 GNU/Linux. Porting to other unix-like platforms should be easy, but we haven't done so yet. * If you run into bugs or problems or have suggestions for future versions please contact me at [email protected] License ======= * The hash functions used are from the MurmurHash Library, version 3, released under the MIT License. http://code.google.com/p/smhasher/ * The kseq functions for reading fast(a|q)(.gz) files are copyrighted by Heng Li and released under the MIT License. http://lh3lh3.users.sourceforge.net/kseq.shtml * The Google sparsehash library is copyrighted by Google and released under a BSD License. http://code.google.com/p/google-sparsehash/ * This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.
The url in you paper is unavailable so that I cannot get the example data sets.
Hi!
I am trying to quantify short kmers - so in this case 9mers.
I used the following config:
./BFCounter count temp_${file}.fastq -k 9 -n 300000 -t 10 -o ${file}_bfoutput
There are about 260000 possible 9mers so I set the -n to 300000.
Once I dump the binary output, I get
AAAAAAAAA 1006644076
So somehow it only added the AAAAAAAAA kmer but no others.
Any ideas what I'm doing wrong?
Thanks in advance for your support!
Hi
We are working on analysis of Bioinformatics tools (related to Kmer counting) and BFCounter is one of them. We have gone through readme file and it is very helpful. As we are doing analysis so we want to be very sure about details. It would be great if you help us validating below details.
Data structure and Sorting Algo: Bloom Filter, Hash table/ Hashing
Approach: In-Memory
The limit of k-size : Arbitrary large k-mer lengths (any ideal length)
Supports online k-mer frequency retrieval : No
Supports compressed file processing : Yes
Thanks
Tarang
both count and dump create no output with my fasta files
./BFCounter dump -k 5 -i ~/faireparsed/e16-18hr_faire/macs2.fa -o outfile
help ..
head ~/faireparsed/e16-18hr_faire/macs2.fa
2L:287-539
TCTTATATTACCGCAAACCCAAAAAGACAATACACGACAGAGAGAGAGAGCAGCGGAGATATTTAGATTGCCTATTAAATATGATCGCGTATGCGAGAGTAGTGCCAACATATTGTGCTCTCTATATAATGACTGCCTCTCATTCTGTCTTATTTTACCGCAAACCCAAATCGACAATGCACGACAGAGGAAGCAGAACAGATATTTAGATTGCCTCTCATTTTCTCTCCCATATTATAGGGAGAAATATGA
2L:5543-5989
CAAACACAAAATGACAATGCA
....
Is the program easy to modify to show k-mers with a count of 1? This would be a helpful option.
Hi
I have quick query and it is related to support for compressed datatset.
Does BFCounter supports compressed datatset(gzip). I am not sure on #this.
Hi ... I would very much like to test your script ...but I do not seem to get it to work
I have complied it ate testing running with:
BFCounter count --kmer-size=10 --num-kmers=10000 --threads=10 -o output.txt --verbose -c 1 test.fa
Using bloom filter size: 4 bits
Estimated false positive rate: 0.146342
Segmentation fault
Can you help?
Also ... I am not sure what the --num-kmers is exactly can you explain ? Many thanks
Duarte
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.