carlobaldassi / fastaio.jl Goto Github PK

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Warning: static parameter T does not occur in signature for FastaReader at /home/eric/.julia/FastaIO/src/FastaIO.jl:55.
The method will not be callable.

The offending line seems to be introduced at 0cc564f. Unfortunately my julia-fu is too weak to suggest a fix.

I'm on Julia master at 349c336 (latest as of posting).

Can you add support for fastq files?

Hi, I'm a long time user of your package and every once in a while, using it in my scripts, I feel an itch to somehow avoid that annoying 80 char warning for long headers. I have little experience in making pull requests, so I decided to ask first. What do you think about adding optional argument to your write function and its dependents (smth like ; nowarn=false) to be able to suppress that warning at will? If you OK with that, I could prepare a proper PR.

As for now I use custom macro to suppress warnings inside a function call:

using Logging

macro no_log(ex)
  quote
    with_logger(Logging.NullLogger()) do
      $(esc(ex))
    end
  end
end

Hei there!
Thank you coding the FastaIO library for Julia, I'm using it quite often. Recently I had a problem when I had to transform a very large multifasta file (~15,9 mil records) into a folder of individual files.

This is my function
`function mfasta2fasta(mfasta_p::FnaP, write_p::String; addprefix::Bool=false, prefix::String="", extension::String="fasta",
removeprev::Bool=false)
if removeprev
rm_mkpaths([write_p])
else
mkpath(write_p)
end
i=0
FastaReader(mfasta_p.p) do FASTA
for fr in FASTA
if addprefix == false
outp = "$(write_p)/$(fr[1]).$(extension)"
else
outp = "$(write_p)/$(prefix)_$(fr[1]).$(extension)"
end

        i = i+1

        try
            FastaWriter(outp) do fw
                @suppress writeentry(fw, "$(prefix)_$(fr[1])", "$(fr[2])")
            end
        catch e
            if e isa SystemError
                println(i)
                println(e)
                println("$(fr[1])")
                println("$(fr[2])")
                println(length(fr[2]))
                break
            end
        end
    end
end 

return nothing

end`

The "i" and the "try-catch" were introduced later, after it kept giving an "out of disk space error" always at the same fasta record, after ~ 5.7 mil fasta records were writen as individual files in the folder. I've tested and the respective record is not broken (I can see its name, bases, and length is of 5.7 kb). If I write just the respective record in a separate folder, it works. And there is plenty of space on the disk, I've checked that as well.
In the end, I had to run this function several times, saving only ~5 mil records per folder. I created a total of 4 folders.

The code used was:
function mfasta2fastaIMG(mfasta_p::FnaP, write_p::String; addprefix::Bool=false, prefix::String="", extension::String="fasta",
removeprev::Bool=false)
if removeprev
rm_mkpaths([write_p])
else
mkpath(write_p)
end

i=0
FastaReader(mfasta_p.p) do FASTA
    for fr in FASTA
        i = i+1

        if i < 15000000 #1 #5000000 #10000000 #15000000  (which numbers were used here, sequencially and in correspondence with the number for elseif below)
            continue
        elseif i < 16000000 #5000000 #10000000 #15000000 #16000000
            outp = "$(write_p)/$(fr[1]).$(extension)"

            if length(fr[2]) >= 2000
                FastaWriter(outp) do fw
                    @suppress writeentry(fw, "$(prefix)_$(fr[1])", "$(fr[2])")
                end
            end
        else
            break
        end
    end
end 

return nothing

end

I don't know what the problem is. It might be related with a faulty file system on the server. Or, it might be related to your library and Julia itself. So, I thought to make you aware of it.

best

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.4

• On 2015-01-22 the testing status was Tests pass.
• On 2015-01-23 the testing status changed to Tests fail, but package loads.

Tests pass. means that PackageEvaluator found the tests for your package, executed them, and they all passed.

Tests fail, but package loads. means that PackageEvaluator found the tests for your package, executed them, and they didn't pass. However, trying to load your package with using worked.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("FastaIO")' log
INFO: Installing FastaIO v0.1.3
INFO: Installing GZip v0.2.13
INFO: Package database updated

>>> 'using FastaIO' log
Julia Version 0.4.0-dev+2876
Commit f164ac1 (2015-01-22 22:58 UTC)
Platform Info:
  System: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

>>> test log

WARNING: deprecated syntax "{a,b, ...}" at /home/idunning/pkgtest/.julia/v0.4/FastaIO/test/runtests.jl:64.
Use "Any[a,b, ...]" instead.

WARNING: deprecated syntax "{}" at /home/idunning/pkgtest/.julia/v0.4/FastaIO/test/runtests.jl:110.
Use "[]" instead.
ERROR: LoadError: test failed: ArgumentError("invalid ASCII sequence") was thrown instead of ErrorException
 in expression: readfasta(infile)
 in error at error.jl:19
 in default_handler at test.jl:27
 in do_test_throws at test.jl:69
 in anonymous at no file:157
 in include at ./boot.jl:249
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:319
 in _start at ./client.jl:403
while loading /home/idunning/pkgtest/.julia/v0.4/FastaIO/test/runtests.jl, in expression starting on line 154

INFO: Testing FastaIO
===============================[ ERROR: FastaIO ]===============================

failed process: Process(`/home/idunning/julia04/usr/bin/julia --check-bounds=yes --code-coverage=none --color=no /home/idunning/pkgtest/.julia/v0.4/FastaIO/test/runtests.jl`, ProcessExited(1)) [1]

================================================================================
INFO: No packages to install, update or remove
ERROR: FastaIO had test errors
 in error at error.jl:19
 in test at pkg/entry.jl:717
 in anonymous at pkg/dir.jl:28
 in cd at ./file.jl:20
 in cd at pkg/dir.jl:28
 in test at pkg.jl:68
 in process_options at ./client.jl:241
 in _start at ./client.jl:403


>>> end of log