HTS-lib dependency about hts-nim HOT 15 CLOSED

This is now in master. Use the binary here: https://github.com/brentp/hts-nim/releases/tag/v0.2.8

I am interested to hear any feedback.

from hts-nim.

ok. I'll update the docker image once the new version of htslib comes out so this will be resolved. it only manifests with large numbers of queries. thanks for giving it a try!

from hts-nim.

hi @jaudoux, in theory, this is possible. see, for example: https://nim-lang.org/docs/nimc.html#dynliboverride

I think in practice, this would be a challenge due to the many dependencies of htslib. I will have a look again. Last time, I didn't get too far. If there's enough interest, I might also see how/if it's possible to sponsor this (for some nim/C expert) as it would be nice to have.

from hts-nim.

I just played around with this. In mosdepth I added a nim.cfg containing:

@if static:
  passl:"/home/brentp/src/htslib/libhts.a"
  passl:"/home/brentp/src/libdeflate/libdeflate.a"
  passl:"-lhts"
  passl:"-ldeflate"
  passl:"-lz"
  passl:"-llzma"
  passl:"-lcurl"
  passl:"-lbz2"
  passl:"-lcrypto"
  passl:"-lpthread"
  dynlibOverride:"hts"
@end

and then build with:

nim c -d:static -d:release mosdepth

this makes a binary that does not depend on libhts.so (just bz2.so, crypto, pthread, etc. :( ). But, it's a start.
Might be able to use a docker image to help make a static binary for all these libs since we can know the location of the static binaries, e.g. /usr/lib/x86_64-linux-gnu/libbz2.a.

This now seems more doable. I'll look into it more once I clear out my queue of things I need to finish.

from hts-nim.

Hi @brentp,

Thanks a lots for looking into this. This is a nice solution, and yes a Docker image ready to compile nim-hts based software into static binary would help. I'll try on my side to do something and share it on this thread.

Also many thanks for developing hts-nim I recently had a small bioinformatic project with nim and it is really great, I can finally get rid of my perl scripts !!!

Best,
Jérôme.

from hts-nim.

sounds good. the somalier docker file: https://github.com/brentp/somalier/blob/master/docker/Dockerfile

could be a good place to start. it has htslib and nim installed. then you'd just track down the locations of all the static libs. I think we'd have to build lib-curl instead of relying on the apt version.

it's great to hear that hts-nim is getting use.

from hts-nim.

see also: https://github.com/kaushalmodi/hello_musl

from hts-nim.

Okay thanks a lot for all those informations.

FYI : This is the small project that I was working one for to evaluate nim/hts-nim : https://github.com/SeqOne/kraf

from hts-nim.

cool. I added that here: https://github.com/brentp/hts-nim/wiki/Examples

I hope to have time to get an example going with a static build. I think the first step will be getting a static curl build (or just building htslib without libcurl) for the static lib.

from hts-nim.

I have succesfully built a static binary for mosdepth. The nim.cfg in the mosdepth repo contains this:

# in htslib: ./configure --with-libdeflate --disable-libcurl
# nim c -a -d:static -d:release mosdepth.nim
@if static:
  passC:"-static"
  passl:"-static"

  passl:"/home/brentp/src/htslib/libhts.a"
  passl:"/home/brentp/src/libdeflate/libdeflate.a"
  passl:"/usr/lib/x86_64-linux-gnu/liblzma.a"
  passl:"/usr/lib/x86_64-linux-gnu/libz.a"
  passl:"-lbz2"
  passl:"-llzma"
  passl:"-lpthread"
  dynlibOverride:"hts"
  dynlibOverride:"bz2"
  dynlibOverride:"pthread"
@end

I am working to make a docker image in the hts-nim repo that will facilitate making static binaries for anything that uses hts-nim. For now, it will not support libcurl (so no S3 or http/https support). We may be able to add that later.

from hts-nim.

I have this working in a branch: https://github.com/brentp/hts-nim/tree/static#static-builds

You can try this binary (which I built using this static machinery).
static_builder.gz

It will take a bit on the first run as it pulls hts-nim docker but it's pretty fast after that. I have nice static binaries for slivar and mosdepth working fine.

I'd be happy to have anyone who is interested read that section and then give this binary a try.

from hts-nim.

The static binary that I created for https://github.com/papaemmelab/split_bed_by_reads, appears to now have a memory leak.

from hts-nim.

this is (was) a bug in htslib. can you confirm that if you specify threads=0 that there is no leak?

this has been fixed in devel (samtools/htslib#822)
but I'm using the htslib 1.9 release so it will be present for now.

from hts-nim.

Yep threads=0 fixes it!

from hts-nim.

since I know at least 1 other project (besides mine) has static builds, I think this is solved. I hope to have libcurl support, but that will depend on updates for alpine linux itself.

from hts-nim.