CRAM fetch size limit about igv.js HOT 18 CLOSED

from igv.js.

Thanks for the test data. I can reproduce the issue here but do not have a solution. The cram-js package makes thousands of requests for small bits of data, then at some point the browser freezes. This could be a consequence of the extreme coverage depth, which is on the order of 100,000X, but that is just a guess. This is out of igv.js control, you might open a ticket at https://github.com/GMOD/cram-js. They will want a reproducible test case.

I did confirm that a BAM file will work. It is slow to load but does eventually display the data. You can speed up loading by setting a small visibilityWindow.

from igv.js.

Hi @cmdcolin , thanks for responding. I updated the cram-js library to the latest from the repository before testing this.

A range fetcher library makes sense, it would not only help with this problem but others as well. I will check that out.

@taniagmangolini You might consider checking out JBrowse as it can handle files of this type, whereas (at the moment) igv.js cannot. @cmdcolin Can give you pointers to install instructions if you need them.

from igv.js.

I can look into making it adjustable. I am curious how you are hitting the default limit, this is the first report. The size of the CRAM file is not the issue, many of our cram files are > 1 GB and some > 5 GB.

from igv.js.

I have tried to load the same files on the https://igv.org/app/ and got the same error.
First I load the reference (NC_045512.2.fasta and NC_045512.2.fasta.fai). After that, I load a track with the CRAM and CRAI files.
One thing I noticed is that when I increase the zoom the error stops to happen.

from igv.js.

Our crams have high depth coverage. Could be it a problem?

from igv.js.

Yes, you might try reducing the visibilityWindow, which defaults to 30kb. For example add this to your configuration

visibilityWindow: 10000

from igv.js.

I have tried this parameter without success. Can I share with you the cram and the crai files by e-mail?

from igv.js.

Its not necessary, I have made fetchSizeLimit configurable and the default 10000000. If you'd like to pull the latest code and build you can use it, or wait until the next release. Alternatively you can use the development build at https://igv.org/web/snapshot/dist, or the development webapp at https://igv.org/app-test

from igv.js.

Thank you! I have tried to load the files on https://igv.org/app-test and I got the same error as the image attached:

from igv.js.

The adjust solved the error message, but seems to be some problem related to the bundle-cram because it cannot load the cram. After some time loading an error page is displayed. I have tried other cram/crai and could reproduce the error.

from igv.js.

That indicates your browser is running out of memory, most likely. It is not an application error per se (igv.js or the cram bundle), and there is probably nothing we can do about it. You might try using a BAM file. This might work because it loads incrementally and downsamples the data as it loads. To decode a CRAM the entire data block must first be loaded into memory.

from igv.js.

Also, visibilityWindow will not necessarily help with CRAM files as they are encoded in larger blocks, but might help with BAM files for very deep coverage. How deep approximately is the coverage?

from igv.js.

It is highly variable, but usually the coverage is between 10-50x.

from igv.js.

That's not very deep at all. If you could share an example cram & crai file I will look into this, you can use our email [email protected] to send links, I can also give you a secure dropbox link to upload the files.

from igv.js.

I'm going to close this as there is nothing we can do in igv.js.

from igv.js.

@jrobinso I managed to load the users cram file in jbrowse 2 but it does have fairly high memory usage though. i might also get rid of the fetchsizelimit...it had perhaps good intentions but it causes nothing but trouble :)

do you know what version of the cram library you are using?

from igv.js.

regarding the thing where the cram library is generating many many http requests: that is something we combat in jbrowse 2 using the http-range-fetcher library (which rob wrote in tandem with writing the cram library). ideally this library would not be needed though, it might be useful to create a solution in the cram-js library that batches requests

from igv.js.