researchit / igv-scanfold Goto Github PK

View Code? Open in Web Editor NEW

4.0 4.0 2.0 169 KB

IGV + RNA folding tools

Shell 4.97% Java 32.48% Batchfile 0.12% Python 62.43%

igv-scanfold's People

Contributors

Stargazers

Watchers

Forkers

rjandr wbrouse

igv-scanfold's Issues

have rnastructure/rnafold-only runs generate a base pair track file

bp / ct file

Add debug mode to run_scanfold.py

Look for an environment variable or something and

Re-enable stderr output
Print every command being run so it's easy to run them individually on the command line

Warn users to wait for jobs to finish before starting new one

We need to add a warning to let users know another job is running. Or at the very least should tell users to consider waiting for their first job to run before submitting a new one.

Create a single IGV-ScanFold .exe file for running on windows

We should have parity between the windows and mac version, where each is simply run from a single icon.

https://superuser.com/questions/868340/how-can-i-convert-a-windows-batch-script-to-a-exe

Add predicted runtime or progress bar to Scanfold window

make a docker container

low priority since we have pre-built releases for windows and mac

wip on docker branch; doesn't work yet

https://www.google.com/search?q=x11+docker

Mac bundle pops up messages about missing hash functions

It still works. Find a way to silence the warnings from Python.

numpy-1.19.4 incompatibility

Windows version of ScanFoldIGV (version 0.1.3) uses numpy-1.19.4 which throws this error in Windows:

ERROR [2020-12-01T10:18:27,733] [RuntimeUtils.java:88] RuntimeError: The current Numpy installation ('C:\\Users\\rjand\\SCANFO~1\\scanfold\\python-3.8.6-embed-amd64\\site-packages\\numpy\\__init__.py') fails to pass a sanity check due to a bug in the windows runtime. See this issue for more information: https://tinyurl.com/y3dm3h86

Manually replacing numpy folder with numpy-1.19.2 fixes the error.

Color varna popup with zavg data

During the varna loading process, it would be helpful to load the calculated zavg values from the wig file as a color map.

Windows bundle not running ScanFold

When trying to run ScanFold on both the entire visible region and a targeted region the log output window says done immediately after clicking run. No data is generated but it says that is completed. The screenshot below displays this issue. The release being used is 1.0.7.

Remove need for a run_me.command in mac build

Rename IGV.app to ScanFoldIGV.app

Move contents of run_me.command into ScanFoldIGV.app
/Contents/MacOS/IGV

runme.bat doesn't handle spaces in folder paths

Add a readme to mac and windows bundles

things to include

how to properly start IGV

PI Suggestions

On Mac with 1.0.7, I still get the ERROR:root:code message when trying to run ScanFold, but it still runs and gives me outputs. The other error that occurs is a null error for Varna if I uncheck the global refold, but that is always sent at the end of the run just stating that the file is empty. When I was talking to Walter he was hoping we could get rid of these error messages to avoid any reviewer and user complaints.

Walter also would like us to remove the "gui" part of the text in the "Run ScanFoldgui" tab and simply have it displayed as Run ScanFold.

Reverse strand Zavg track not populating

The final partners zscore wig file is not being written/loaded correctly when scanning the reverse strand.

Might just need to pass strand data to the function that writes the file (line 419 of ScanFold-Scan_IGV.py):

def write_wig_dict(nucleotide_dictionary, outputfilename, name):

    w = open(outputfilename, 'w')
    #write wig file header
    w.write("%s %s %s %s %s\n" % ("fixedStep", "chrom="+name, "start="+str(start_coordinate), "step=1", "span=1"))

    #write values of zscores
    if strand == "reverse":
        for k, v, in sorted(nucleotide_dictionary, reverse=True):
            w.write("%f\n" % (v.zscore))
    else:
        for k, v in nucleotide_dictionary.items():
            w.write("%f\n" % (v.zscore))

Release 0.1.7 strand error

Right now there is a line in ScanFold-Scan_IGV.py that overrides the strand settings.

line 270 print("Remove LINES about REVERSE STRAND")
line 271 strand = "reverse"

We need to remove this.

Generate extracted structure tracks for ScanFold RNAstructure runs

As reported in #31 #32

Disable run button after a job is launched until it finishes

Problem with running on M1 Mac

Hi,
I tried to run ScanFold on M1 Mac and have below information about the missing folder

add option to run rnafold/runastructure without scanfold

Mac Action #60

When installing the program, we are back to 2 zipped files before the icon appears, I don't have a problem with it or know if it can be changed but I am assuming some reviewers may have an issue with that.

For ScanFold RNAfold the reverse strand extracted structures still appear to be flipped for visible and selected scans.

For ScanFold RNAstructure no extracted structure tracks are generated for visible or selected scans on forward or reverse strand scans. If we can make a track for it that would be great, but if not then that is fine as well.

Improve ScanFold update process

This project uses its own version of the ScanFold scripts. These are stored in the ScanFold repo (_IGV.py) . As a result, an update to those scripts requires a pull request in that repository and then a submodule update in this repository.

Potential multiprocessing error on Windows

IGV-ScanFold on Windows 10 Home (Version 10.0.19042) does not complete scans.

igv.log info:

ERROR [2021-03-07T15:29:32,523] [RuntimeUtils.java:88]  Traceback (most recent call last):
ERROR [2021-03-07T15:29:32,524] [RuntimeUtils.java:88]    File "D:\rjand\DDownloads\scanfoldigv-windows\scanfold\ScanFold-Scan_IGV.py", line 338, in <module>
ERROR [2021-03-07T15:29:32,525] [RuntimeUtils.java:88]      energy_list = energies(seqlist, temperature)
ERROR [2021-03-07T15:29:32,525] [RuntimeUtils.java:88]    File "D:\rjand\DDownloads\scanfoldigv-windows\scanfold\ScanFoldSharedIGV.py", line 234, in energies
ERROR [2021-03-07T15:29:32,535] [RuntimeUtils.java:88]      energy_list = multiprocessing(rna_folder, [(sequence, temperature) for sequence in seq_list], 12)
ERROR [2021-03-07T15:29:32,536] [RuntimeUtils.java:88]    File "D:\rjand\DDownloads\scanfoldigv-windows\scanfold\ScanFoldSharedIGV.py", line 255, in multiprocessing
ERROR [2021-03-07T15:29:32,536] [RuntimeUtils.java:88]      return list(res)
ERROR [2021-03-07T15:29:32,536] [RuntimeUtils.java:88]    File "concurrent\futures\process.py", line 484, in _chain_from_iterable_of_lists
ERROR [2021-03-07T15:29:32,536] [RuntimeUtils.java:88]    File "concurrent\futures\_base.py", line 611, in result_iterator
ERROR [2021-03-07T15:29:32,537] [RuntimeUtils.java:88]    File "concurrent\futures\_base.py", line 432, in result
ERROR [2021-03-07T15:29:32,537] [RuntimeUtils.java:88]    File "concurrent\futures\_base.py", line 388, in __get_result
ERROR [2021-03-07T15:29:32,538] [RuntimeUtils.java:88]  concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
ERROR [2021-03-07T15:29:32,872] [RuntimeUtils.java:88]  Traceback (most recent call last):
ERROR [2021-03-07T15:29:32,872] [RuntimeUtils.java:88]    File "D:\rjand\DDownloads\scanfoldigv-windows\scanfold\ScanFold-Fold_IGV.py", line 681, in <module>
ERROR [2021-03-07T15:29:32,873] [RuntimeUtils.java:88]      step_size, window_size = get_step_win(lines)
ERROR [2021-03-07T15:29:32,873] [RuntimeUtils.java:88]    File "D:\rjand\DDownloads\scanfoldigv-windows\scanfold\ScanFold-Fold_IGV.py", line 539, in get_step_win
ERROR [2021-03-07T15:29:32,873] [RuntimeUtils.java:88]      row2 = lines[1].split()
ERROR [2021-03-07T15:29:32,873] [RuntimeUtils.java:88]  IndexError: list index out of range
INFO [2021-03-07T15:29:32,873] [MessageUtils.java:76]  IndexError: list index out of range<br>See igv.log for more details
INFO [2021-03-07T15:29:32,907] [IGV.java:1316]  Loading 1 resources.
INFO [2021-03-07T15:29:32,908] [TrackLoader.java:123]  Loading resource, path C:\Users\rjand\AppData\Local\Temp\scanfold-results3741362279704692303\output8vtkmuoz.IGV.bp
ERROR [2021-03-07T15:29:32,909] [BasePairFileParser.java:97]  Error parsing base pair file
java.lang.NullPointerException: null
	at org.broad.igv.feature.basepair.BasePairFileParser.loadData(BasePairFileParser.java:37) [igv.jar:?]
	at org.broad.igv.feature.basepair.BasePairTrack.<init>(BasePairTrack.java:44) [igv.jar:?]
	at org.broad.igv.track.TrackLoader.loadBasePairFile(TrackLoader.java:1113) [igv.jar:?]
	at org.broad.igv.track.TrackLoader.load(TrackLoader.java:192) [igv.jar:?]
	at org.broad.igv.ui.IGV.load(IGV.java:1399) [igv.jar:?]
	at org.broad.igv.ui.IGV.loadResources(IGV.java:1331) [igv.jar:?]
	at org.broad.igv.ui.IGV$4.run(IGV.java:474) [igv.jar:?]
	at org.broad.igv.util.LongRunningTask.call(LongRunningTask.java:72) [igv.jar:?]
	at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
	at java.lang.Thread.run(Unknown Source) [?:?]
ERROR [2021-03-07T15:29:32,910] [TrackLoader.java:252]  java.lang.NullPointerException
java.lang.RuntimeException: java.lang.NullPointerException
	at org.broad.igv.feature.basepair.BasePairFileParser.loadData(BasePairFileParser.java:101) ~[igv.jar:?]
	at org.broad.igv.feature.basepair.BasePairTrack.<init>(BasePairTrack.java:44) ~[igv.jar:?]
	at org.broad.igv.track.TrackLoader.loadBasePairFile(TrackLoader.java:1113) ~[igv.jar:?]
	at org.broad.igv.track.TrackLoader.load(TrackLoader.java:192) [igv.jar:?]
	at org.broad.igv.ui.IGV.load(IGV.java:1399) [igv.jar:?]
	at org.broad.igv.ui.IGV.loadResources(IGV.java:1331) [igv.jar:?]
	at org.broad.igv.ui.IGV$4.run(IGV.java:474) [igv.jar:?]
	at org.broad.igv.util.LongRunningTask.call(LongRunningTask.java:72) [igv.jar:?]
	at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
	at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.lang.NullPointerException
	at org.broad.igv.feature.basepair.BasePairFileParser.loadData(BasePairFileParser.java:37) ~[igv.jar:?]
	... 11 more
ERROR [2021-03-07T15:29:32,910] [IGV.java:1334]  Error loading track
org.broad.igv.exceptions.DataLoadException: java.lang.NullPointerException
	at org.broad.igv.track.TrackLoader.load(TrackLoader.java:254) ~[igv.jar:?]
	at org.broad.igv.ui.IGV.load(IGV.java:1399) ~[igv.jar:?]
	at org.broad.igv.ui.IGV.loadResources(IGV.java:1331) [igv.jar:?]
	at org.broad.igv.ui.IGV$4.run(IGV.java:474) [igv.jar:?]
	at org.broad.igv.util.LongRunningTask.call(LongRunningTask.java:72) [igv.jar:?]
	at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
	at java.lang.Thread.run(Unknown Source) [?:?]
INFO [2021-03-07T15:29:32,911] [MessageUtils.java:76]  Error loading C:\Users\rjand\AppData\Local\Temp\scanfold-results3741362279704692303\output8vtkmuoz.IGV.bp: java.lang.NullPointerException
INFO [2021-03-07T15:29:32,923] [IGV.java:1316]  Loading 1 resources.
INFO [2021-03-07T15:29:32,924] [TrackLoader.java:123]  Loading resource, path C:\Users\rjand\AppData\Local\Temp\scanfold-results3741362279704692303\outputaatu22bs.final_partners_zscore.wig
INFO [2021-03-07T15:29:32,946] [IGV.java:1316]  Loading 1 resources.
INFO [2021-03-07T15:29:32,947] [TrackLoader.java:123]  Loading resource, path C:\Users\rjand\AppData\Local\Temp\scanfold-results3741362279704692303\outputkznuual7.mfe.wig
INFO [2021-03-07T15:29:32,963] [IGV.java:1316]  Loading 1 resources.
INFO [2021-03-07T15:29:32,964] [TrackLoader.java:123]  Loading resource, path C:\Users\rjand\AppData\Local\Temp\scanfold-results3741362279704692303\output934ttvol.zscore.wig
INFO [2021-03-07T15:29:33,017] [IGV.java:1316]  Loading 1 resources.
INFO [2021-03-07T15:29:33,018] [TrackLoader.java:123]  Loading resource, path C:\Users\rjand\AppData\Local\Temp\scanfold-results3741362279704692303\output_bsqdh6r.pvalue.wig
INFO [2021-03-07T15:29:33,038] [IGV.java:1316]  Loading 1 resources.
INFO [2021-03-07T15:29:33,039] [TrackLoader.java:123]  Loading resource, path C:\Users\rjand\AppData\Local\Temp\scanfold-results3741362279704692303\outputh___iw6i.ed.wig
INFO [2021-03-07T15:29:33,064] [IGV.java:1316]  Loading 1 resources.
INFO [2021-03-07T15:29:33,065] [TrackLoader.java:123]  Loading resource, path C:\Users\rjand\AppData\Local\Temp\scanfold-results3741362279704692303\outputwgdlc5d_.-2filter.scanfoldvarna
ERROR [2021-03-07T15:29:33,066] [TrackLoader.java:252]  null
java.lang.NullPointerException: null
ERROR [2021-03-07T15:29:33,066] [IGV.java:1334]  Error loading track
java.lang.NullPointerException: null
	at org.broad.igv.exceptions.DataLoadException.<init>(DataLoadException.java:38) ~[igv.jar:?]
	at org.broad.igv.track.TrackLoader.load(TrackLoader.java:254) ~[igv.jar:?]
	at org.broad.igv.ui.IGV.load(IGV.java:1399) ~[igv.jar:?]
	at org.broad.igv.ui.IGV.loadResources(IGV.java:1331) [igv.jar:?]
	at org.broad.igv.ui.IGV$4.run(IGV.java:474) [igv.jar:?]
	at org.broad.igv.util.LongRunningTask.call(LongRunningTask.java:72) [igv.jar:?]
	at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
	at java.lang.Thread.run(Unknown Source) [?:?]
INFO [2021-03-07T15:29:33,066] [MessageUtils.java:76]  Error loading C:\Users\rjand\AppData\Local\Temp\scanfold-results3741362279704692303\outputwgdlc5d_.-2filter.scanfoldvarna: null

Windows Action #52

When installing the program, we are back to 2 zipped files before the batch script appears, which is not a problem to me but I could see a reviewer having issues with that. If it can be changed to one zipped file that would be great, but if not that is fine too.

When trying to run ScanFold the "Run" button in the parameters window is not visible to click even when the computer's task bar is removed, but you can just hit enter to start the scan. The resolution of the testing computer is 1366x768, which is the max setting for the machine.

For ScanFold RNAfold the reverse strand extracted structures still appear to be flipped for visible and selected scans.

The -2filter.varnascanfold file is sometimes empty, causing error

right now, the -2filter.varnascanfold file sometimes doesn't populate. We should find a way to generate this file either way.

ScanFoldIGV name change to IGV-ScanFold

The language we landed on for the implementation of ScanFold into IGV is IGV-ScanFold. We need to go through and change front facing names in the code accordingly (e.g. ScanFoldIGV.app should just be IGV-ScanFold.app).

Prevent running on sequences over a certain size

trying to run it on entire chromosomes is slow

Version 0.1.9 Mac Testing Issues

When RNAfold is used with the reverse strand for the entire visible region and selected regions the extracted structures track is incorrect (i.e. it output the correct data to the track but the bars do not align with the arc diagram/bp track, they seem to be flipped as if it is forward strand). The screenshot below shows what I mean (bottom track).

When RNAstructure is used on the entire visible region or a selected region for forward and reverse strand no extracted structures track is made and the ED track that is generated is empty (top 4 tracks of second screenshot).

I also noticed that the text box at the top of the scanfoldgui pop up window telling users whether they will be scanning the selected region or the entire visible region can be edited by the user.

Java CI with gradle build failing

The build fails at igv/build.gradle line 472:
"Class-Path": configurations.default.collect { it.getName() }.join(' ')

Unsure how to fix. May need to link to a different IGV commit?

Version 0.1.9 Windows Testing Issues

When RNAfold is used with the reverse strand for the entire visible region and selected regions the extracted structures track is incorrect (i.e. it output the correct data to the track but the bars do not align with the arc diagram/bp track, they seem to be flipped as if it is forward strand). The first screenshot below shows what I mean (bottom track). When RNAstructure is used on the entire visible region or a selected region for forward and reverse strand no extracted structures track is made and the ED track that is generated is empty (top 4 tracks of second screenshot).

I also noticed that the text box at the top of the scanfoldgui pop up window telling users whether they will be scanning the selected region or the entire visible region can be edited by the user.

One additional thing I noticed is that the scanfoldgui pop up window has some weird things happening. The LogOutput box covers the LogOutput label, and both the Run and Close tabs are very difficult to see and select. I tried adjusting its size but nothing helped so I actually had to get rid of my taskbar to be able to start ScanFold runs. The second screenshot demonstrates what I mean.