gitter-lab / singe Goto Github PK

A user reports

I have not succeeded to run it on local MATLAB installation. I received errors trying to run the SINGE_Example locally (MATLAB R2019b) on a MacBook running macOS 10.15.2. Seems to be an issue with libgfortran, which is required by glmnet.

$ SINGE_Example
…

Invalid MEX-file
'…/SINGE-master/glmnet_matlab/glmnetMex.mexmaci64':
dlopen(…/SINGE-master/glmnet_matlab/glmnetMex.mexmaci64,
6): Library not loaded: /usr/local/gfortran/lib/libgfortran.3.dylib
Referenced from:
…/SINGE-master/glmnet_matlab/glmnetMex.mexmaci64
Reason: image not found

Error in glmnetControl (line 79)
ivals.pmin, ivals.exmx, ivals.prec, ivals.mxit] =
glmnetMex();

Error in glmnet (line 329)
inparms = glmnetControl();

Error in iLasso_for_SINGE (line 100)
fit = glmnet(Am, bm, params.family, opt);

Error in run_iLasso_row (line 27)
[metric] = iLasso_for_SINGE(m, outs,
lambda,p1,dT,std_dev,params);

Error in SINGE_GLG_Test (line 58)
[for_metric] = run_iLasso_row(m,outs,params,irow);

Error in SINGE (line 20)
SINGE_GLG_Test(Data,'--outdir',outdir,args{:})

Error in SINGE_Example (line 16)
SINGE(data,gene_list,outdir,hyperparameter_file);

I do not have a local machine running macOS, so I cannot reproduce this issue yet. In #54 I started working on testing SINGE with the MATLAB runtime as a workaround.

We would like to automatically detect if the compiled SINGE binary is stale with respect to the MATLAB files in the repository. The intended behavior is that a Travis CI build fails if the code has been updated but the compiled binary has not.

We can explore using OSG Connect to allow users to run SCINGE with HTCondor on full scale datasets.

I am trying to run Singe_Example.m on an Ubuntu machine. However, I am getting the following error from glmnetMex.mexa64. I unfortunately cannot install libgfortran3 on my machine as it is deprecated. Is there any workaround for this?

In SINGE (line 20)
In SINGE_Example (line 16)
In run (line 112)

randomizer =

  737892

Invalid MEX-file '/home/ubuntu/SINGE/SINGE/glmnet-matlab/glmnetMex.mexa64':
libgfortran.so.3: cannot open shared object file: No such file or directory

Error in glmnetControl (line 79)
ivals.pmin, ivals.exmx, ivals.prec, ivals.mxit] = glmnetMex();

Error in glmnet (line 329)
inparms = glmnetControl();

Error in iLasso_for_SINGE (line 111)
fit = glmnet(Am, bm, params.family, opt);

Error in run_iLasso_row (line 27)
[metric] = iLasso_for_SINGE(m, outs, lambda,p1,dT,std_dev,params);

Error in SINGE_GLG_Test (line 79)
[for_metric] = run_iLasso_row(m,outs,params,irow);

Error in SINGE (line 20)
SINGE_GLG_Test(Data,'--outdir',outdir,args{:})

Error in SINGE_Example (line 16)
SINGE(data,gene_list,outdir,hyperparameter_file);

Error in run (line 112)
evalin('caller', strcat(scriptStem, ';'));

The validation for the outdir parameter uses the validString function, which does not allow subdirectories. We should update this to use a validPath that does allow subdirectories.

This issue was reported in Murali-group/Beeline@bace690

We're changing the abbreviation from SINGE to SCINGE. I'll change the repo name immediately and leave this open until we have fixed all of the individual files.

One example is that we do not recommend lambda=0 with large datasets. We can also recommend more strongly that SINGE is run with an ensemble of many different hyperparameters.

Copying the original report from Murali-group/Beeline#33 (comment):

At this line https://github.com/gitter-lab/SINGE/blob/master/code/iLasso_for_SINGE.m#L56 , values of m.fullKp are written into a copy of the input mat file before a variable declaration for fullKp, and MATLAB should infer that fullKp is a struct array. This works as expected when using version 7.3 input mat files written by MATLAB, which support partial writes of each value without loading fullKp into memory https://www.mathworks.com/help/matlab/import_export/load-parts-of-variables-from-mat-files.html . As I understand it, the issue I ran into when using a version 7 input mat file written by Octave, which does not support writing version 7.3 mat files, is that MATLAB first loads the fullKp variable into memory, and because that variable hasn't been declared yet MATLAB initializes it, but at that point MATLAB fails to correctly infer that the type should be a struct array. The actual error I get is that the Kp2 struct cannot be assigned to a value in m.fullKp, that full error message copied below:

Warning: The file '/usr/local/SINGE/TempMat_5.mat' was saved in a format that does not support partial loading. Temporarily loading variable 'fullKp' into memory. To use partial loading efficiently, save MAT-files with the -v7.3 flag.

Error using iLasso_for_SINGE (line 56)
Conversion to double from struct is not possible.

The workaround I implemented in that commit was to initialize the fullKp variable as a struct array in the input mat file, which appears to work but obviously it is not ideal to initialize a specific internal variable that could change in another SINGE version. I think this might be addressed in SINGE to support a version 7 mat file written by Octave by either initializing fullKp as a struct array before that write, loading and saving the input file as a version 7.3 mat file instead of directly copying to a temporary file when parsing the input, or otherwise writing that variable somewhere other than the mat file.

#16 confirmed that it is possible to run SINGE inside a Docker to container using compiled MATLAB code. Our next step will be to create a general purpose SINGE Docker container. Specific tasks include:

• Create a command line interface #17
• Release a new version of SINGE
• Compile that new release and host it somewhere stable
• Remove the temp versions of SINGE at https://www.biostat.wisc.edu/~gitter/tmp/
• Create a Docker container using the strategy in #16 and the new release

Hi, I am trying to reproduce Fig 3 from your paper and I was wondering if you could share the GRN for ESC to endoderm differentiation which you determined from the ESCAPE database? Thanks a lot!

We should create a tests subdirectory that can initially store the expected output files from the example data. Those will be needed for continuous integration testing.

In addition to the final outputs, we can also store some intermediate output files if they are not too large. This will be necessary if we ever want to port the code to Python or R.

Follows up on #37. We should add usage instructions and additional documentation about the arguments for standalone_SINGE.sh within the script.

Thanks very much for developing this useful tool and I would like to use it to construct GRN with my own data. Since I'm not familiar with Matlab, I tried to run it with bash script. However, I did not find any detailed intros for the input files in 'mat' format. I knew it should be a saved Matlab workspace file with several variables.
So I tried to dissect the content of these input 'mat' files by myself, where I found two variables in 'X_SCODE_data.mat', 'X' and 'ptime', one variable in 'gene_list.mat', 'gene_list'. I also found 'X' in 'X_SCODE_data.mat' is a sparse matrix, 'ptime' is a matrix, and 'gene_list' is a cell. I created a similar 'mat' file with similar organization with my own data but got warnings: "Unable to read some of the variables due to unknown MAT-file error."
I'm more sure whether there are more hidden informations for these variables in 'mat' format ?
Is it possible to directly take more common files types, such as 'txt/csv' as inputs ?

The latest Docker image build on GitHub Actions failed because of apt-get update failures. They appear to refer to Debian stretch no longer being supported.

I disabled building the Docker image from our GitHub Actions workflow temporarily until we can fix the Docker image.

The macOS MATLAB runtime installation now fails. The Java runtime cannot be detected. Instead of continuing to install this manually, it will simplify the GitHub Actions workflow to switch to the setup-matlab action.

Before we create a standalone SINGE aggregate script for #6, we should standardize the order of the arguments to these scripts:

• standalone_SINGE.sh Data gene_list outdir hyperparameter_file runtime_dir
• run_SINGE_GLG_Test.sh $runtime $data --outdir $outdir $arg
• run_SINGE_Aggregate.sh $runtime $gene_list $data $outdir

SINGE_Aggregate expects the arguments in a different order than the others. Swapping $gene_list and $data would correct this.

While testing #67 we found that running with a single regulator index can cause an error within glmnet. For now, the regulator index list must contain two or more regulators.

We would like to have a script that shows how to run SCINGE from the command line. It will provide all of the input data files and GLG hyperparmeters. A demo script will reproduce the test run in SCINGE_Example.m.

Initially this will assume that all GLG runs are done locally. We can parallelize the GLG runs for high-thoughput computing (#6) as a later step.

https://github.com/scottclowe/matlab-continuous-integration provides an example of Travis CI testing for MATLAB code via Octave. If we can run SCINGE on Octave, we can set up basic continuous integration testing.

This would likely require recompiling or adapting the Glmnet mex files for Octave.

The current implementation of the function that drops samples has a bug that only drops randomly-selected zero-valued samples instead of any sample:

We plan to correct this and release a new version of SINGE soon. We will update the test cases as well.

GENIE3 is a popular network inference approach. It provides a way to output all regulator-target edges: https://github.com/aertslab/GENIE3/blob/master/vignettes/GENIE3.Rmd#L116

SINGE already has a similar output format but does not output edges with a score of 0. The 0s in the output are edges that had a non-zero score before rounding

We could consider adding an optional parameter in the config file that would output all edge scores and all regulator scores even if they are 0. That would help users use SINGE in downstream analyses set up for GENIE3.

I was testing SINGE in the Docker image and provided a hyperparameters files that did not exist. However, SINGE continued running with default hyperparameters. SINGE.sh and SINGE.m should exit with an error in this case.

Here is the start of the output:

SINGE operating in standalone mode
standalone mode running GLG tests
grep: input/default_hyperparameters.txt: No such file or directory
hypenum: 1
arg:
sed: can't read input/default_hyperparameters.txt: No such file or directory
------------------------------------------
Setting up environment variables
---
LD_LIBRARY_PATH is .:/usr/local/MATLAB/MATLAB_Runtime/v94/runtime/glnxa64:/usr/l
ocal/MATLAB/MATLAB_Runtime/v94/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/
sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v94/sys/opengl/lib/glnxa64
Creating MATLAB Runtime Cache at location: /tmp/.mcrCache9.4
.max_size not found.  Using default size of 33554432 bytes.
MATLAB Runtime cache extracting component: SINGE_GLG_Te_DE1F291F7B6F70644FA3E2EA
0D86137D
Acquiring MATLAB Runtime cache root-level directory lock... acquire succeeded.
Reading cache index file...
File open failed for /tmp/.mcrCache9.4/.mcr_cache_index
MATLAB Runtime cache: extractDir is /tmp/.mcrCache9.4/SINGE_0

Adding component SINGE_GLG_Te_DE1F291F7B6F70644FA3E2EA0D86137D to the cache.
MATLAB Runtime Cache: performing maintenance...
Processing cached components...
Done with cache maintenance.
Creating component directory: /tmp/.mcrCache9.4/SINGE_0
Acquiring component directory WRITE lock... acquire succeeded.
Extracting component... Component extracted to cache.  Writing creation timestam
p...
Timestamp successfully created.
done.
Downgrading WRITE lock to READ lock... downgrade successful.
Component SINGE_GLG_Te_DE1F291F7B6F70644FA3E2EA0D86137D has successfully been ac
cessed from the cache.
MATLAB Runtime Cache: performing maintenance...
Processing cached components...
Done with cache maintenance.
Checking whether index file /tmp/.mcrCache9.4/.mcr_cache_index needs to be writt
en...
Write is needed.
Writing cache index file: /tmp/.mcrCache9.4/.mcr_cache_index
Writing cache index entry:
SINGE_GLG_Te_DE1F291F7B6F70644FA3E2EA0D86137D
SINGE_0
2519071
2021-Jan-23 21:43:46.477392


params =

  struct with fields:

                   Data: 'input/X_SCODE_data.mat'
                   date: '23-Jan-2021'
                     dT: 1
                 family: 'gaussian'
                     ID: 0
           kernel_width: 2
                 lambda: 0.0100
               num_lags: 15
                 outdir: 'output'
    prob_remove_samples: 0.2000
      prob_zero_removal: 0
              replicate: 0
                     p1: 15
             DateNumber: 738179

We may want to support a pre-defined list of allowed regulators. This would be a subset of the gene list and could be used if someone wants to restrict the candidate source nodes using prior knowledge.

The date hyperparameter should follow the YYYY-MM-DD format. This is the ISO standard https://www.iso.org/iso-8601-date-and-time-format.html

Provide example with high throughput computing.
Could include specific example with broad guidelines to facilitate the adaptation on other high-throughput systems.

We should describe the current way in which a user can control the seed and can also discuss possible changes.

We should investigate strategies to automate the workflow of splitting a branching trajectory into multiple cell-fate based subdirectories, followed by SINGE analysis on each subdirectory and obtaining GRNs based on these analyses. Some initial thoughts are provided in USAGE.md.

Hi,

I'm working on integrating the latest SINGE release into BEELINE repo. However, when I tried building a docker image using the Dockerfile provided, I'm getting the following error:

:~/SINGE/docker$ docker build -t singe -f Dockerfile .
Sending build context to Docker daemon  5.632kB
Step 1/8 : FROM amarburg/matlab-runtime
 ---> 32585984ab81
Step 2/8 : RUN apt-get update &&     apt-get -y install libxt6 bzip2
 ---> Running in ab8830c0e025
Get:1 http://security.debian.org stable/updates InRelease [39.1 kB]
Get:2 http://deb.debian.org/debian stable InRelease [122 kB]
Get:3 http://deb.debian.org/debian stable-updates InRelease [49.3 kB]
Get:4 http://security.debian.org stable/updates/main amd64 Packages [120 kB]
Get:5 http://deb.debian.org/debian stable/main amd64 Packages [10.6 MB]
Get:6 http://deb.debian.org/debian stable-updates/main amd64 Packages.diff/Index [1720 B]
Get:6 http://deb.debian.org/debian stable-updates/main amd64 Packages.diff/Index [1720 B]
Get:7 http://deb.debian.org/debian stable-updates/main amd64 Packages [6292 B]
Reading package lists...
W: Conflicting distribution: http://security.debian.org stable/updates InRelease (expected stretch but got buster)
W: Conflicting distribution: http://deb.debian.org/debian stable InRelease (expected stretch but got buster)
W: Conflicting distribution: http://deb.debian.org/debian stable-updates InRelease (expected stretch-updates but got buster-updates)
E: Could not open file /var/lib/apt/lists/deb.debian.org_debian_dists_stable-updates_main_binary-amd64_Packages.diff_Index - open (2: No such file or directory)
The command '/bin/sh -c apt-get update &&     apt-get -y install libxt6 bzip2' returned a non-zero code: 100

I'm running this on Ubuntu 18.04 and Docker version 18.09.5, build e8ff056. I also got this error when I used the Dockerfile provided in the release version 0.3.0.

I'm not sure if this is a known issue, but README file tells me that the Docker support is still being improved. Do you may be have an estimate of when this would be completed? Please let me know if there is any other information I can provide about the issue that would be useful to you.

We see a significant percentage of the jobs resulting in a segmentation violation when glmnet is called with the params.family = 'poisson' option (example at bottom). We may need to involve the glmnet maintainer at Stanford at some point.

A temporary workaround would be to transform count-based transcriptomic data and using the params.family = 'gaussian' option.

   Segmentation violation detected at Tue Jul 02 17:21:13 2019 -0500

Configuration:
Crash Decoding : Disabled - No sandbox or build area path
Crash Mode : continue (default)
Default Encoding : US-ASCII
Deployed : true
GNU C Library : 2.17 stable
Graphics Driver : Unknown software
MATLAB Architecture : glnxa64
MATLAB Entitlement ID : Unknown
MATLAB Root : /var/lib/condor/execute/slot1/dir_24929/v94
MATLAB Version : 9.4.0.813654 (R2018a)
OpenGL : software
Operating System : Linux 5.0.8-1.el7.elrepo.x86_64 #1 SMP Wed Apr 17 10:11:44 EDT 2019 x86_64
Process ID : 25220
Processor ID : x86 Family 6 Model 23 Stepping 10, GenuineIntel
Session Key : 8f018b8e-68d7-47a0-bf6d-26a0bf86030f
Static TLS mitigation : Disabled: Unable to open display
Window System : No active display

Fault Count: 1

Abnormal termination

Register State (from fault):
RAX = 0000000000003b16 RBX = 0000150d175d84a0
RCX = 0000150d175c9840 RDX = ffffffffc767d558
RSP = 0000150d13ffbe00 RBP = 0000150d13ffbf60
RSI = 0000150d17622240 RDI = 000000003fefffff

R8 = 0000000000000006 R9 = 0000000000000000
R10 = 0000150d17cf8830 R11 = 0000000000000005
R12 = 000000003ff00000 R13 = 0000000000000012
R14 = 0000000000000000 R15 = 0000150d17622250

RIP = 0000150cde6ea912 EFL = 0000000000010202

CS = 0033 FS = 0000 GS = 0000

Stack Trace (from fault):
[ 0] 0x0000150cde6ea912 /tmp/.mcrCache9.4/GLG_In0/glmnet_matlab/glmnetMex.mexa64+00252178
[ 1] 0x0000150cde6b566e /tmp/.mcrCache9.4/GLG_In0/glmnet_matlab/glmnetMex.mexa64+00034414 mexfunction_+00030329
[ 2] 0x0000150d2d2721ea bin/glnxa64/libmex.so+00414186
[ 3] 0x0000150d2d272447 bin/glnxa64/libmex.so+00414791
[ 4] 0x0000150d2d272f2b bin/glnxa64/libmex.so+00417579
[ 5] 0x0000150d2d25d30c bin/glnxa64/libmex.so+00328460
[ 6] 0x0000150d2edb52ad bin/glnxa64/libmwm_dispatcher.so+00979629 ZN8Mfh_file16dispatch_fh_implEMS_FviPP11mxArray_tagiS2_EiS2_iS2+00000829
[ 7] 0x0000150d2edb5bae bin/glnxa64/libmwm_dispatcher.so+00981934 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00000030
[ 8] 0x0000150d29ad7da1 bin/glnxa64/libmwm_lxe.so+12619169
[ 9] 0x0000150d29ad8982 bin/glnxa64/libmwm_lxe.so+12622210
[ 10] 0x0000150d29bc0e79 bin/glnxa64/libmwm_lxe.so+13573753
[ 11] 0x0000150d29b623e1 bin/glnxa64/libmwm_lxe.so+13186017
[ 12] 0x0000150d293685a8 bin/glnxa64/libmwm_lxe.so+04822440
[ 13] 0x0000150d2936acbc bin/glnxa64/libmwm_lxe.so+04832444
[ 14] 0x0000150d2936701d bin/glnxa64/libmwm_lxe.so+04816925
[ 15] 0x0000150d29360ba1 bin/glnxa64/libmwm_lxe.so+04791201
[ 16] 0x0000150d29360dd9 bin/glnxa64/libmwm_lxe.so+04791769
[ 17] 0x0000150d29366846 bin/glnxa64/libmwm_lxe.so+04814918
[ 18] 0x0000150d2936692f bin/glnxa64/libmwm_lxe.so+04815151
[ 19] 0x0000150d29495503 bin/glnxa64/libmwm_lxe.so+06055171
[ 20] 0x0000150d29498cf3 bin/glnxa64/libmwm_lxe.so+06069491
[ 21] 0x0000150d299a8f6d bin/glnxa64/libmwm_lxe.so+11378541
[ 22] 0x0000150d29ac57c4 bin/glnxa64/libmwm_lxe.so+12543940
[ 23] 0x0000150d29ac5d6b bin/glnxa64/libmwm_lxe.so+12545387
[ 24] 0x0000150d2edb52ad bin/glnxa64/libmwm_dispatcher.so+00979629 ZN8Mfh_file16dispatch_fh_implEMS_FviPP11mxArray_tagiS2_EiS2_iS2+00000829
[ 25] 0x0000150d2edb5bde bin/glnxa64/libmwm_dispatcher.so+00981982 ZN8Mfh_file22dispatch_fh_with_reuseEiPP11mxArray_tagiS2+00000030
[ 26] 0x0000150d29be5d4e bin/glnxa64/libmwm_lxe.so+13725006
[ 27] 0x0000150d29955416 bin/glnxa64/libmwm_lxe.so+11035670
[ 28] 0x0000150d2995558c bin/glnxa64/libmwm_lxe.so+11036044
[ 29] 0x0000150d299eaae8 bin/glnxa64/libmwm_lxe.so+11647720
[ 30] 0x0000150d299ec229 bin/glnxa64/libmwm_lxe.so+11653673
[ 31] 0x0000150d2ea14f80 bin/glnxa64/libmwm_interpreter.so+00688000 _Z44inCallFcnWithTrapInDesiredWSAndPublishEventsiPP11mxArray_tagiS1_PKcbP15inWorkSpace_tag+00000080
[ 32] 0x0000150d2d7b586d bin/glnxa64/libmwiqm.so+00768109 _ZN3iqm15BaseFEvalPlugin7executeEP15inWorkSpace_tagRN7mwboost10shared_ptrIN14cmddistributor17IIPCompletedEventEEE+00000525
[ 33] 0x0000150d301f14a1 bin/glnxa64/libmwmcr.so+00849057
[ 34] 0x0000150d2d7abab1 bin/glnxa64/libmwiqm.so+00727729
[ 35] 0x0000150d2d78ea95 bin/glnxa64/libmwiqm.so+00608917
[ 36] 0x0000150d301bffe5 bin/glnxa64/libmwmcr.so+00647141
[ 37] 0x0000150d301c06a4 bin/glnxa64/libmwmcr.so+00648868
[ 38] 0x0000150d301b93f1 bin/glnxa64/libmwmcr.so+00619505
[ 39] 0x0000150d37574dd5 /lib64/libpthread.so.0+00032213
[ 40] 0x0000150d3a1eaead /lib64/libc.so.6+01040045 clone+00000109
[ 41] 0x0000000000000000 +00000000

This error was detected while a MEX-file was running. If the MEX-file
is not an official MathWorks function, please examine its source code
for errors. Please consult the External Interfaces Guide for information
on debugging MEX-files.
** This crash report has been saved to disk as /tmp/matlab_crash_dump.25220-1 **

MATLAB is exiting because of fatal error
/var/lib/condor/execute/slot1/dir_24929/condor_exec.exe: line 40: 25220 Killed "/var/lib/condor/execute/slot1/dir_24929/GLG_Instance" "X_BMSparse" "lambda" "[0.01,0.02,0.05,0.1,0.001]" "dT" "3" "num_lags" "5" "kernel_width" ".5" "ID" "0" "replicate" "1" "family" "poisson" "date" "07/02/2019" "firsttarget" "1" "targetincr" "300" "prob_remove_samples" "0.3" "prob_zero_removal" "0.6"
[adeshpande4@submit-1 GitterNew]$ cat logs/hello-chtc_7936564_0.err

   Segmentation violation detected at Tue Jul 02 17:21:13 2019 -0500

Configuration:
Crash Decoding : Disabled - No sandbox or build area path
Crash Mode : continue (default)
Default Encoding : US-ASCII
Deployed : true
GNU C Library : 2.17 stable
Graphics Driver : Unknown software
MATLAB Architecture : glnxa64
MATLAB Entitlement ID : Unknown
MATLAB Root : /var/lib/condor/execute/slot1/dir_24929/v94
MATLAB Version : 9.4.0.813654 (R2018a)
OpenGL : software
Operating System : Linux 5.0.8-1.el7.elrepo.x86_64 #1 SMP Wed Apr 17 10:11:44 EDT 2019 x86_64
Process ID : 25220
Processor ID : x86 Family 6 Model 23 Stepping 10, GenuineIntel
Session Key : 8f018b8e-68d7-47a0-bf6d-26a0bf86030f
Static TLS mitigation : Disabled: Unable to open display
Window System : No active display

Fault Count: 1

Abnormal termination

Register State (from fault):
RAX = 0000000000003b16 RBX = 0000150d175d84a0
RCX = 0000150d175c9840 RDX = ffffffffc767d558
RSP = 0000150d13ffbe00 RBP = 0000150d13ffbf60
RSI = 0000150d17622240 RDI = 000000003fefffff

R8 = 0000000000000006 R9 = 0000000000000000
R10 = 0000150d17cf8830 R11 = 0000000000000005
R12 = 000000003ff00000 R13 = 0000000000000012
R14 = 0000000000000000 R15 = 0000150d17622250

RIP = 0000150cde6ea912 EFL = 0000000000010202

CS = 0033 FS = 0000 GS = 0000

Stack Trace (from fault):
[ 0] 0x0000150cde6ea912 /tmp/.mcrCache9.4/GLG_In0/glmnet_matlab/glmnetMex.mexa64+00252178
[ 1] 0x0000150cde6b566e /tmp/.mcrCache9.4/GLG_In0/glmnet_matlab/glmnetMex.mexa64+00034414 mexfunction_+00030329
[ 2] 0x0000150d2d2721ea bin/glnxa64/libmex.so+00414186
[ 3] 0x0000150d2d272447 bin/glnxa64/libmex.so+00414791
[ 4] 0x0000150d2d272f2b bin/glnxa64/libmex.so+00417579
[ 5] 0x0000150d2d25d30c bin/glnxa64/libmex.so+00328460
[ 6] 0x0000150d2edb52ad bin/glnxa64/libmwm_dispatcher.so+00979629 ZN8Mfh_file16dispatch_fh_implEMS_FviPP11mxArray_tagiS2_EiS2_iS2+00000829
[ 7] 0x0000150d2edb5bae bin/glnxa64/libmwm_dispatcher.so+00981934 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00000030
[ 8] 0x0000150d29ad7da1 bin/glnxa64/libmwm_lxe.so+12619169
[ 9] 0x0000150d29ad8982 bin/glnxa64/libmwm_lxe.so+12622210
[ 10] 0x0000150d29bc0e79 bin/glnxa64/libmwm_lxe.so+13573753
[ 11] 0x0000150d29b623e1 bin/glnxa64/libmwm_lxe.so+13186017
[ 12] 0x0000150d293685a8 bin/glnxa64/libmwm_lxe.so+04822440
[ 13] 0x0000150d2936acbc bin/glnxa64/libmwm_lxe.so+04832444
[ 14] 0x0000150d2936701d bin/glnxa64/libmwm_lxe.so+04816925
[ 15] 0x0000150d29360ba1 bin/glnxa64/libmwm_lxe.so+04791201
[ 16] 0x0000150d29360dd9 bin/glnxa64/libmwm_lxe.so+04791769
[ 17] 0x0000150d29366846 bin/glnxa64/libmwm_lxe.so+04814918
[ 18] 0x0000150d2936692f bin/glnxa64/libmwm_lxe.so+04815151
[ 19] 0x0000150d29495503 bin/glnxa64/libmwm_lxe.so+06055171
[ 20] 0x0000150d29498cf3 bin/glnxa64/libmwm_lxe.so+06069491
[ 21] 0x0000150d299a8f6d bin/glnxa64/libmwm_lxe.so+11378541
[ 22] 0x0000150d29ac57c4 bin/glnxa64/libmwm_lxe.so+12543940
[ 23] 0x0000150d29ac5d6b bin/glnxa64/libmwm_lxe.so+12545387
[ 24] 0x0000150d2edb52ad bin/glnxa64/libmwm_dispatcher.so+00979629 ZN8Mfh_file16dispatch_fh_implEMS_FviPP11mxArray_tagiS2_EiS2_iS2+00000829
[ 25] 0x0000150d2edb5bde bin/glnxa64/libmwm_dispatcher.so+00981982 ZN8Mfh_file22dispatch_fh_with_reuseEiPP11mxArray_tagiS2+00000030
[ 26] 0x0000150d29be5d4e bin/glnxa64/libmwm_lxe.so+13725006
[ 27] 0x0000150d29955416 bin/glnxa64/libmwm_lxe.so+11035670
[ 28] 0x0000150d2995558c bin/glnxa64/libmwm_lxe.so+11036044
[ 29] 0x0000150d299eaae8 bin/glnxa64/libmwm_lxe.so+11647720
[ 30] 0x0000150d299ec229 bin/glnxa64/libmwm_lxe.so+11653673
[ 31] 0x0000150d2ea14f80 bin/glnxa64/libmwm_interpreter.so+00688000 _Z44inCallFcnWithTrapInDesiredWSAndPublishEventsiPP11mxArray_tagiS1_PKcbP15inWorkSpace_tag+00000080
[ 32] 0x0000150d2d7b586d bin/glnxa64/libmwiqm.so+00768109 _ZN3iqm15BaseFEvalPlugin7executeEP15inWorkSpace_tagRN7mwboost10shared_ptrIN14cmddistributor17IIPCompletedEventEEE+00000525
[ 33] 0x0000150d301f14a1 bin/glnxa64/libmwmcr.so+00849057
[ 34] 0x0000150d2d7abab1 bin/glnxa64/libmwiqm.so+00727729
[ 35] 0x0000150d2d78ea95 bin/glnxa64/libmwiqm.so+00608917
[ 36] 0x0000150d301bffe5 bin/glnxa64/libmwmcr.so+00647141
[ 37] 0x0000150d301c06a4 bin/glnxa64/libmwmcr.so+00648868
[ 38] 0x0000150d301b93f1 bin/glnxa64/libmwmcr.so+00619505
[ 39] 0x0000150d37574dd5 /lib64/libpthread.so.0+00032213
[ 40] 0x0000150d3a1eaead /lib64/libc.so.6+01040045 clone+00000109
[ 41] 0x0000000000000000 +00000000

This error was detected while a MEX-file was running. If the MEX-file
is not an official MathWorks function, please examine its source code
for errors. Please consult the External Interfaces Guide for information
on debugging MEX-files.
** This crash report has been saved to disk as /tmp/matlab_crash_dump.25220-1 **

MATLAB is exiting because of fatal error
/var/lib/condor/execute/slot1/dir_24929/condor_exec.exe: line 40: 25220 Killed "/var/lib/condor/execute/slot1/dir_24929/GLG_Instance" "X_BMSparse" "lambda" "[0.01,0.02,0.05,0.1,0.001]" "dT" "3" "num_lags" "5" "kernel_width" ".5" "ID" "0" "replicate" "1" "family" "poisson" "date" "07/02/2019" "firsttarget" "1" "targetincr" "300" "prob_remove_samples" "0.3" "prob_zero_removal" "0.6"

Running SCINGE_Example on Windows 10 with MATLAB R2014b generated a system error: MATLAB has encountered an internal problem and needs to close.

The error details include the message Access violation detected. The first lines of the stack trace (with paths edited) are:

Stack Trace (from fault):
[  0] 0x00000000fe160b03 ...\MATLAB\R2014b\bin\win64\libmx.dll+00396035 MXGETPR+00000003
[  1] 0x00007ff879dc19dc ...\SCINGE\glmnet_matlab\glmnetMex.mexw64+00006620 MEXFUNCTION+00002524

Possibly related to these issues:

• https://stackoverflow.com/questions/45323117/matlabs-run-and-time-crashes-with-mex-file
• https://stackoverflow.com/questions/31388417/matlab-mex-file-crashing-every-third-run

Re-compiling the Glmnet code may be the next step.

Issue: Depending on the data size, large TempMat*.mat files are created during SINGE_GLG_Test, which could overrun user's storage budget leading to subsequent jobs being held.
Fix: We should include the line
eval "rm Temp*.mat"
near the end of run_SINGE_GLG_Test.sh

EDIT: The rm should be more targeted, especially if the storage is shared, with only Temp_<ID>.mat to be deleted.

Support for compatibility with simple tab-separated text files containing expression data, gene lists and so on, instead of the MAT files currently used.

@Alexander-Barth well over a decade ago, you wrote to the octave-maintainers list about interfacing Fortran and Octave with the mex API. I'm now facing a similar issue while attempting to port a MATLAB program to Octave on this project. Octave does not support a Fortran mex interface, so I'm curious whether you ever pursued this further. Thanks.

I am trying to run the SINGE_Example.m in MATLABR2020a on macOS Catalina.

ver -support

MATLAB Version: 9.8.0.1873465 (R2020a) Update 8
MATLAB License Number: 40707400
Operating System: Mac OS X Version: 10.15.7 Build: 19H1824
Java Version: Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode

MATLAB Version 9.8 (R2020a) License 40707400

I get the following Warning message:

Warning: from glmnet Fortran code (error code -5); Convergence for 5th lambda value not reached
after maxit=10000 iterations; solutions for larger lambdas returned

In elnet (line 33)
In glmnet (line 443)
In iLasso_for_SINGE (line 111)
In run_iLasso_row (line 27)
In SINGE_GLG_Test (line 79)
In SINGE (line 20)
In SINGE_Example (line 16)

After several iterations MATLAB crashes.
According to MathWorks technical support, the crash was detected while the MEX-file glmnetMex.mexmaci64 was running.
Any suggestion to solve this issue?