rivas-lab / snpnet Goto Github PK

snpnet - Efficient Lasso Solver for Large-scale genetic variant data

R 89.86% Shell 9.30% Dockerfile 0.84%

statistical-learning lasso prediction genetics biobank snpnet-cox cox-regression sparse linear-regression survival-analysis

snpnet's Introduction

Snpnet - Efficient Lasso Solver for Large-scale SNP Data

License: GPL-2

Reference:

Junyang Qian, Yosuke Tanigawa, Wenfei Du, Matthew Aguirre, Chris Chang, Robert Tibshirani, Manuel A. Rivas, Trevor Hastie. "A Fast and Scalable Framework for Large-Scale and Ultrahigh-Dimensional Sparse Regression with Application to the UK Biobank." PLOS Genetics. 16, e1009141 (2020). https://doi.org/10.1371/journal.pgen.1009141
Ruilin Li, Christopher Chang, Johanne M. Justesen, Yosuke Tanigawa, Junyang Qiang, Trevor Hastie, Manuel A. Rivas, Robert Tibshirani. "Fast Lasso Method for Large-Scale and Ultrahigh-Dimensional Cox Model with Applications to UK Biobank." Biostatistics. kxaa038, 2020. https://doi.org/10.1093/biostatistics/kxaa038.

Installation:

Most of the requirements of snpnet are available from CRAN. It also depends on the pgenlibr, glmnet/glmnetPlus and cindex (for survival analysis) packages. One can install them by running the following commands in R. Notice that the installation of pgenlibr requires zstd(>=1.4.4). It can be built from source or simply available from conda, pip or brew.

library(devtools)
install_github("junyangq/glmnetPlus")
install_github("chrchang/plink-ng", subdir="/2.0/cindex")
install_github("chrchang/plink-ng", subdir="/2.0/pgenlibr")

We assume the users already have PLINK 2.0. Otherwise it can be installed from https://www.cog-genomics.org/plink/2.0/.

snpnet's People

Contributors

Stargazers

Watchers

Forkers

galateabio cdadams-harvard magdazeb

snpnet's Issues

Error: Invalid --memory argument '-1224'.

Can anyone help me understand this error?

[2021-02-18 12:09:08 snpnet] Iteration 35
[2021-02-18 12:10:31 snpnet]     Start computeProduct()
[2021-02-18 12:10:31 snpnet]       Start plink2 --variant-score
PLINK v2.00a3 AVX2 (23 Sep 2020)               www.cog-genomics.org/plink/2.0/
(C) 2005-2020 Shaun Purcell, Christopher Chang   GNU General Public License v3
Logging to /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_35.log.
Options in effect:
  --keep /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_35.tsv
  --memory 1253
  --out /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_35
  --pfile ukb_imp_chr1_v3_trimmed_2m_remove_geno_with_NA_pheno vzs
  --read-freq /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/meta/snpnet.train.gcount
  --threads 2
  --variant-score /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_35.tsv zs bin

Start time: Thu Feb 18 12:10:33 2021
16384 MiB RAM detected; reserving 1253 MiB for main workspace.
Using up to 2 compute threads.
92546 samples (48786 females, 43760 males; 92546 founders) loaded from
ukb_imp_chr1_v3_trimmed_2m_remove_geno_with_NA_pheno.psam.
71109 variants loaded from
ukb_imp_chr1_v3_trimmed_2m_remove_geno_with_NA_pheno.pvar.zst.
1 quantitative phenotype loaded (92546 values).
--keep: 92546 samples remaining.
92546 samples (48786 females, 43760 males; 92546 founders) remaining after main
filters.
92546 quantitative phenotype values remaining after main filters.
--read-freq: PLINK 2 --geno-counts file detected.
--read-freq: Frequencies for 71109 variants loaded.
--variant-score: 2 score-vectors loaded for 92546 samples.
--variant-score: Score matrix written to
/var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_35.vscore.bin
, and associated column and variant ID labels written to
/var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_35.vscore.cols
and
/var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_35.vscore.vars.zst
.
End time: Thu Feb 18 12:10:40 2021
[2021-02-18 12:10:40 snpnet]         End plink2 --variant-score. Time elapsed: 9.2781 secs
[2021-02-18 12:10:40 snpnet]       End computeProduct(). Time elapsed: 9.2811 secs
[2021-02-18 12:10:40 snpnet]   KKT check obj done ...
[2021-02-18 12:10:40 snpnet] metric train
[2021-02-18 12:10:40 snpnet]   Training metric:
[2021-02-18 12:10:40 snpnet]   - Lambda idx 65. Training: 0.0850063908352156. 
[2021-02-18 12:10:40 snpnet]   End iteration 35. Time elapsed: 1.5287 mins
[2021-02-18 12:10:40 snpnet]     The total time since start. Time elapsed: 19.2366 mins
[2021-02-18 12:10:40 snpnet] Iteration 36
[2021-02-18 12:12:28 snpnet]     Start computeProduct()
[2021-02-18 12:12:28 snpnet]       Start plink2 --variant-score
PLINK v2.00a3 AVX2 (23 Sep 2020)               www.cog-genomics.org/plink/2.0/
(C) 2005-2020 Shaun Purcell, Christopher Chang   GNU General Public License v3
Logging to /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_36.log.
Options in effect:
  --keep /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_36.tsv
  --memory 896
  --out /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_36
  --pfile ukb_imp_chr1_v3_trimmed_2m_remove_geno_with_NA_pheno vzs
  --read-freq /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/meta/snpnet.train.gcount
  --threads 2
  --variant-score /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_36.tsv zs bin

Start time: Thu Feb 18 12:12:32 2021
16384 MiB RAM detected; reserving 896 MiB for main workspace.
Using up to 2 compute threads.
92546 samples (48786 females, 43760 males; 92546 founders) loaded from
ukb_imp_chr1_v3_trimmed_2m_remove_geno_with_NA_pheno.psam.
71109 variants loaded from
ukb_imp_chr1_v3_trimmed_2m_remove_geno_with_NA_pheno.pvar.zst.
1 quantitative phenotype loaded (92546 values).
--keep: 92546 samples remaining.
92546 samples (48786 females, 43760 males; 92546 founders) remaining after main
filters.
92546 quantitative phenotype values remaining after main filters.
--read-freq: PLINK 2 --geno-counts file detected.
--read-freq: Frequencies for 71109 variants loaded.
--variant-score: 2 score-vectors loaded for 92546 samples.
--variant-score: Score matrix written to
/var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_36.vscore.bin
, and associated column and variant ID labels written to
/var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_36.vscore.cols
and
/var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_36.vscore.vars.zst
.
End time: Thu Feb 18 12:12:38 2021
[2021-02-18 12:12:38 snpnet]         End plink2 --variant-score. Time elapsed: 9.8992 secs
[2021-02-18 12:12:38 snpnet]       End computeProduct(). Time elapsed: 9.9019 secs
[2021-02-18 12:12:38 snpnet]   KKT check obj done ...
[2021-02-18 12:12:38 snpnet]   End iteration 36. Time elapsed: 1.9689 mins
[2021-02-18 12:12:38 snpnet]     The total time since start. Time elapsed: 21.2055 mins
[2021-02-18 12:12:38 snpnet] Iteration 37
[2021-02-18 12:26:59 snpnet]     Start computeProduct()
[2021-02-18 12:26:59 snpnet]       Start plink2 --variant-score
PLINK v2.00a3 AVX2 (23 Sep 2020)               www.cog-genomics.org/plink/2.0/
(C) 2005-2020 Shaun Purcell, Christopher Chang   GNU General Public License v3
Logging to /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_37.log.
Options in effect:
  --keep /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_37.tsv
  --memory -1224
  --out /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_37
  --pfile ukb_imp_chr1_v3_trimmed_2m_remove_geno_with_NA_pheno vzs
  --read-freq /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/meta/snpnet.train.gcount
  --threads 2
  --variant-score /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_37.tsv zs bin

Start time: Thu Feb 18 12:27:05 2021
Error: Invalid --memory argument '-1224'.
For more info, try "plink2 --help <flag name>" or "plink2 --help | more".
zstd: can't stat /var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_37.vscore.vars.zst : No such file or directory -- ignored 
Error in data.table::fread(paste0(fhead, ".cols"), head = F) : 
  File '/var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/results/residuals_iter_37.vscore.cols' does not exist or is non-readable. getwd()=='/Users/garyzhubc/Desktop/hli/data'
In addition: Warning message:
In data.table::fread(cmd = paste0(configs[["zstdcat.path"]], " ",  :
  File '/var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T//RtmpLoYQ1B/file14c417a3e4435' has size 0. Returning a NULL data.table.

Use `mem.save=TRUE` option in `glmnetPlus::glmnet()` for gaussian family

We might want to consider the use of mem.save=TRUE option in glmnetPlus::glmnet() for gaussian family.

https://github.com/junyangq/glmnetPlus/pull/5/files

Is it normal to have a lot of variables with rank 100 when L=100 (default)?

Installation error: Bad CPU type in executable

When I was trying to install glmnetPlus on latest Mac OS this error showed up so that installation cannot complete.

> install_github("junyangq/glmnetPlus")
Downloading GitHub repo junyangq/glmnetPlus@HEAD
✓  checking for file ‘/private/var/folders/w8/9_5f04c54b585y1kjq898d180000gn/T/RtmpkfueRA/remotesac6688274dc/junyangq-glmnetPlus-755b0e5/DESCRIPTION’ ...
─  preparing ‘glmnetPlus’:
✓  checking DESCRIPTION meta-information ...
─  cleaning src
✓  checking vignette meta-information ...
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  looking to see if a ‘data/datalist’ file should be added
─  building ‘glmnetPlus_2.0-16.tar.gz’
   
* installing *source* package ‘glmnetPlus’ ...
** using staged installation
** libs
gfortran -mmacosx-version-min=10.13 -fno-optimize-sibling-calls  -fPIC  -Wall -g -O2  -c glmnet5dp.f -o glmnet5dp.o
gfortran-4.2: error trying to exec '/usr/local/bin/i686-apple-darwin8-gfortran-4.2': execvp: Bad CPU type in executable

myglmnet not available for snpnet installation

Hello!

I'm not sure if this is correct venue to submit an issue as it may be user error so apologies if there is a better way to do this.

I'm installing the snpnet package on a linux machine using R version 4.1.0 as per these instructions: https://github.com/rivas-lab/snpnet/tree/compact.

I'm getting this error message:
ERROR: dependency ‘myglmnet’ is not available for package ‘snpnet’

It looks like myglmnet isn't available on CRAN or in github any longer (devtools::install_github("harryprince/myglmnet") failed).

Thank you!
Keri

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.