scitools / conda-gitenv Goto Github PK

View Code? Open in Web Editor NEW

11.0 11.0 10.0 84 KB

⛔ [DEPRECATED] Track conda environment specifications using a git repo

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

conda-gitenv's People

Contributors

Stargazers

Watchers

Forkers

corinnebosley mahatmacane pp-mo djkirkham bjlittle dpeterk lbdreyer pombredanne cedadev hercules261188

conda-gitenv's Issues

CI test failing

As seen in #34 (comment), the CI tests are failing with what looks like it could be a change in the gitpython codebase.

We need to investigate and fix.

Disable use cache in index resolution

https://github.com/SciTools/conda-gitenv/blob/master/conda_env_tracker/cli.py#L38

generated env.manifest should have final newline

Make a valid text file, and better for the examples where you indicate to cat env.manifest

Remove dependency on conda execute

Manifest doesn't match what I'd expect

I'm trying to build samtools-1.7-2. Using straight conda as follows it installs fine:

$ conda create --override-channels -c conda-forge -c bioconda -c default -n samtooltest samtools

Fetching package metadata .................
Solving package specifications: .

Package plan for installation in environment /.../miniconda3/envs/samtooltest:

The following NEW packages will be INSTALLED:

    bzip2:           1.0.6-1          conda-forge
    ca-certificates: 2018.1.18-0      conda-forge
    curl:            7.55.1-0         conda-forge
    krb5:            1.14.2-0         conda-forge
    libgcc:          7.2.0-h69d50b8_2            
    libgcc-ng:       7.2.0-h7cc24e2_2            
    libssh2:         1.8.0-2          conda-forge
    libstdcxx-ng:    7.2.0-h7a57d05_2            
    ncurses:         5.9-10           conda-forge
    openssl:         1.0.2n-0         conda-forge
    samtools:        1.7-2            bioconda   
    xz:              5.2.3-0          conda-forge
    zlib:            1.2.11-0         conda-forge

Using conda gitenv my env.spec looks like this:

channels:
 - conda-forge
 - bioconda
 - default
env: 
 - samtools

But the generated manifest is nothing like the one above, specifically the samtools version:

https://conda.anaconda.org/conda-forge/linux-64	ca-certificates-2018.1.18-0
https://conda.anaconda.org/conda-forge/linux-64	curl-7.55.1-0
https://conda.anaconda.org/conda-forge/linux-64	krb5-1.14.2-0
https://conda.anaconda.org/conda-forge/linux-64	libssh2-1.8.0-2
https://conda.anaconda.org/conda-forge/linux-64	ncurses-5.9-10
https://conda.anaconda.org/conda-forge/linux-64	openssl-1.0.2n-0
https://conda.anaconda.org/bioconda/linux-64	samtools-1.3.1-4
https://conda.anaconda.org/conda-forge/linux-64	zlib-1.2.11-0

p.s. I think this tool could be very beneficial for our team and the premise is excellent.

Deal with cleanup

Empty unused pkgs from the package cache
Remove unused deployed tags (i.e. those without a label), considering a gentle deprecation process

Ability to deploy single environment

It would be useful if conda gitenv deploy could deploy as specific environment as well as deploying all of them.

After talking through seeing this deployed for real, it is clear that there is an ambition to be able to define one spec for multiple architectures and environments. Aside from the fact that the env.spec currently produces just one env.manifest, there are specialisations which would be useful to be able to add on a per-platform basis.

Something like:

primary env.spec:

env:
 - python 2.7.*
 - iris 1.9.*

site specific env.spec:

extends: primary
env:
 - secret_sauce 3.14.*

platform site specific env.spec:

extends: site specific (these are just names, and will probably just reference git repos)
env:
 - my_platform_specific_package
features:
 - mkl

Of course, conda uses selectors for this purpose, but:

our specifications (currently) are on a per platform basis (i.e. there is 1 env.spec for 1 manifest)
we do not want a specific platform support team to necessarily have merge permissions on the global environment specification
for site specific software (which may have no public release) it is desirable to be able to define these in a non-public facing environment specification whilst sharing as much as possible the core environment

Re-create links seems to fail

Traceback (most recent call last):
File "conda-gitenv", line 9, in
load_entry_point('conda-gitenv==0.1.1.dev1+g677f4f6', 'console_scripts', 'conda-gitenv')()
File "lib/python3.4/site-packages/conda_gitenv/cli.py", line 22, in main
return args.function(args)
File "lib/python3.4/site-packages/conda_gitenv/deploy.py", line 144, in handle_args
deploy_repo(repo, args.target)
File "lib/python3.4/site-packages/conda_gitenv/deploy.py", line 131, in deploy_repo
os.symlink(label_target, label_location)
FileExistsError: [Errno 17] File exists: '2015_12_16-2' -> 'default/current'

Why not use environment.yaml with fully qualified URLs?

Work on CLI interface

n.b. currently:

conda-env-tracker <repo>
conda-env-tracker-timestamp <repo>
conda-env-tracker-labeltag <repo>
conda-env-tracker-deploy <repo> <destination>

It would also be nice to come up with a suitable name which covers all of this functionality (i.e. the "conda environments in a git repo" bit).

Env branch with '-' can't be deployed

I suspect this is due to matching against the tag names.

For example an environment of casm-perl will resolve and tag, but not deploy. No warnings are emitted. Renaming it to casmPerl handles the problem.

Suggest having gitenv resolve throw an error when a hyphenated branch is detected.

Documentation

E.g. Get something up on readthedocs...?