Giter Club home page Giter Club logo

setup-miniconda's Introduction

conda-incubator/setup-miniconda

This action sets up a base conda environment by one of:

  • locating the conda installation bundled with the available runners and available in $CONDA
  • installing a specific (or latest) version of

A conda-build-version or mamba-version may be provided to install specific versions of conda or mamba into base.

The base condabin/ folder is added to $PATH and shell integration is initialized across all platforms.

By default, this action will then create, and activate, an environment by one of:

  • creating a mostly-empty test environment, containing only the latest python-version and its dependencies
  • creating a test environment described in a given environment-file including:
    • an environment.yml-like file (which can be patched with python-version). Note: the patched environment will be cleaned up unless clean-patched-environment-file: false is given
    • a lockfile

This action correctly handles activation of environments and offers the possibility of automatically activating the test environment on all shells.

Please see the IMPORTANT notes on additional information on environment activation.

Example Overview

Each of the examples below is discussed in a dedicated section below.

Documentation Workflow Status
Basic usage Basic Usage Status
Other shells Other Shells Status
Other options Other Options Status
Channels Channels Status
Custom installer Custom Installer Status
Mamba Mamba Status
Lockfiles Lockfiles Status
Miniforge Miniforge Status
Alternative Architectures Alternative Architectures
Configure conda solver Configure conda solver
Caching packages Caching Example Status
Caching environments Caching Env Example Status
Apple Silicon Apple Silicon

Other Workflows

These are quality control and test workflows, and are not described in depth.

QA Workflow Linting Catch Invalid Enviroments Handle Empty Channels
Workflow Status Linting Status Catch Invalid Environments Status Handle Empty Channels Status

Environment activation

This action will, by default, activate an environment called test and not activate the base environment. This encourages the recommended practice of not installing workflow packages into the base environment and leaving it with only conda (and/or mamba).

Inputs and outputs

For a full list of available inputs and outputs for this action see action.yml.

Use a different environment name or path

You can change the default test environment to have a different name or path by setting the activate-environment input option.

- uses: conda-incubator/setup-miniconda@v3
  with:
    activate-environment: whatever

This will create a named env in $CONDA/envs/whatever, where $CONDA is the path to the infrequently-updated, but very fast to start, "bundled" Miniconda installation.

  • If activate-environment contains either POSIX or Windows slashes, it will be interpreted as a path, or prefix in conda terminology. Use this to avoid "path too long"-style errors, especially on Windows.
  • Self-hosted runners can emulate the "bundled" Miniconda approach by pre-installing a constructor-based installer and ensuring $CONDA is set prior to starting setup-miniconda

Activate base environment

If your specific workflow still needs to activate and use base you will need to do both of:

  • set activate-environment to an empty string
  • set auto-activate-base to true
- uses: conda-incubator/setup-miniconda@v3
  with:
    auto-activate-base: true
    activate-environment: ""

Usage examples

Example 1: Basic usage

This example shows how to set a basic python workflow with conda using the cross-platform available shells: bash and pwsh. In this example an environment named test will be created with the specific python-version installed for each operating system, resulting in 6 build workers.

jobs:
  example-1:
    name: Ex1 (${{ matrix.python-version }}, ${{ matrix.os }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: ["ubuntu-latest", "macos-latest", "windows-latest"]
        python-version: ["3.7", "3.11"]
    steps:
      - uses: conda-incubator/setup-miniconda@v3
        with:
          auto-update-conda: true
          python-version: ${{ matrix.python-version }}
      - name: Conda info
        shell: bash -el {0}
        run: conda info
      - name: Conda list
        shell: pwsh
        run: conda list

Example 2: Other shells

This example shows how to use all other available shells for specific operating systems. In this example we download the latest anaconda version then create and activate a default environment named foo.

jobs:
  example-2-linux:
    name: Ex2 Linux
    runs-on: "ubuntu-latest"
    steps:
      - uses: conda-incubator/setup-miniconda@v3
        with:
          miniconda-version: "latest"
          activate-environment: foo
      - name: Bash
        shell: bash -el {0}
        run: |
          conda info
          conda list
      - name: PowerShell Core
        shell: pwsh
        run: |
          conda info
          conda list

  example-2-mac:
    name: Ex2 Mac
    runs-on: "macos-latest"
    steps:
      - uses: conda-incubator/setup-miniconda@v3
        with:
          miniconda-version: "latest"
          activate-environment: foo
      - name: Sh
        shell: sh -l {0}
        run: |
          conda info
          conda list
      - name: Bash
        shell: bash -el {0}
        run: |
          conda info
          conda list
      - name: PowerShell Core
        shell: pwsh
        run: |
          conda info
          conda list

  example-2-win:
    name: Ex2 Windows
    runs-on: "windows-latest"
    steps:
      - uses: conda-incubator/setup-miniconda@v3
        with:
          miniconda-version: "latest"
          activate-environment: foo
      - name: Bash
        shell: bash -el {0}
        run: |
          conda info
          conda list
      - name: PowerShell
        shell: powershell
        run: |
          conda info
          conda list
      - name: PowerShell Core
        shell: pwsh
        run: |
          conda info
          conda list
      - name: Cmd.exe
        shell: cmd /C CALL {0}
        run: >-
          conda info && conda list

Example 3: Other options

This example shows how to use environment.yml for easier creation of test/build environments and .condarc files for fine grained configuration management. In this example we use a custom configuration file, install an environment from a yaml file, and disable autoactivating the base environment before activating the anaconda-client-env.

jobs:
  example-3:
    name: Ex3 Linux
    runs-on: "ubuntu-latest"
    defaults:
      run:
        shell: bash -el {0}
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          activate-environment: anaconda-client-env
          environment-file: etc/example-environment.yml
          python-version: 3.5
          condarc-file: etc/example-condarc.yml
          auto-activate-base: false
      - run: |
          conda info
          conda list

Example 4: Conda options

This example shows how to use the channels option and other extra options. The priority will be set by the order of the channels. The following example will result in these priorities (from highest to lowest):

  • conda-forge
  • spyder-ide
  • defaults
jobs:
  example-4:
    name: Ex4 Linux
    runs-on: "ubuntu-latest"
    defaults:
      run:
        shell: bash -el {0}
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          activate-environment: foo
          python-version: 3.6
          channels: conda-forge,spyder-ide
          allow-softlinks: true
          channel-priority: flexible
          show-channel-urls: true
          use-only-tar-bz2: true
      - run: |
          conda info
          conda list
          conda config --show-sources
          conda config --show

Example 5: Custom installer

Any installer created with the constructor tool (which includes conda) can be used in place of Miniconda. For example, conda-forge maintains additional builds of miniforge for platforms not yet supported by Miniconda. For more details, see Example 10.

Note:

  • Installer downloads are cached based on their full URL: adding some non-functional salt to the URL will prevent this behavior, e.g., #${{ github.run_number }}
jobs:
  example-5:
    name: Ex5 Miniforge for PyPy
    runs-on: "ubuntu-latest"
    defaults:
      run:
        shell: bash -el {0}
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          installer-url: https://github.com/conda-forge/miniforge/releases/download/4.8.3-2/Miniforge-pypy3-4.8.3-2-Linux-x86_64.sh
          allow-softlinks: true
          show-channel-urls: true
          use-only-tar-bz2: true
      - run: |
          conda info
          conda list
          conda config --show-sources
          conda config --show

Example 6: Mamba

Note: conda 23.10+ uses conda-libmamba-solver by default, which provides comparable performance to mamba. Most users won't need this setting with recent conda versions.

Experimental! Use mamba to enable much faster conda installs. mamba-version accepts a version string x.y (including "*"). It requires you specify conda-forge as part of the channels, ideally with the highest priority.

Notes:

  • If a custom installer provides mamba, it can be prioritized wherever possible (including installing mamba-version) with use-mamba: true.
jobs:
  example-6:
    name: Ex6 Mamba
    runs-on: "ubuntu-latest"
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          python-version: 3.6
          mamba-version: "*"
          channels: conda-forge,defaults
          channel-priority: true
          activate-environment: anaconda-client-env
          environment-file: etc/example-environment.yml
      - shell: bash -el {0}
        run: |
          conda info
          conda list
          conda config --show-sources
          conda config --show
          printenv | sort
      - shell: bash -el {0}
        run: mamba install jupyterlab

Example 7: Lockfiles

conda list --explicit and conda-lock support generating explicit environment specifications, which skip the environment solution step altogether, as they contain the ordered list of exact URLs needed to reproduce the environment.

This means explicitly-defined environments which:

  • are much faster to install, as several expensive steps are skipped:
    • channels are not queried for their repo data
    • no solver is run
  • are not cross-platform, as the URLs almost always contain platform/architecture information
  • can become broken if any file becomes unavailable

This approach can be useful as part of a larger system e.g., a separate workflow that runs conda-lock for all the platforms needed in a separate job.

jobs:
  example-7:
    name: Ex7 Explicit
    runs-on: "ubuntu-latest"
    defaults:
      run:
        shell: bash -el {0}
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          auto-update-conda: false
          activate-environment: explicit-env
          environment-file: etc/example-explicit.conda.lock
      - run: |
          conda info
          conda list
          conda config --show-sources
          conda config --show
          printenv | sort

Example 10: Miniforge

Miniforge provides a number of alternatives to Miniconda, built from the ground up with conda-forge packages and with only conda-forge in its default channel(s).

If only miniforge-version is provided then Miniforge3 will be used.

jobs:
  example-10-miniforge:
    name: Ex10 (${{ matrix.os }}, Miniforge)
    runs-on: ${{ matrix.os }}-latest
    strategy:
      matrix:
        os: ["ubuntu", "macos", "windows"]
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          environment-file: etc/example-environment.yml
          miniforge-version: latest

In addition to Miniforge3 with conda, mamba and CPython, you can also install Miniforge-pypy3, which replaces CPython with `PyPy.

Example 11: Alternative Architectures

In addition to the default 64-bit builds of Miniconda, 32-bit versions are available for Windows. Note that although some x86 builds are available for Linux and MacOS, these are too old (<4.6) to be supported by this action.

jobs:
  example-11:
    name:
      Ex11 (os=${{ matrix.os }} architecture=${{ matrix.architecture }}
      miniconda-version=${{ matrix.miniconda-version }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: ["windows-latest"]
        architecture: ["x86"]
        miniconda-version: ["latest"]
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          architecture: ${{ matrix.architecture }}
          miniconda-version: $${{ matrix.miniconda-version }}
          auto-update-conda: true
          python-version: "3.8"

Example 12: Configure conda solver

Set the conda solver plugin to use. Only applies to the conda client, not mamba. Starting with Miniconda 23.5.2 and Miniforge 23.3.1, you can choose between classic and libmamba. Best when combined with auto-update-conda: true.

jobs:
  example-12:
    name: Ex12 (os=${{ matrix.os }} solver=${{ matrix.solver }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        solver: ["classic", "libmamba"]
        os: ["ubuntu-latest", "windows-latest"]
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        id: setup-miniconda
        continue-on-error: true
        with:
          auto-update-conda: true
          conda-solver: ${{ matrix.solver }}
          python-version: "3.9"

Example 13: Apple Silicon

jobs:
  example-13:
    name: Ex13 (os=${{ matrix.os }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: ["macos-14"]
    steps:
      - uses: actions/checkout@v4
      - uses: ./
        id: setup-miniconda
        continue-on-error: true
        with:
          miniconda-version: latest
      - name: Check arm64
        shell: bash -el {0}
        run: |
          conda install -y python
          python -c "import platform; assert platform.machine() == 'arm64', platform.machine()"

Caching

Caching packages

If you want to enable package caching for conda you can use the cache action using ~/conda_pkgs_dir as path for conda packages.

The cache will use an explicit key for restoring and saving the cache.

This can be based in the contents of files like:

  • setup.py
  • requirements.txt
  • environment.yml
jobs:
  caching-example:
    name: Caching
    runs-on: "ubuntu-latest"
    steps:
      - uses: actions/checkout@v4
      - name: Cache conda
        uses: actions/cache@v3
        env:
          # Increase this value to reset cache if etc/example-environment.yml has not changed
          CACHE_NUMBER: 0
        with:
          path: ~/conda_pkgs_dir
          key:
            ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{
            hashFiles('etc/example-environment.yml') }}
      - uses: conda-incubator/setup-miniconda@v3
        with:
          activate-environment: anaconda-client-env
          channel-priority: strict
          environment-file: etc/example-environment.yml
          use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!

If you are using pip to resolve any dependencies in your conda environment then you may want to cache those dependencies separately, as they are not included in the conda package cache.

Caching environments

The first installation step should setup a Miniconda variant without specifying a environment file.

- name: Setup Miniforge
  uses: conda-incubator/setup-miniconda@v3
  with:
    miniforge-version: latest
    activate-environment: anaconda-client-env

It's a good idea to refresh the cache every 24 hours to avoid inconsistencies of package versions between the CI pipeline and local installations. Here we ensure that this happens by adding the current date to the cache key. You can remove the "Get Date" step below if you use a resolved environment file product of conda env export or conda list --explicit.

- name: Get Date
  id: get-date
  run: echo "today=$(/bin/date -u '+%Y%m%d')" >> $GITHUB_OUTPUT
  shell: bash

- name: Cache Conda env
  uses: actions/cache@v3
  with:
    path: ${{ env.CONDA }}/envs
    key:
      conda-${{ runner.os }}--${{ runner.arch }}--${{
      steps.get-date.outputs.today }}-${{
      hashFiles('etc/example-environment-caching.yml') }}-${{ env.CACHE_NUMBER
      }}
  env:
    # Increase this value to reset cache if etc/example-environment.yml has not changed
    CACHE_NUMBER: 0
  id: cache

Keep in mind that hashing etc/example-environment-caching.yml is not the same as hashing a resolved environment file. conda (and mamba) resolves the dependencies declared in the YAML file according to the packages available on the channels at installation time. Since packages are updated all the time, you will not see these changes reflected in the cache until the key gets updated by date.

This means that the same environment file can make your tests pass locally but fail on CI, or the other way around. In that case, reset the cache manually to see if that leads to consistent results, or use a resolved environment file.

Finally, update the environment based on the environment file if the cache does not exist.

- name: Update environment
  run:
    mamba env update -n anaconda-client-env -f
    etc/example-environment-caching.yml
  if: steps.cache.outputs.cache-hit != 'true'

Use a default shell

If you use the same shell for every step in your workflow you don't have to add a shell directive to every step (e.g., shell: bash -el {0} when using bash).

You can add a defaults section and specify the desired directive (e.g., bash -el {0} or equivalent). All steps in the job will then default to using that value.

For other shells, make sure to use the correct shell parameter as the default value. Check the section below for some examples.

For more information see the Github Actions help page.

jobs:
  default-shell:
    name: Default shell
    runs-on: "ubuntu-latest"
    defaults:
      run:
        shell: bash -el {0}
    steps:
      - uses: actions/checkout@v4
      - uses: conda-incubator/setup-miniconda@v3
        with:
          activate-environment: anaconda-client-env
          environment-file: etc/example-environment-caching.yml
      - run: conda info
      - run: conda list
      - run: conda config --show

IMPORTANT

  • Conda activation does not correctly work on sh. Please use bash.
  • Bash shells do not use ~/.profile or ~/.bashrc so these shells need to be explicitly declared as shell: bash -el {0} on steps that need to be properly activated (or use a default shell). This is because bash shells are executed with bash --noprofile --norc -eo pipefail {0} thus ignoring updated on bash profile files made by conda init bash. See Github Actions Documentation and this community thread.
  • Sh shells do not use ~/.profile or ~/.bashrc so these shells need to be explicitly declared as shell: sh -l {0} on steps that need to be properly activated (or use a default shell). This is because sh shells are executed with sh -e {0} thus ignoring updates on bash profile files made by conda init bash. See Github Actions Documentation.
  • Cmd shells do not run Autorun commands so these shells need to be explicitly declared as shell: cmd /C call {0} on steps that need to be properly activated (or use a default shell). This is because cmd shells are executed with %ComSpec% /D /E:ON /V:OFF /S /C "CALL "{0}"" and the /D flag disables execution of Command Processor/Autorun Windows registry keys, which is what conda init cmd.exe sets. See Github Actions Documentation.
  • For caching to work properly, you will need to set the use-only-tar-bz2 option to true.
  • Some options (e.g. use-only-tar-bz2) are not available on the default conda installed on Windows VMs, be sure to use auto-update-conda or provide a version of conda compatible with the option.
  • If you plan to use a environment.yaml file to set up the environment, the action will read the channels listed in the key (if found). If you provide the channels input in the action they must not conflict with what was defined in environment.yaml, otherwise the conda solver might find conflicts which cause very long install times or install failures.

Security / Reproducibility

Security and reproducibility is important especially when workflows deal with secrets. No matter how much individual Github action repositories are secured, git branches and tags are always mutable. It is thus good practice to:

  1. pin the action to a specific sha1 with tag as comment, instead of e.g. using v2 or v2.2.1 (which are mutable tags): uses: conda-incubator/setup-miniconda@9f54435e0e72c53962ee863144e47a4b094bfd35 # v2.3.0 see example
  2. keep the non-human-readable pinning updated to not run behind recent updates and fixes via automation like renovate or dependabot
  3. use conda-lock files, see conda-lock

Project History and Contributing

See the CHANGELOG for project history, or CONTRIBUTING to get started adding features you need.

Similar Actions to work with conda packages

Contributors

Thanks to all the contributors that make this awesome project possible!

Meet our contributors!

Made with contributors-img.

License

The scripts and documentation in this project are released under the MIT License

setup-miniconda's People

Contributors

andreasalbertqc avatar bollwyvl avatar bryant1410 avatar bryanwweber avatar chrisjbillington avatar cjmartian avatar dbast avatar dependabot[bot] avatar epassaro avatar goanpeca avatar isuruf avatar jaimergp avatar jezdez avatar jjhelmus avatar jonashaag avatar klieret avatar ktbarrett avatar lnaden avatar marcelm avatar marcoesters avatar martin-g avatar mathieu1 avatar mgough avatar mxschmitt avatar ras44 avatar veckothegecko avatar yuvipanda avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

setup-miniconda's Issues

Problem with environment file

Hi! I can't create my env with a YML file located at the top of my GitHub repository. I don't know what I'm doing wrong. Tried a lot of combinations but always end with this NOENT error.

##[error]ENOENT: no such file or directory, open '/home/runner/work/gh_actions/gh_actions/tardis_env3.yml'

Supporting arbitrary constructors?

Thanks for this tool!

While the name of the project says it all, it seems like the action could install and use any constructor-generated installer: the immediate candidate would be miniforge, and I'm kinda itching to try out their pypy releases for speedier CI (haven't really benchmarked it yet). But I also see the potential for just any-old-constructor, especially as part of a longer pipeline.

The most blunt instrument, just an installer-url, could override all the detection, listing, etc. mechanisms. I'm very new to github actions, but would be interested in looking into this, if you are amenable!

Python packages installed with pip are not in PATH

Using windows-latest as OS and trying to install a few Python wheels using pip I encounter the following warning messages that then lead to the packages not being imported when running tests.

Installing collected packages: GDAL, numpy, scipy, future, Cython, Rtree
  The script f2py.exe is installed in 'C:\Miniconda\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  The scripts futurize.exe and pasteurize.exe are installed in 'C:\Miniconda\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  The scripts cygdb.exe, cython.exe and cythonize.exe are installed in 'C:\Miniconda\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed Cython-0.29.15 GDAL-3.0.4 Rtree-0.9.3 future-0.18.2 numpy-1.18.1 scipy-1.4.1

Resulting import errors:

___________________ ERROR collecting tests/test_testing.py ____________________
ImportError while importing test module 'D:\a\myrpoj\myrpoj\tests\test_testing.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests\test_testing.py:13: in <module>
    import numpy
E   ModuleNotFoundError: No module named 'numpy'

The workflow file is set up as:

- name: setup-miniconda
      uses: goanpeca/setup-miniconda@v1
      with:
        auto-update-conda: true
        activate-environment: anaconda-client-env
        condarc-file: etc/pythonpackage-condarc.yml
        python-version: ${{ matrix.python-version }}
        auto-activate-base: false
    - name: Install dependencies
      shell: bash -l {0}
      run: conda upgrade -y pip setuptools

    - name: Install MyProject (Windows)
      if: matrix.os == 'windows-latest'
      env:
          PIP_EXTRA_INDEX_URL: "http://pypi.locationtomore.org/simple/"
          PIP_TRUSTED_HOST: "pypi.locationtomore.org"
          PIP_PREFER_BINARY: 1
      shell: bash -l {0}
      run: |
          $CONDA/python -m pip install shapely --index-url http://pypi.locationtomore.org/simple/ -c requirements.txt
          $CONDA/python -m pip install -r requirements.txt
          conda install $PACKAGES
          $CONDA/python setup.py install

    - name: Test with pytest
      shell: bash -l {0}
      run: pytest -s

I see that during setup-miniconda the path is set to:

# Locating Miniconda...
#######################

C:\Miniconda

# Setup environment variables...
################################

Add "C:\Miniconda\condabin" to PATH

Is there a way to allow pip to install packages to where they can be seen in the environment?

Micromamba + cache full example

Hi, I'm trying to get the absolute fastest times, especially on windows. I see that over at mamba, micromamba is soon going to support windows soon (link) and support for installing form files at some point (mamba-org/mamba#461). It would be really cool to be able to use this action + cache + micromamba for what I assume is going to be blazing fast installs. Once they get windows working (or maybe before) it would be great to get support and a full example of that combination here.

I would imagine this would look something like this:

     - name: Cache conda/mamba
        uses: actions/cache@v2
        env:
          CACHE_NUMBER: 0
        with:
          path: ~/conda_pkgs_dir
          key: conda-${{ runner.os }}-${{ env.CACHE_NUMBER }}-${{ hashFiles('test-env.yml') }}

      - name: Setup micromamba
        uses: goanpeca/setup-miniconda@v1
        with:
          channels: conda-forge
          activate-environment: test
          micromamba-version: "*"
          use-only-tar-bz2: true  # requried for caching
          environment-file: test-env.yml

      - name: Show conda environment
        shell: bash -l {0}
        run: |
          micromamba list

Hopefully this would the first run download and install micromamba and then use micromamba to install the test environment using test-env.yml.

Subsequent runs would then use micromamba from the cache (i.e. no need to re-download and re-install micromamba since it's a single file) and just activate/update the test env.

too long setup phase & caching

Quite frequently we have failing test just after init of Conda environment. From the log, it seems to be cancelled but there is no obvious reason why it happened and I do not think that anyone did it manually...
This is a link to one of this pseudo cancelled jobs
https://github.com/PyTorchLightning/pytorch-lightning/pull/2486/checks?check_run_id=847392959
and this is the cut log

2020-07-07T21:19:14.8870348Z ##[group]Run goanpeca/[email protected]
2020-07-07T21:19:14.8870519Z with:
2020-07-07T21:19:14.8870641Z   auto-activate-base: false
2020-07-07T21:19:14.8870762Z   miniconda-version: 4.7.12
2020-07-07T21:19:14.8870877Z   python-version: 3.7
2020-07-07T21:19:14.8870975Z   environment-file: environment.yml
2020-07-07T21:19:14.8871096Z   activate-environment: pl-env
2020-07-07T21:19:14.8871213Z   use-only-tar-bz2: true
2020-07-07T21:19:14.8871332Z   auto-update-conda: false
2020-07-07T21:19:14.8871449Z   remove-profiles: true
2020-07-07T21:19:14.8871561Z ##[endgroup]
2020-07-07T21:19:14.9800149Z 
2020-07-07T21:19:14.9800380Z # 
2020-07-07T21:19:14.9800612Z # Downloading Miniconda...
2020-07-07T21:19:14.9800680Z 
2020-07-07T21:19:14.9805777Z ##############################
2020-07-07T21:19:14.9805898Z 
2020-07-07T21:19:14.9809615Z Miniconda3-4.7.12-Linux-x86_64.sh
2020-07-07T21:19:15.9515494Z Saving cache...
2020-07-07T21:19:16.6935049Z 
2020-07-07T21:19:16.6935850Z # Installing Miniconda...
2020-07-07T21:19:16.6936568Z #########################
2020-07-07T21:19:16.6937014Z 
2020-07-07T21:19:16.6999519Z [command]/usr/bin/bash /home/runner/work/_temp/64e1902d-ffc0-4207-b662-496b67f44c2e.sh -b -p /usr/share/miniconda3
2020-07-07T21:19:17.3422900Z PREFIX=/usr/share/miniconda3
2020-07-07T21:19:18.1341837Z Unpacking payload ...
2020-07-07T21:19:19.3115710Z 
2020-07-07T21:38:25.4494202Z   0%|          | 0/35 [00:00<?, ?it/s]
2020-07-07T21:38:25.4781118Z ##[error]The operation was canceled.

Any idea what could be happening there?

CondaValueError: prefix already exists: C:\Miniconda

Getting this weird error, any ideas?

Error:

image

C:\windows\system32\cmd.exe /D /S /C "C:\Miniconda\condabin\conda.bat env create -f env.yml --quiet"
##[warning]
CondaValueError: prefix already exists: C:\Miniconda



CondaValueError: prefix already exists: C:\Miniconda

##[error]The process 'C:\Miniconda\condabin\conda.bat' failed with exit code 1

workflow file:

name: Python package

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:
    env:
      CONDA_ENV_NAME: my-test-env
    runs-on: [windows-latest]
    strategy:
      matrix:
        python-version: [3.6, 3.7, 3.8]

    steps:
    - uses: actions/checkout@v2
    - uses: goanpeca/setup-miniconda@v1
      with:
       auto-update-conda: true
       activate-environment: ${{ env.CONDA_ENV_NAME }}
       environment-file: env.yml
       python-version: ${{ matrix.python-version }}
       auto-activate-base: false
    - name: Show Conda env stuff
      shell: powershell
      run: |
        conda info -a
        conda list
        conda list -n $ENV:CONDA_ENV_NAME

Conda env.yml file:

channels:
  - conda-forge
  - defaults
dependencies:
  # dev tools
  - black=19.10b0
  - flake8=3.7.9
  - hypothesis=4.54.2
  - ipython=7.11.1
  - matplotlib=3.1
  - mypy=0.761
  - pre_commit=2.*
  - pytest=5.3.4
  - pytest-cov=2.8.1
  - spyder-kernels=0.*

  # core
  - pip=20.*
  - pandas=1.*
  - pyodbc=4.0.28
  - sqlalchemy=1.3.13
  
  # pip dev tools
  - pip:
    - codetiming==1.1

2.0 release plans

We have merged a number of changes since 1.7. Let's figure out what we need to do to get a new release out!

Looking over the docs, here's my stab at it:

  • bump versions (#83)
  • docs
    • better example links (pick one)
    • add changelog (#85)
  • make new tag
    • make/move ๐Ÿ˜ฑ convenience tag
  • update release process with lessons learned
  • blog post?

GHA Deprecation: "add-path" and "set-env"

GitHub Actions is deprecating set-env and add-path due to a security risk: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/

Currently, the `setup-miniconda' action has these lines:

$ grep setup-miniconda/ -nre "set-env"
setup-miniconda//dist/delete/index.js:577: *   ::set-env name=MY_VAR::some value
setup-miniconda//dist/delete/index.js:700:        command_1.issueCommand('set-env', { name }, convertedVal);
setup-miniconda//dist/setup/index.js:17524: *   ::set-env name=MY_VAR::some value
setup-miniconda//dist/setup/index.js:20008:        command_1.issueCommand('set-env', { name }, convertedVal);
$ grep setup-miniconda/ -nre "add-path"
setup-miniconda//dist/delete/index.js:722:        command_1.issueCommand('add-path', {}, inputPath);
setup-miniconda//dist/setup/index.js:20030:        command_1.issueCommand('add-path', {}, inputPath);

Which cause this warning when using it in a GHA workflow:

The `add-path` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
The `set-env` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/

Cannot uninstall/update package (via pip) on Windows

The full checks result is here: https://github.com/OpenSourceEconomics/econsa/pull/77/checks?check_run_id=1336112538

System setup

Environment:

  - pip:
    - isort
    - pytest-stress
    - pytest-randomly
    - black>=20.8b0
    - black-nb
    - flake8-nb
    - numpydoc
    - cairosvg
    - sphinxcontrib-svg2pdfconverter[CairoSVG]

Workflow:

jobs:
  run-tests:
    name: Run tests for ${{ matrix.os }} on ${{ matrix.python-version }}
    runs-on: ${{ matrix.os }}

    strategy:
      fail-fast: false
      matrix:
        os: ['ubuntu-latest', 'macos-latest', 'windows-latest']
        python-version: ['3.7']

    steps:
    - uses: actions/checkout@v2

    - uses: goanpeca/setup-miniconda@v1
      with:
          auto-update-conda: false
          activate-environment: econsa
          environment-file: environment.yml
          python-version: ${{ matrix.python-version }}

Error message (on Windows)

Installing collected packages: isort, pytest-stress, pytest-randomly, black, black-nb, flake8-nb, numpydoc, cairocffi, tinycss2, cssselect2, cairosvg, sphinxcontrib-svg2pdfconverter

  Attempting uninstall: black

    Found existing installation: black 19.10b0

    Uninstalling black-19.10b0:

      Successfully uninstalled black-19.10b0

Warning: Pip subprocess error:
ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\pip-uninstall-vxstmnu2\\black.exe'

Consider using the `--user` option or check the permissions.


CondaEnvException: Pip failed


Error: The process 'C:\Miniconda\condabin\conda.bat' failed with exit code 1

Expected result (on Linux)

 Installing collected packages: isort, pytest-stress, pytest-randomly, black, black-nb, flake8-nb, numpydoc, tinycss2, cssselect2, cairocffi, cairosvg, sphinxcontrib-svg2pdfconverter
  Attempting uninstall: black
    Found existing installation: black 19.10b0
    Uninstalling black-19.10b0:
      Successfully uninstalled black-19.10b0
Successfully installed black-20.8b1 black-nb-0.3.0 cairocffi-1.2.0 cairosvg-2.5.0 cssselect2-0.4.1 flake8-nb-0.2.5 isort-5.6.4 numpydoc-1.1.0 pytest-randomly-3.4.1 pytest-stress-1.0.1 sphinxcontrib-svg2pdfconverter-1.1.0 tinycss2-1.1.0

Use miniconda with other actions

I'm trying to setup my first GitHub Actions workflow, but I am having troubles getting it to work in combination with other actions. I couldn't find an example of this, and I am not even sure if it's supposed to work. In my case I am trying to combine the code coverage action with the miniconda environment I made:

- name: Setup Miniconda
  uses: goanpeca/[email protected]
  with:
   activate-environment: peaksql
   environment-file: environment.yml
   python-version: ${{ matrix.python-version }}

- name: Test & publish code coverage
  env:
    CC_TEST_REPORTER_ID: ${{secrets.CodeCoverage}}
  uses: paambaati/[email protected]
  with:
    coverageCommand: coverage run -m unittest discover test
    debug: true

Adding shell: bash -l {0} gives the error:

shell cannot be used with uses, with

Is it possible to combine setup-miniconda with other actions? If so, how?

Caching troubles & questions

I'm trying to take advantage of caching of dependencies. It seems like whatever I do thesetup-miniconda action takes just as long regardless of whether I've cached my dependencies or not.

    - name: Cache conda
      uses: actions/cache@v1
      env:
        # Increase this value to reset cache if environment.yml has not changed
        CACHE_NUMBER: 5
      with:
        path: ~/conda_pkgs_dir
        key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{ hashFiles('environment.yml') }}
    - uses: goanpeca/setup-miniconda@v1
      with:
        activate-environment: myenvname
        channel-priority: strict
        environment-file: environment.yml
        use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!

Have I made a mistake here that is preventing setup-miniconda from loading the packages from ~/conda_pkgs_dir as specified in the docs?

Important note Possibly worth documenting, removing python-version: 3.7 from the step resulted in a faster build, but still no advantage when caching. I was seeing much slower build times as the step would hang on installing python 3.7.

My environment file looks a bit like this:

name: myenvname
channels:
  - pytorch
  - anaconda
  - conda-forge
  - defaults
dependencies:
  - python=3.7
  - jupyter=1.0.0
  - numpy=1.17.4
  - pandas=0.25.3
  - pytorch=1.4.0
  - pip=20.0.2
  - pip:
    - coverage==5.0.3

It seems like some dependencies are downloading regardless of the fact that they were stored in the cache (the caching step is successful and it shows a reasonable size).

# Updating conda environment from yaml file...
##############################################

/usr/share/miniconda/condabin/conda env update --file environment.yml --name myenvname
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Ran pip subprocess with arguments:
['/usr/share/miniconda/envs/my-org/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/runner/work/my-org/my-repo/condaenv.p1divyve.requirements.txt']
Pip subprocess output:
Collecting coverage==5.0.3
  Downloading coverage-5.0.3-cp37-cp37m-manylinux1_x86_64.whl (227 kB)

It seems like it's not picking up the files that were restored from the cache, what am I missing?
Also should the updated cache@v2 be supported?

Getting deprecation warnings for "add-path" and "set-env"

Thanks for the nice project! Seems to work rather nicely - except that apart from warnings regarding duplicate channels that have already been reported (#57), I'm also getting rather worrying warnings from GitHub Actions regarding the use of add-path and set-env:

# Locating Miniconda...
#######################

/usr/share/miniconda

# Setup environment variables...
################################

Add "/usr/share/miniconda/condabin" to PATH
Warning: The `add-path` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
/usr/share/miniconda/condabin/conda config --add pkgs_dirs /home/runner/conda_pkgs_dir
Warning: The `set-env` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/

My workflow config:

on: [push, pull_request]

jobs:
  build:

    runs-on: ubuntu-latest
    defaults:
      run:
        shell: bash -l {0}
    strategy:
      matrix:
        python-version: [3.7, 3.8]

    steps:
    - uses: actions/checkout@v2
    - name: Setup Conda environment with Python ${{ matrix.python-version }}
      uses: conda-incubator/[email protected]
      with:
        activate-environment: install-deps
        auto-activate-base: false
        auto-update-conda: true
        environment-file: environment.yml
        python-version: ${{ matrix.python-version }}
  # ... 

Apparently this is related to a mild security vulnerability and the deprecation warnings have only been added a couple of days ago. See this blog post for more info, including how to mitigate the issue: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/

changing the cwd

I'm trying to use setup-miniconda with my python package setup. The default github workflow for python package ci was working:

name: traitar3

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.6, 3.7, 3.8]

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v1
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install pytest
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    - name: Test with pytest
      run: |
        pytest -s --script-launch-mode=subprocess

...and here's my yaml when incorporating setup-miniconda:

name: traitar3

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:
    name: build (${{ matrix.python-version }}, ${{ matrix.os }})
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.6, 3.7, 3.8]
    steps:
    - uses: goanpeca/setup-miniconda@v1
      with:
        miniconda-version: 'latest'
        auto-update-conda: true
        python-version: ${{ matrix.python-version }}
        channels: conda-forge,bioconda
        activate-environment: traitar3
    - name: conda env create
      shell: bash -l {0}
      run: |
        conda info -a
        conda install python=${{ matrix.python-version }} bioconda::hmmer scipy pandas numpy matplotlib
    - name: package install
      run: |
        python -m pip install --upgrade pip
        pip install pytest pytest-dependency pytest-console-scripts
        python setup.py install
    - name: Test with pytest
      run: |
        pytest -s --script-launch-mode=subprocess

The setup-miniconda version always throws the error: python: can't open file 'setup.py': [Errno 2] No such file or directory. I'm guessing that either uses: goanpeca/setup-miniconda@v1 or shell: bash -l {0} is changing the working directory, so then setup.py cannot be found. I've been trying to debug this for a while with a number of approaches and haven't found a solution. Any help would be appreciated.

It would be helpful to include python package setup in one of the setup-miniconda examples.

Advice for how to cache the conda installation?

We use this action for Manubot, see workflow and example build. It's great, thanks for making it! Our current configuration is:

      - name: Install Environment
        uses: goanpeca/setup-miniconda@v1
        with:
          activate-environment: manubot
          environment-file: build/environment.yml
          auto-activate-base: false
          miniconda-version: 'latest'

In manubot/rootstock#316, I'm looking into using actions/cache to cache a directory of data created during action runtime.

However, the slowest part of our build is often installing the conda environment, which currently takes about 100 seconds of the 200 second total time. So I was wondering if the conda environment would be cacheable.

For example what about the following:

      - name: Cache Conda
        id: cache-conda
        uses: actions/cache@v1
        with:
          path: WHAT_GOES_HERE
          key: conda-cache-${{ hashFiles('build/environment.yml') }}
      - name: Some conditional set here?
        if: steps.cache-conda.outputs.cache-hit != 'true'
        uses: goanpeca/setup-miniconda@v1

Basically our goal is use a pre-existing conda installation unless our environment.yml specification is changed. @goanpeca does that sound like a good idea? Any advice on what directory we should cache and how to configure this action to use a cached directory?

Setup-miniforge

Hi @conda-incubator/setup-miniconda-team,

I created https://github.com/conda-incubator/setup-miniforge as a convenience for dealing directly with conda forge on defaults.

My plan was to have a script update the parts that needed updating so the it would follow this repo, and just have the script run every now and then to keep the things updated.

Maybe there is a better way of doing this.

Thoughts?

Activated environment with windows and bash

I'm doing some experiments with setup-miniconda, but I have a problem with the following code:

steps:
      - name: Checkout Repo
        uses: actions/checkout@master
      - uses: goanpeca/setup-miniconda@master
        with:
          auto-update-conda: false
          python-version: ${{ matrix.python-version }}  
          activate-environment: test-environment
          environment-file: environment.yml
          condarc-file: condarc.yml
          auto-activate-base: false
      - name: Flake8
        shell: bash -l {0}
        run: |
           conda env list
           flake8 src tests

environment.yml and condarc.yml are ok, all dependencies are installed and the environment test-environment is created correctly, but in the next step (Flake8), the activated environment is base and not test-environment if the os is windows and the shell is bash or sh. For example with linux/bash or windows/pwsh all works as expected.
If can be useful, this is .bash_profile on windows:

Run cat $HOME/.bash_profile

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
eval "$('/C/Miniconda/Scripts/conda.exe' 'shell.bash' 'hook')"
# <<< conda initialize <<<

As a quick hack, if I add the following step:

   - name: Activate `test-environment` (Windows)
        if: runner.os == 'Windows'
        run: echo "activate test-environment" >> ~/.bash_profile

the Flake8 step works.
Is that a normal behaviour? Maybe a bug?

Thanks.

CondaValueError in Microconda setup

Hi, thx for this nice tool, just need some help to let it run...
My use case is to restore packages according to given environment.yaml file, so I followed your example, but I am getting its strange error

/usr/share/miniconda3/condabin/conda env create -f environment.yml
##[warning]
CondaValueError: prefix already exists: /usr/share/miniconda3

CondaValueError: prefix already exists: /usr/share/miniconda3

##[error]The process '/usr/share/miniconda3/condabin/conda' failed with exit code 1

the full run you can find here https://github.com/PyTorchLightning/pytorch-lightning/pull/2412/checks?check_run_id=821315545
also with the cat of actual config file...

Caching and mamba

mamba is a very nice feature!

When I use it + caching it looks like the action reinstalls mamba from scratch and so it's slow (~2min) while when I don't use it the action is very fast (~15s).

Any idea what could the cause of this?

Add mamba support

I have found that using mamba for dependency solving complex environments MUCH faster. You previously mentioned you may be convinced to add support. I am making a separate issue to track this.

Micromamba based setup?

Thoughts on this? https://github.com/mamba-org/mamba#micromamba

micromamba is a tiny version of the mamba package manager. It is a pure C++ package with a separate command line interface. It can be used to bootstrap environments (as an alternative to miniconda), but it's currently experimental. The benefit is that it's very tiny and does not come with a default version of Python.

It might be too soon because it's still experimental, but it could be helpful in the future.

Use password and username instead of a token

I use anaconda login instead of a token to access private packages. Could that be added as an option? Currently, my setup looks like the following:

      - name: Setup conda
        uses: conda-incubator/setup-miniconda@v2
        with:
          python-version: 3.7
          mamba-version: "*"
          channels: conda-forge,defaults
          channel-priority: true
          use-only-tar-bz2: true
          activate-environment: my-env

      - name: Install Dependencies
        shell: bash -l {0}
        run: |
          mamba install -y anaconda-client
          yes 2>/dev/null | anaconda login --username $ANACONDA_USER  --password $ANACONDA_PASSWORD || true

          mamba env update -f env.yml

Would be nice and probably faster to do everything within the conda-incubator/setup-miniconda action:

      - name: Setup conda
        uses: conda-incubator/setup-miniconda@v2
        with:
          mamba-version: "*"
          channels: conda-forge,defaults
          channel-priority: true
          use-only-tar-bz2: true
          activate-environment: my-env
          anaconda_username: ${{ env.ANACONDA_USER }}
          anaconda_password: ${{ env.ANACONDA_PASSWORD }}
          environment-file: env.yml

Setup conda step takes too long

Thank you for creating/maintaining this great Action!

It seems that setting up conda alone takes longer than usual. For example, take a look at this log, the "Set up conda" step. It takes 3 and a half minutes setting up conda (excluding the installation of dependencies). Is that expected or am I doing something wrong?

Thank you once again!

Example described in Setup miniconda doesn't work

Thanks for creating such an amazing & useful action.

  example-2-linux:
    name: Ex2 Linux
    runs-on: "ubuntu-latest"
    steps:
      - uses: goanpeca/setup-miniconda@v1
        with:
          miniconda-version: "latest"
          activate-environment: foo
      - name: Sh
        shell: sh -l {0}
        run: |
          conda info
          conda list
      - name: Bash
        shell: bash -l {0}
        run: |
          conda info
          conda list
      - name: PowerShell Core
        shell: pwsh
        run: |
          conda info
          conda list

The following task mentioned here is no longer working. Sample run: https://github.com/karrtikr/vscode-python/runs/776967236?check_suite_focus=true

Also, I wonder why the windows installation is so fast?? Installing it on my local windows machine takes forever,

Channels added twice

I'm seeing channels being added twice with this action, once under:

# Applying conda configuration...
#################################

and another under:

# Applying conda configuration after update...
##############################################

This results in warnings:

C:\windows\system32\cmd.exe /D /S /C "C:\Miniconda3\condabin\conda.bat config --add channels conda-forge"
##[warning]Warning: 'conda-forge' already in 'channels' list, moving to the top

Warning: 'conda-forge' already in 'channels' list, moving to the top
C:\windows\system32\cmd.exe /D /S /C "C:\Miniconda3\condabin\conda.bat config --add channels conda-forge/label/testing"
##[warning]Warning: 'conda-forge/label/testing' already in 'channels' list, moving to the top

Warning: 'conda-forge/label/testing' already in 'channels' list, moving to the top

The problem is that these warnings appear in the top-level output of my workflow, which is kinda of annoying since they will never go away. Here's my config if it helps:

    - name: Set up Python ${{ matrix.python-version }}
      uses: goanpeca/setup-miniconda@v1
      with:
        auto-update-conda: true
        miniconda-version: "latest"
        python-version: ${{ matrix.python-version }}
        channel-priority: strict
        channels: conda-forge/label/testing,conda-forge
        show-channel-urls: true
        use-only-tar-bz2: true

Python/Pip not setup correctly when using macos-latest

When using macos-latest, the python and pip are not linked properly. For instance my workflow has specified:

- uses: conda-incubator/[email protected]
        with:
          miniconda-version: "latest"
          activate-environment: cgal
          channels: conda-forge
          python-version: 3.7

- name: Install
        run: |
          pip install -r requirements-dev.txt 
          pip install -e .

However I'm getting an error:

Run pip install -r requirements-dev.txt 
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Obtaining file:///Users/runner/work/compas_cgal/compas_cgal (from -r requirements-dev.txt (line 18))
    ERROR: Command errored out with exit status 1:
     command: /usr/local/opt/[email protected]/bin/python2.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/runner/work/compas_cgal/compas_cgal/setup.py'"'"'; __file__='"'"'/Users/runner/work/compas_cgal/compas_cgal/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
         cwd: /Users/runner/work/compas_cgal/compas_cgal/
    Complete output (9 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/Users/runner/work/compas_cgal/compas_cgal/setup.py", line 65, in <module>
        get_eigen_include(),
      File "/Users/runner/work/compas_cgal/compas_cgal/setup.py", line 43, in get_eigen_include
        return os.path.join(conda_prefix, 'include', 'eigen3')
      File "/usr/local/Cellar/[email protected]/2.7.17_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 70, in join
        elif path == '' or path.endswith('/'):
    AttributeError: 'NoneType' object has no attribute 'endswith'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Error: Process completed with exit code 1.

Seems it is still using the system python, instead of the one under conda

Invitation to co-contribute and co-maintain

Hello @bollwyvl, @bryanwweber, @jaimergp, @MGough, @marcelm, @jjhelmus, @HiromuHota and others I may have left out ๐Ÿ˜ถ

As this action has gained more traction (it rhymes... you see what I did here? ๐Ÿ™ƒ ... ๐Ÿคฆ) and moved to incubation I would like to invite some of you have been very active in the short life span of this project.

I would like to invite you to help co-contribute and co-maintain the project.

If you would like to accept, this would involve:

  • Getting commit rights ๐Ÿฅณ
  • Making releases ๐Ÿ“ฆ
  • Helping other users ๐Ÿ†˜
  • Review & merge PRs ๐Ÿค“
  • Improve the action ๐Ÿ“ˆ
  • Maybe meet from time to time (like once per semester ๐Ÿคท ) to discuss whatever needs live discussion โ˜Ž๏ธ

Let me know what you think, also I could do a small onboarding meeting to go over some details so we can document it together.

Thanks a lot!


This is also an open invitation to users, in case you would like to get involved in an open source project. This is a small one and can help in providing a gentle introduction for newcomers to the OS world!

Add caching

Explore something like

- uses: actions/cache@v1
  with:
    path: ~/conda_pkgs
    key: ${{ runner.os }}-conda-${{ hashFiles('**/environment.yml') }}
    restore-keys: |
      ${{ runner.os }}-conda-
- run: conda config --add ~/conda_pkgs

conda build support?

Hi,

Sorry this may be very stupid, but is conda build supported by this? Thanks a lot!

Bob

How to use it with other github actions

I tried to use this with an other github action that builds and checks sphinx documentation.

https://github.com/marketplace/actions/sphinx-build

Of course it didn't pick up the activated environment, so the build failed. I can get it working if I run the shell directly with shell: bash -l {0}.

But the sphinx-build action has some nice features which I like to use.

It does provide a way to run a pre build command. Can that be used to activate the conda-environment?

      - uses: ammaraskar/sphinx-action@master
        with:
          pre-build-command: ..XXXXXX...

I once did something similar in the gitlab ci system where I ran . /opt/conda/etc/profile.d/conda.sh && conda activate sphinx" to get the environment activated. Could I do something similar here?

mamba on windows "mamba: command not found"

on a currently private repo I am trying to use mamba to build on mac/linux/windows but I am running into a command not found error on the windows worker, seems like mamba is not available even after successful install as show in the logs.

partial config:

test_runner:
    name: (${{ matrix.python-version }}, ${{ matrix.os }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: ["ubuntu-latest", "macos-latest", "windows-latest"]
        python-version: ["3.8"]
    steps:
      - uses: actions/checkout@v2
      - name: setup miniconda
        uses: goanpeca/setup-miniconda@v1
        with:
          mamba-version: "*"
          python-version: ${{ matrix.python-version }}
          channels: conda-forge,defaults
          channel-priority: true
          activate-environment: pangeo_test
          auto-update-conda: true
          use-only-tar-bz2: true
      - name: conda info
        shell: bash -l {0}
        run: |
          conda info
      - name: mamba install packages
        shell: bash -l {0}
        run: mamba install ... (command not found error here)

"Warning - a newer version of conda exists" - logged as a warning on Windows

Below a snippet of the log, installing conda on windows-latest with this config:

with:
  auto-update-environment: false
  activate-environment: testenv

If you notice the ##[warning] below, I think that's what's causing the warning to get logged and reported as an annotation in actions ui? Interestingly, it does only happen on windows runners.

Is there any way to suppress that?

:: --------------------------------------------------------------------------- 

# Create test environment...
############################

C:\windows\system32\cmd.exe /D /S /C "C:\Miniconda\condabin\conda.bat create --name testenv"
Collecting package metadata: ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: C:\Miniconda\envs\testenv

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
##[warning]

==> WARNING: A newer version of conda exists. <==
  current version: 4.6.14
  latest version: 4.8.3

Please update conda by running

    $ conda update -n base -c defaults conda

==> WARNING: A newer version of conda exists. <==
  current version: 4.6.14

How to use with cache mechanism?

Thanks for this, I'm finding it very useful.

Is it possible to cache the miniconda3 setup using actions/cache? I think all I need is a path to the miniconda3 directory, but I'm not sure where that would be in the container.

Problems when creating a new env

Hi, @goanpeca!

First of all, congratulations for this action! And thanks for providing it!

Well, unfortunately, I have been facing a problem in Windows builds with your GH actions. I use conda-devenv to build conda environments in a development mode. It provides extended capabilities when compared with regular conda, I can define environment variables, use jinja2 and etc.

The problem is: when I do my tests, I create a new environment using conda devenv. For reason unknown, when I use your Action, the created environment receive an empty name.

Below I provide my windows build file:

name: windows

on:
  push:
    branches:
      - master

  pull_request:

  schedule:
    - cron: "0 5 * * 1"  # runs at 05:00 UTC on Mondays

jobs:
  build:

    runs-on: windows-latest
    env:
      configuration: Release
    strategy:
      fail-fast: true
      max-parallel: 4

    steps:
    - uses: actions/checkout@v1
    - uses: goanpeca/[email protected]
      with:
        auto-update-conda: true
        activate-environment: autodiff
    - name: Install conda-devenv
      shell: powershell
      run: |
        conda install -c conda-forge conda-devenv
        conda info -a
    - name: Build
      shell: powershell
      run: |
        conda devenv
        conda info --envs
        conda activate autodiff
        cmake -S . -B build
        cmake --build build --config ${{ env.configuration }} --target install
    - name: Tests
      shell: powershell
      run: call build\test\${{ env.configuration }}\tests.exe

I even tried to put the same name of my new environment as activate-environment, since conda-devenv can re-generate an env on an existing env without problem. However, when I do it, the new env is created without a name. When I run conda info --envs in the CI (windows only), I get the following output:

2020-01-29T22:35:24.9862529Z # conda environments:
2020-01-29T22:35:24.9863192Z #
2020-01-29T22:35:24.9863428Z base                     C:\Miniconda
2020-01-29T22:35:24.9863652Z autodiff              *  C:\Miniconda\envs\autodiff
2020-01-29T22:35:24.9863862Z                          C:\Miniconda\envs\autodiff\envs\autodiff

Do you know how I could solve such problem?

Cheers!

Channels option (README ex. 4) not working (not found)

Hi, I am trying to use your Action for a project I co-develop, 'cf-python'. However, it is not working as documented when I try to access the variant of the Action with channels available (as I need to use channels), namely 'Example 4: Conda options' from the README. In particular, my job fails after repeatedly trying & failing to download the Action goanpeca/setup-miniconda@enh/channels itself:

Failed to download action 'https://api.github.com/repos/goanpeca/setup-miniconda/tarball/enh/channels'. Error Response status code does not indicate success: 404 (Not Found).

(you can view the job in question here if that helps.)

When I try this in the URL, the component before the '@' symbol is picked up, but looking at this repo there does not seem to be a enh/channels branch. I see there's a enh/rever branch but looking at it it does not seem to be the same thing under a different name as far as I can tell.

Should the channels option Action variant work as far as you know? If not, it would be helpful if you removed it to prevent users like myself trying it out in vain, & could you please advise as to whether there is a way to access channels somehow using this Action, as without these it will be more difficult for me to setup a desired custom Action on my project.

Thanks.

The action stopped working a few minutes ago

Hello @goanpeca ,

First of all, thanks for your GitHub Action - it was really helpful when I switched my CI to GitHub Actions a few weeks ago (cf. my related article in TDS).

Very recently I've seen my miniconda jobs failing with error messages like this one:

An action could not be found at the URI 'https://api.github.com/repos/goanpeca/setup-miniconda/tarball/v1'

See for instance this build, which was the first one to show the issue.

Do you know how to fix this? I've restarted the build, and then tried to take v1.4, but it did not work.

No such file or directory, scandir '/home/runner/conda_pkgs_dir'

Thank you for an awesome GitHub workflow / conda integration, has been using it for a little while and it is super helpful.

Starting today (15/05/2019) I noticed a bunch of CIs failed in Run goanpeca/setup-miniconda@v1 with:

Append to "/home/runner/.xonshrc":
 
# ----------------------------------------------------------------------------
# Conda Setup Action: Basic configuration
set -eo pipefail
# Conda Setup Action: Custom activation
conda activate my_env
# ---------------------------------------------------------------------------- 

Append to "/home/runner/.config/powershell/profile.ps1":
 
# ----------------------------------------------------------------------------
# Conda Setup Action: Custom activation
conda activate my_env
# ---------------------------------------------------------------------------- 


# Removing uncompressed packages to trim down cache folder...
##[error]ENOENT: no such file or directory, scandir '/home/runner/conda_pkgs_dir'

An example workflow where I can repro:

  build:
    runs-on: ubuntu-latest

...

    - uses: goanpeca/setup-miniconda@v1
      with:
        activate-environment: my_env

It appears to be related to the new release 1.3.0 which was pushed just yesterday; when I locked version to goanpeca/[email protected] the builds resumed successfully.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.