Giter Club home page Giter Club logo

venvlink's Introduction

GitHub tag (latest by date) PyPI Downloads GitHub

🔗 venvlink

The job of venvlink is create a link for the ./venv/Scripts/activate script inside the project folder and, if needed, create a virtual environment outside your project folder, using python -m venv <project_name>.

Table of Contents

Without venvlink

This is how work with virtual environments looks like with python -m venv venv:

With venvlink

With venvlink the virtual environment is created in a centralized folder, and only a proxy ("link") of the activate script is created to the project folder:

Motivation

The reason for one using venvlink might be one or multiple from below.

🚫📤 don't backup venv/*

If you want to keep your virtual environments from being backed up to e.g. OneDrive/Dropbox or other backup service with your project files, venvlink is one way to do it. You might want to do this to save space on your cloud storage or resources on your computer / network (prevent checking and uploading new/modified files).

🗂️🧹centralized & tidy

If you want to centralize your virtual environments just because you think it is clean or it makes you happy.

✨♻️ reusability

If you want to reuse some of your virtual environments in multiple projects (sharing same imports) or want to use a general (tmp) virtual environment for all of your one-off testing, for example.

🏗️✔️relocating projects

If you want to be able to relocate projects on hard disk (on same machine) by just copy-pasting them, even with virtual environments. Or, if you want to be able rename a part of the project path without breaking the virtual environment.

🔑🎯 centralized, but with an activate script

If you want to have centralized virtual environments, but would want to activate them, without having to remember the name of the virtual environment, just by running

./venv/Scripts/activate

The activate script in your project folder is just a proxy ("link") for the real activate script located elsewhere.

🗑️🤝 remove anytime

The venvlink philosophy is that a tool should not chain it's user. You can remove it anytime and your virtual environments and activate proxy scripts will still work. Venvlink is only used for creation of virtual environments.

💼✔️ works with an existing venv collection

If you happen to have already a collection of virtual environments inside one folder, you can start using them with venvlink right away. Just configure the venv_folder in the .venvlinkrc, and you're good to go.

🐍🐍 multipython

It is not a problem if you have multiple python versions installed. The syntax is

<path_to_python_exe> -m venvlink ...

Therefore, you can use venvlink on Python 3.6.4 32-bit and Python 3.9.2 64-bit at the same time. What you need to do is to install venvlink for both python versions with

<path_to_python_exe> -m pip install venvlink
<path_to_another_python_exe> -m pip install venvlink

and the rest you can use normally. Note that virtual environments in the centralized folder will have the same python version which was used to create the virtual environment in the first place. You can later on use your main python executable to create more links for an existing virtual environment, no matter what version of python it is using. Use descriptive venv names to distinguish the venvs with different python versions, if needed.

✨🔥 auto-activate

The venvlink supports a Powershell addon called venvlink-autoenv which will automatically activate/deactivate the virtual environment on Windows Powershell! (Works also with example VS Code Integrated Terminal, when Powershell is selected )
New in version 0.5.0

Installing

Requirements

venvlink is currently supporting only Windows, but it shouldn't be too hard to create Linux/maxOS support. I personally use only Windows, but I would be really happy to receive pull request(s) for Linux/macOS support.

Installation with pip

pip install venvlink

For installation for development, see CONTRIBUTING.md.

Getting started

✨ Creating a virtual environment

Assume that you have a project at

C:\workdir\someproject\

and you would like to create virtual environment for that folder. Instead of the regular

PS C:\workdir\someproject> python -m venv venv

using venvlink one would type

PS C:\workdir\someproject> python -m venvlink project-name

This would create, inside your project directory, a venv folder with only few files (<1Kb in total), such as the activate script.:

venv
├── Scripts
│   ├── Activate.ps1
│   ├── activate
│   └── activate.bat
└── venvlink

and inside your venv_folder (located elsewhere), the actual virtual environment files (can be up to hundreds of Mb, and thousands of files):

project-name/
├── Include
├── Lib
│   └── site-packages
├── Scripts
│   ├── Activate.ps1
│   ├── activate
│   ├── activate.bat
│   └── deactivate.bat
├── pyvenv.cfg
└── share

To activate the virtual environment (project-name), you would call

PS C:\workdir\someproject> .\venv\Scripts\activate

which would then call transparently*

C:\Python\venvs\project-name\Scripts\activate

and result you having that virtual environment activated, as:

(project-name) PS C:\workdir\someproject> 

[*] assuming that you have defined the venv_folder to be C:\Python\venvs\ in the .venvlinkrc.

📖❔ Getting help

You can use the -h flag:

PS C:\somepath\project> python -m venvlink -h

usage: venvlink [-h] [--init] [-d] [-S] [projectname]

venvlink 0.3.1

positional arguments:
  projectname

optional arguments:
  -h, --help            show this help message and exit
  --init                Initiate the venvlink configuration file (.venvlinkrc)
  -d, --delete          Delete the virtual environment associated with project_name (instead of creating)
  -S, --system-site-packages
                        Give the virtual environment access to the system site-packages dir.

🗑️ Removing virtual environments

If you want to fully remove a virtual environment and all its contents, you can either run

python -m venvlink -d <venv_to_be_removed>

or just navigate to the centralized virtual env folder, and remove the folder(s) you want. There is no centralized bookkeeping between the virtual environments and the activate proxies (just a one-way link from proxy to the venv), and therefore all activate proxies associated with the venv will stop working after deletion, or until a venv with same name is created.

If you want to just remove the link between a virtual environment and your project, you can safely remove the activate proxy/link from your project folder. If you are planning to create a link to another virtual environment, you can simply

python -m venvlink anothervenv

All what this does is recreates the proxy and, if anothervenv does not exist, creates anothervenv.

🛠️ Configuration

See: Configuration.

Considerations

🔍❕ Special cases

Moving your virtual environment to centralized place means that you'll might have to tell to some tools where you have located your virtual environments. One example is pylint, which needs to kown where the project venv is located, to prevent false positive import-errors. See: Usage with linters (e.g. pylint).

💡📁 File structure tip

Following file structure has found to be useful:

C:\Python\Python365\          # python installation
C:\Python\Python386\          # python installation
C:\Python\Python386-32\       # python installation
C:\Python\Python392\          # python installation
C:\Python\venvs\              # virtual environments

since it is easy to find python.exe behind for creating a venv

C:\Python\Python386\python.exe -m venvlink myproj

and it is also easy to browse the venvs when they are after a short path.

🍎🍌 venvlink and other tools

These tools work well together and have synergies with venlink:

  • venvlink-autoenv: The Powershell addon for venvlink >= 0.5.0.
  • pyenv-win: Python version management/switcher
  • pew: Virtual environment management

Other tools and how they relate to venvlink:

Contributing

What? How?
🐞 Found a bug? 🎟 File an Issue
🙋‍♂️ Need help? Ask a question on StackOverflow
💡 Got a suggestion? 🎫 File an Issue (feature request)
🧙 Want to write code? 🔥 Here's how you get started!

venvlink's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

integracore2

venvlink's Issues

"OSError: [WinError 145] The directory is not empty" while removing a virtual environment

From time to time, when removing a virtual environment, a OSError: [WinError 145] is raised.

Example:

PS C:\repos\own-git-repos\some_project> python -m venvlink -d my-virtual-env
Deleting venv for "my-virtual-env"
11-28 19:59 venvlink     INFO     Removing C:\Python\venvs\my-virtual-env
Traceback (most recent call last):
  File "C:\Python\Python 3.8.6-64\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python\Python 3.8.6-64\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\repos\own-git-repos\venvlink\venvlink\__main__.py", line 70, in <module>   
    delete_env(args)
  File "c:\repos\own-git-repos\venvlink\venvlink\__main__.py", line 60, in delete_env 
    vlink.delete_env(args.projectname.strip())
  File "c:\repos\own-git-repos\venvlink\venvlink\__init__.py", line 92, in delete_env 
    shutil.rmtree(folder)
  File "C:\Python\Python 3.8.6-64\lib\shutil.py", line 737, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Python\Python 3.8.6-64\lib\shutil.py", line 610, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Python\Python 3.8.6-64\lib\shutil.py", line 619, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Python\Python 3.8.6-64\lib\shutil.py", line 617, in _rmtree_unsafe
    os.rmdir(path)
OSError: [WinError 145] The directory is not empty: 'C:\\Python\\venvs\\my-virtual-env\\Lib'

This is somehow related to Windows using the files, as running the same command again will succesfully remove the virtual environment.

Suggesting that some retrying is added to the removal function.

Add option for using shared python packages

Some libraries, like pip, setuptools and wheel are almost always needed for a virtual environments. Also, these tools should be kept updated for their latest version. I addition, some users might want to always install some debugging / linting tools to new virtual environments.

Proposal

Add and option to venvlinkrc, like shared_packages, that should be empty or a list, like [pip, setuptools, wheel]. The [pip, setuptools, wheel] could be default, since probably at least these tools are needed for >90% of virtual environments.

This would give users the option to add there other packages they use (I have personal interest on adding IPython there, since I use the IPython.embed() for debugging. Maybe someone else would like to have pdbpp there, or pylint, etc.)

Preliminary working logic

  • Packaging libraries ( pip, setuptools, wheel) are installed to a special virtual environment __venvlink__, which could be located in the venv_folder. This would assume users would not use the special name __venvlink__ as their virtual environment name (maybe give a warning if user tries to use it).
  • Then, these packaging libraries would be added to any new virtual environment by adding a special .pth file to the newly created virtual environment (inside venv_folder). This is how for example pipx works.
  • The packaging libraries should be kept up-to-date. The options are (1) Update to latest every time new virtual environment is created, like pipx does. (2) Update every time virtual environment is created, but only once per day or week, and keep a "latest_shared_libraries_check" timestamp in a special file, for example venvlinkrc. This would make creating virtual environments slightly faster, if many of them are created in a short period of time. Perhaps first option would be the included in the first implementation.
  • In the future, the could be a "no-update" option like disable_shared_autoupdate, where values could be all or a list of packages. If even more fine-grained control is needed, the could be some option for disallowing auto-updates for major or minor releases (and allowing only minor or patch releases). This could be something like a list of tuples.

Other possible implementations

There would be also another way to implement similar outcome: Adding a list for packages in venvlinkrc that should be automatically installed for every new virtual environment (install_packages or something like that) The difference is that when using shared_packages, the shared packages would be only ever needed to install once; this would save some disk space. The outcome would not be entirely similar, so I guess both are needed. For example, if someone would like to always install numpy in every new virtual environment but keep the numpy packages isolated from each other (which is the purpose of virtual environments anyway). Therefore, some packages could be shared across virtual environments, and some could be auto-installed. (creating separate issue for that)

Add some help text for plain "python -m venvlink"

Currently (venvlink v.0.5.0), running

python -m venvlink

Will output

The positional argument "projectname" is required for creating a virtual environment!

It would be handy to have some help text (or instructions on how to see the help text) printed.

Add option for templates (for automatic package installation)

This is related to #4.

Proposal

Sometimes it would be handy to install some packages automatically, but putting all useful packages to autoinstall_packages does not make sense. Something like "virtual environment templates" could be useful. In short, add to venvlinkrc something like:

[templates]
datascience = [numpy, pandas, matplotlib]
datascience2 = [numpy, pandas, plotly]
django = [django~=3.1.5, django-debug-toolbar, djangorestframework]

There could be optional PEP440 compatible version identifiers (like in the django case in the example above) to help keeping versions compatible.

Usage could looks something like this. Creating new virtual environment:

python -m venvlink --template datascience

or

python -m venvlink -t datascience

use always the newest pip in new virtual environments?

Very often python comes with an old pip version which may produce different kinds of problems. After using pip command in a fresh virtual environment, user usually sees a text like this:

WARNING: You are using pip version 20.2.1; however, version 21.3.1 is available.
You should consider upgrading via the 'c:\python\venvs\someproj\scripts\python.exe -m pip install --upgrade pip' command.

it might be actually handy to have the venvlink new venv creation process to automatically do this. This could be also an option in the venvlinkrc.

Add option for automatic installation of packages

This was briefly mentioned in #3.

Proposal

It would be handy to have an option for automatic installation of packages in venvlinkrc, like install_packages or autoinstall_packages. This could be a list, like [numpy, matplotlib].

Versions could be defined as defined in PEP440; numpy~=1.19.5 for numpy versions 1.19.x but greater or equal than 1.19.5.

Example (venvlinkrc):

autoinstall_packages = [numpy~=1.19.5, matplotlib]

Great tool, let's keep it alive and bump an update?

I love this tool, and use it on all my machines. Seeing, however, that it's not receiving much attention, perhaps it would be a good idea to give it a small heart-beat update, just to show the community that it's still alive.

@fohrloop
What do you think?

How disactivate an already activated environment?

Was running .\venv\Scripts\activate to install some packages.
However, I wanted to exit that environment and was execting to find something like disactivate...

How can we get an already activated environment back to the pre-activation one?

Add usage examples to README

Hey Niko,
Great package incentive. Found it from discussions on SO.

However, I'm missing a good case usage example.
For example, I would like to know how to implement the following scenario:

  • Using Windows Powershell (Core) [pwsh 7.2.0] on W10.
  • I have a project called proj-A in C:\Users\XXX\dev\proj-A\ which contains all my source code and is also git connected.
  • I want all my virtual environment package files stay under a common directory like C:\python\venvs\proj-A\.
  • How do I pip install from my developer proj-A directory?
  • How do I copy my proj-A into proj-B?
    (With and without .git?)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.