A Documented/Preferred way of installing PyPi dependencies about conda.jl HOT 13 CLOSED

Let's not use non-public interface. (That's one of the source of the problem in the recent Pip/Pipenv incompatibility.) The right way to do it is python -m pip install PACKAGES...

A quick implementation of Pip.add would be something like:

module Pip
using PyCall

is_non_system(python) = iswritable(python)
# For a real implementation, it at least has to check:
# * Is it generated by venv? (Does it have `pyvenv.cfg`?)
# * Is it generated by virtualenv? (Does `sys` module has `real_prefix`?)
# * Is it installed by conda? (Look at version string?)

function add(packages::Vectro{String})
    options = ``
    if !is_non_system(PyCall.pyprogramname)
        options = `--user $options`
    end
    run(`$(PyCall.pyprogramname) -m pip install $options -- $packages`)
    # Once #578 is merged:
    # run(PyCall.python_cmd(`-m pip install $options $packages`))
end

end

from conda.jl.

I actually think making a Pip.jl package that acts a lot like Conda.jl (or HomeBrew.jl kinda).
would be pretty good at this point.
Because I gather a lot of the improvements that Conda has over Pip have now been added to pip (like binaries in wheels? idk I am not a python person.)

PyPi has way move than Conda,
and for most use cases Conda.jl is used exclusively for installing python stuff.
(Even though it can do more.)

Big downside is that it would be impossible to have PyCall using both with in the same process.
So the whole ecosystem would need to move or support both.

from conda.jl.

As far as I can tell, PyCall.py_import_conda, does not help here, since it only downloads things from Conda channels.

Yes, PyCall.py_import_conda only uses Conda.

Is this the best way? If so, it should be documented.
If not, a better way should be documented.

There is no absolute best way, just do whatever suites your need. I see at least five ways of having a Python dependency for Julia code:

• Use Conda by creating a conda package and using a custom channel;
• Use Conda to install pip and use pip to install your dependency;
• Use PyCall with whichever Python distribution is installed, and then use git-pip.y to install pip. This could be the base of a Pip.jl package actually;
• Include the Python dependency in your repo as a git submodule/vendor direcory;
• Use the system package manager to get the dependencies (python-* packages are available in most Linux distributions), and rely on the existing BinDep providers;

The first four can be equally convenient, depending on the dependency, and your requirements. If you are going for Linux only code, the last one if the one that require the less work.

All these could be documented, but I would rather have them documented in a Python specific package that in Conda.jl. conda is not a Python package manager, it is a generic package manager written in Python (like yum), and used by the major scientific python distributions.

from conda.jl.

IPython 7.3 will have %pip magic command ipython/ipython#11524 which can be used via IPython.jl https://github.com/tkf/IPython.jl; sorry, kind of an ad :)

from conda.jl.

To add to the first point, you can use, conda-forge to build packages across platforms using CI services.

from conda.jl.

Is there a downside to using Conda's pip for things that don't have Conda packages? That has the advantage of not depending on (or affecting) a system Python. Could using pip with Conda generate conflicts with ordinary Conda packages?

from conda.jl.

Yes, there could be conflicts. If the pip package has a dependency that is given in Conda and the pip package asks for a specific version of the dependency then pip might install a newer version of the package in Conda.

from conda.jl.

I just found this old gist of mine, using only PyCall to install pip as needed and then use it to download the dependencies.

This could be used by PyCall to provide an install function, switching on the backend: if Conda.jl is used, check if the package is available trough conda and install it. In all the other cases, check if it is available with pip and install it.

from conda.jl.

@Luthaf .
•Use Conda to install pip and use pip to install your dependency;

how can i use this feature while installing conda custom package which has dependecy which is not available in conda but in pypi.

from conda.jl.

If you are working with a custom conda package (I assume you are the one publishing it, perhaps in a custom channel), I believe the best option would be to create and upload conda packages for your dependencies too. There are tools around to create a conda package from a setup.py file.

from conda.jl.

Is there a downside to using Conda's pip for things that don't have Conda packages?

As above, how exactly does one do this? @stevengj, I saw that you closed #101, but as far as I can tell this hasn't been addressed since.

from conda.jl.

(To be clear, the gist above looks like it's been deprecated since 2016, and it looks like the pip from Conda.jl doesn't export a pip.main).

from conda.jl.

I think that's a good idea. Alternately you could just wrap a short script to do what we did after installing Conda.jl and Conda.add("pip")

from pip._internal import main 
main(['install', '--pre', 'dfogn'])

from conda.jl.