installation issues with pip and easy_install about dill HOT 15 CLOSED

Interestingly it is easy_installable

(test-pathos)mrocklin@linux2:~$ easy_install dill
Searching for dill
Reading http://pypi.python.org/simple/dill/
Best match: dill 0.2a1
Downloading https://pypi.python.org/packages/source/d/dill/dill-0.2a1.zip#md5=d6a3016f6f0a5b36c059c77c41c3e21f
Processing dill-0.2a1.zip
Writing /tmp/easy_install-katKoO/dill-0.2a1/setup.cfg
Running dill-0.2a1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-katKoO/dill-0.2a1/egg-dist-tmp-RrVUtW
Adding dill 0.2a1 to easy-install.pth file
Installing unpickle.py script to /home/mrocklin/Software/anaconda/envs/test-pathos/bin
Installing get_objgraph.py script to /home/mrocklin/Software/anaconda/envs/test-pathos/bin

Installed /home/mrocklin/Software/anaconda/envs/test-pathos/lib/python2.7/site-packages/dill-0.2a1-py2.7.egg
Processing dependencies for dill
Finished processing dependencies for dill

Sadly this doesn't include all dependencies, in particular this dbm module.

(test-pathos)mrocklin@linux2:~$ python
Python 2.7.6 |Continuum Analytics, Inc.| (default, Nov 11 2013, 10:47:18) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mrocklin/Software/anaconda/envs/test-pathos/lib/python2.7/site-packages/dill-0.2a1-py2.7.egg/dill/__init__.py", line 32, in <module>
    from . import source, temp, detect
  File "/home/mrocklin/Software/anaconda/envs/test-pathos/lib/python2.7/site-packages/dill-0.2a1-py2.7.egg/dill/detect.py", line 26, in <module>
    from . import objects as _objects
  File "/home/mrocklin/Software/anaconda/envs/test-pathos/lib/python2.7/site-packages/dill-0.2a1-py2.7.egg/dill/objects.py", line 20, in <module>
    import dbm
ImportError: No module named dbm
>>> 

Installing from source worked fine for me. Presumably this limits adoption to expert users.

from dill.

To test I create a virtual environment (using conda) with python and pip and nothing else:

conda create -n test-pathos python pip
source activate test-pathos
pip install dill

from dill.

@mrocklin: What you are seeing with ImportError: No module named dbm is that whatever python you are using is missing the dbm module, which ships with most standard python distributions, but oddly enough some of them don't have it -- primarily ubuntu. Ubuntu is also missing sqlite3 and some others. Why I don't know. I did try to have dill understand what modules were optional imports, and skip them. apparently I missed that one. can you verify what form of linux you are using? it also seems like it's the Continuum python... correct?

I have just a very little work to do to make most of these (mostly unnecessary) imports go away... that will also dramatically reduce the import time for dill... and fix any weirdness about missing modules per "standard" python versions.

from dill.

I tested this both with conda's straight python; sqlite and friends would not have been linked. I'm on Ubuntu 12.04.

Reducing unnecessary dependencies sounds good to me. I did notice the longish import time.

from dill.

@mrocklin: So this is an ubuntu problem, and addressed by this commit: 4283aad.
How does this compare with your version? I'm guessing that your easy_install failed, but then you got the code from github and it worked fine... right? If that's the case, then I just need to update the releases to include the patch already in github master. Let me know if that is or is not the case.

And, yes, the long-load was a bit unintentional. It's an easy fix, and I'll get to it soon and rerelease.

from dill.

pip failed due to some version issue
easy_install failed due to dbm but yes, installing from github/source worked fine.

from dill.

Could you specify these libraries as requirements and have pip or easy_install install them if necessary?

from dill.

I could be wrong, but I thought only pip handled dependencies.

Anyway, aren't these not really libraries, but rather different parts of the standard library that may or may not exist on various platforms? Even if separated versions exist, It seems pointless to install them just for the sake of dill, since all dill is doing is checking to see what it needs to care about (or at least, that was my understanding of this; correct me if I'm wrong).

from dill.

@mrocklin: Libraries like dbm are not requirements for dill, so no. dbm is part of the standard library, that for some reason is not distributed with Ubuntu. I've been trying to say that in my past few comments. See this post here: http://stackoverflow.com/questions/6171210/building-python-and-more-on-missing-modules. I have a patch that ignores the missing modules (see my previous comment), and that's why your install from github works. Apparently, I didn't apply the patch to the most recent release.

I'm also saying that the long load times and also the need to import dbm and half of standard library (as I do now) is unnecessary. I was trying to do something tricky (having all the possible picklable types load only after being requested), but apparently I didn't set that up right and they always load... which is not what I intended. That's the reason for the long load time, and I'm getting rid of it as soon as possible -- which is why I probably didn't apply the Ubuntu patch to the release.

I'm pretty sure that the issue with pip is purely due to my naming convention for the releases. I thought I had that fixed, but maybe not. I'll look into it. I believe it's this issue: http://stackoverflow.com/questions/18230956/could-not-find-a-version-that-satisfies-the-requirement-pytz. I believe, either you need to use the "prerelease" flag, or I need to drop the "a" from the version numbers.

from dill.

Ye, pip no longer installs prerelease a by default. Believe me, this is a good thing. You used to have to hide release candidates from it to keep them from being installed by everyone (this was coupled with the PyPI crawling that used to happen to everyone). If you want to indicate that dill is still alpha software, the 0 in front of the version number is sufficient.

from dill.

@asmeurer: the pip install thing is kinda annoying. I will probably have to adjust my naming convention to drop the "a". I'm kinda using it wrong... and I've not gotten to a "b" or higher. So probably best I change as suggested.

from dill.

As of de9df95, the install failure shouldn't be an issue. The load of test objects have been deferred until they are specifically requested. Since that's the bug, I'm closing this ticket. I'll continue to use "0.2a1" as a release, but starting now, "a" will be followed by a "b" and then a full release… which will work with pip. "0.2b1" should be imminent, and "0.2" shortly after.

from dill.

Finally, "0.2"

from dill.

Awesome

mrocklin@notebook:~$ conda create -n test-pathos python=2.7 pip
mrocklin@notebook:~$ source activate test-pathos
(test-pathos)mrocklin@notebook:~$ pip install dill
...
Successfully installed dill
(test-pathos)mrocklin@notebook:~$ python
>>> import dill
>>> dill.__version__
'0.2'

from dill.

I saw you were complaining about it on one of your own tickets. :)

from dill.