juliapy / pycall.jl Goto Github PK

First reported at: https://groups.google.com/forum/?fromgroups=#!topic/julia-users/_aChEE7mZQo

I found that this was introduced after JuliaLang/julia@bec200eb80eb3bf that was an attempt to fix JuliaLang/julia#5319.

I have a relatively long data processing script that was working until Sunday with julia version:

b81307eff3352591bf8d24efce2a792f50a49da0
Version 0.2.0-2301.rb81307ef
Commit b81307eff3 2013-06-30 22:50:00
x86_64-linux-gnu

On Monday, after I pulled the latest sources from julia, the script started segfaulting. Digging trough the script I realized that the segfault happens at a point where a PyCall is made. This is a short test case causing the segfault on my system:

using PyCall
@pyimport scipy.signal as scisig

for i=1:100
    data = rand(4, 20*60*2048)
    f = [0, 0.6, 0.6, 1]; m = [1, 1, 0, 0];
    b = scisig.firwin2(2048, f, m)
end

The segmentation fault occurs both with python2.7 and python3.2 on Debian Wheezy.

If the data matrix above is smaller the segfault doesn't occur. So the problem seems somehow related to memory used. However, if I create another large object again trough python the error doesn't occur, so it seems somehow specific to the scipy.signal function above:

#no segfault with this
using PyCall
@pyimport scipy.signal as scisig
@pyimport numpy.random as nr

for i=1:200
    rec = rand(4, 20*60*2048)
    b = nr.rand(2048*20*60)
end

Not sure how implausible this is, but it would be great if typing
?[python function]
at the julia REPL gave back the docstring of that function/module, just as IPython's ? does.

Julia code would be faster than Python so have you considered translating the Python code into Julia (may be through syntactic macros using metaprogramming system) instead of wrapping. May be you can use wrapping as a fall back until full translation capabilities is achieved.

I've been getting random julia segfaults recently. Nothing specifically reproducible, but in this case, it happened at the REPL on the command immediately after I interrupted a PyCall function:

julia> df = E.extract_db_info()
^CERROR: interrupt
 in pycall at /scratch1/tmp/kmsquire/.julia/PyCall/src/PyCall.jl:62
 in anonymous at /scratch1/tmp/kmsquire/.julia/PyCall/src/conversions.jl:136
 in extract_db_info at /scratch1/tmp/kmsquire/Studies/TCGA_data/patients/ExtractDbInfo.jl:52

julia> df = E.extract_db_info();
Segmentation fault (core dumped)

The core dump gives

Core was generated by `julia'.
Program terminated with signal 11, Segmentation fault.
#0  0x00002b7fe2244967 in PyObject_IsInstance (inst=0x73e96e0, cls=0x2b7fe259b580) at Objects/abstract.c:2959
2959                if (Py_EnterRecursiveCall(" in __instancecheck__")) {

Perhaps pycall needs to be atomic in this case, or something isn't getting cleaned up by my interruption?

After the wx event loop gets started, SIGINT kills the Julia repl instead of interrupting the currently executing command. To reproduce:

julia> using PyCall

julia> @pylab

julia> pylab.plot(1);

At this point, if I close the plot and press ^C, Julia exits.

In the IPython source, it says that wx resets the SIGINT handler when it's loaded, so I tried running ccall(:jl_install_sigint_handler, Void, ()) the first time app.o != pynothing in gui.jl. This is an improvement, in that ^C now works if something is running, but something is still off, since ^C without anything running exits Julia instead of returning to the prompt.

Importing pylab, math, numpy all work. But passing Julia functions (whether anonymous or named) doesn't. Am I doing something wrong?

julia> @pyimport scipy.optimize as so

julia> so.newton(x -> cos(x) - x, 1)
ERROR: no method PyObject(Function,)
 in map at tuple.jl:37
 in pycall at /Users/Nick/.julia/PyCall/src/PyCall.jl:467
 in anonymous at /Users/Nick/.julia/PyCall/src/conversions.jl:82

I'm seeing this:

@pyimport numpy
x=pycall(numpy.random["randn"], PyArray, 3))
x
>> 3-element Float64 PyArray:
 #undef
 #undef
 #undef

Meanwhile,

show(STDOUT, x)
>> [-1.200717713975977,0.09520140647414062,2.2552768468145903]

so the problem must be in display somewhere.

Although PyCall converts objects to opaque wrapper types as a fallback, there is a long list of types for which a native conversion would be preferable:

• Julia Range <---> Python xrange
• Julia Set <---> Python set
• Julia composite types <---> Python classes
• Julia CalendarTime (from Calendar.jl) DateTime <---> Python datetime (via API)

and others.

Because of lack of support for array types I tried using h5py a python module to handle hdf5 files. I didn't found a way to convert a compound type directly. Therefore I wrote a small wrapper with the following tuple conversion

function pytuple2jltuple(t)
  num_elements = t[:__len__]()
  elements = Array(Any, num_elements)
  for i = 1:num_elements
    elements[i] = t[:__getitem__](i-1)
  end  
  return elements
end

But there are some problems with this. Although float scalars and arrays are automatically converted, there seems to be a problem with integer types, they still come out as PyObjects. The code is also painfully slow (I have a dataset with about 20M entries). Is there a way with the type conversion to speed this up?

Would be good to support conversion of Julia BigInt <---> Python long (a.k.a. int in Python 3), and BigFloat <---> Python bigfloat or mpmath.

It would be nicer to convert exception types to/from the corresponding "native" exception type when possible. e.g. TypeError exists in both Python and Julia.

Is there any interest in expanding the scope of this package to include more tools for conveniently wrapping Python libraries? I'm thinking of functionality like

• Functions for creating Julia types corresponding to Python classes. Perhaps the function could take an example instance of the PyObject, examine its __dict__ and slots attributes, and heuristically create a Julia type with corresponding fields.
• Macros for creating functions corresponding to methods for a given Pythton class
• A documented and easy-to-use API for users to hook into the PyAny conversion mechanism

I realize that these things can't be done in full generality since fields and methods can be dynamically created in Python at any time and the types of an instance's attributes might change at any time, but many Python packages don't really rely on that functionality.

Say I have Python object x which a custom getitem method (so that x[1] works in python). The Julia-wrapped version of x seems to require explicit invocation asx.getitem[1]. Is there a way to wrap Julia's getindex to map to Python's getitem, so that x[1] would work in Julia?

Using PyCall with the globally installed python works flawlessly. But I mainly work in virtualenvs, and would like julia to interact with my virtualenv's python.

I hoped just activating the virtualenv would be enough, but @pyimport numpy (and others) won't work when a virtualenv (in the following at /home/beyer/sci-env3-dyn) is activated.

First, everything works with python in the virtualenv:

$ python
Python 3.3.3 (default, Dec 20 2013, 23:51:18) 
[GCC 4.6.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.executable
'/home/beyer/sci-env3-dyn/bin/python'
>>> sys.path
['', '/home/beyer/sci-env3-dyn/lib/python33.zip', '/home/beyer/sci-env3-dyn/lib/python3.3', '/home/beyer/sci-env3-dyn/lib/python3.3/plat-linux', '/home/beyer/sci-env3-dyn/lib/python3.3/lib-dynload', '/home/beyer/insts/lib/python3.3', '/home/beyer/insts/lib/python3.3/plat-linux', '/home/beyer/sci-env3-dyn/lib/python3.3/site-packages']
>>> import numpy
>>>

Now, through julia called while inside the virtualenv:

$ ~/insts/bin/julia-readline 
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.3.0-prerelease+219 (2013-11-28 17:05 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit d114152 (0 days old master)
|__/                   |  x86_64-linux-gnu

julia> using PyCall

julia> @pyimport sys

julia> sys.executable
""

julia> sys.path
5-element Array{Any,1}:
 "/home/beyer/insts/lib/python33.zip"           
 "/home/beyer/insts/lib/python3.3"              
 "/home/beyer/insts/lib/python3.3/plat-linux"   
 "/home/beyer/insts/lib/python3.3/lib-dynload"  
 "/home/beyer/insts/lib/python3.3/site-packages"

julia> @pyimport numpy
ERROR: PyError (PyImport_ImportModule) <class 'ImportError'>
ImportError("No module named 'numpy'",)

 in pyerr_check at /home/beyer/.julia/PyCall/src/exception.jl:58
 in pyimport at /home/beyer/.julia/PyCall/src/PyCall.jl:85

julia> 

So, for some reason, when called through julia, neither sys.executable nor sys.path are correctly set. What is in sys.path (namely, stuff inside /home/beyer/insts/lib/python3.3) is the python install which has been used to create the virtualenv. I think pycall should "just work" when a virtualenv is active.

I don't really know how to debug this further, but in the following are some env-variables which might help debugging:

$ env | grep -i py
(nothing)
$ env | grep -i path
LD_LIBRARY_PATH=/home/beyer/sci-env3-dyn/lib:/opt/ros/groovy/lib
PATH=/home/beyer/sci-env3-dyn/bin:/opt/ros/groovy/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/lib/go/bin
$ env | grep -i virtual
VIRTUAL_ENV=/home/beyer/sci-env3-dyn

In the presence of multiple python interpreters on a single machine, is it possible to select which one to run?

For example, when I do:

julia> @pyimport sys
julia> sys.executable
"/home/drufat/Applications/epd/bin/python"

How can I get PyCall to use a different interpreter?

Relating to the problem we had yesterday. Running

using PyCall
@pyimport pylab as plt
plt.plot(rand(10))
plt.plot(rand(10))
plt.plot(rand(10))
while true
    plt.gcf()["canvas"][:print_figure](IOBuffer(),format="png")
end

under valgrind gives https://gist.github.com/loladiro/6134837 which has a bunch of instances of use-after-free even those that weren't allocated by Pycall. Are you at any point (in the I/O code perhaps) passing raw pointers to python?

I was trying to use PyCall to import ggplot from https://github.com/yhat/ggplot/

The qplot function works fine.

ggplot.qplot([1,2],[3,4])

However, I can never get the ggplot function working. Specially, I can never create the aes class

julia> using PyCall

julia> @pyimport ggplot

julia> ggplot.aes
PyObject <class ggplot.components.aes.aes at 0x11bb026d0>

julia> ggplot.aes(x="x",y="y")
ERROR: type: apply: expected Function, got PyObject

julia> pycall(ggplot.aes, PyObject, x="x",y="y")
ERROR: PyError (PyObject_Call) <type 'exceptions.AttributeError'>
AttributeError("'NoneType' object has no attribute 'f_locals'",)
  File "/Users/Randy/.pyenv/versions/2.7.6/Python.framework/Versions/2.7/lib/python2.7/site-packages/ggplot/components/aes.py", line 68, in __init__
    self.__eval_env__ = EvalEnvironment.capture(1)
  File "/Users/Randy/.pyenv/versions/2.7.6/Python.framework/Versions/2.7/lib/python2.7/site-packages/patsy/eval.py", line 179, in capture
    return cls([frame.f_locals, frame.f_globals],

 in pyerr_check at /Users/Randy/.julia/v0.3/PyCall/src/exception.jl:58
 in pycall at /Users/Randy/.julia/v0.3/PyCall/src/PyCall.jl:85

It seems that ggplot is calling patsy and checking some stack frame stuff.

Further study reveals that the root of the error is .../patsy/eval.py line 179

        frame = inspect.currentframe()
        try:
            for i in xrange(depth + 1):
                if frame is None:
                    raise ValueError, "call-stack is not that deep!"
                frame = frame.f_back
            return cls([frame.f_locals, frame.f_globals],
                       frame.f_code.co_flags & _ALL_FUTURE_FLAGS)

I have no idea what happens here...

I started today with PyCall.
I want to test the next.
On Python, I define a Seq object and get the reverse complement using:

from Bio.Seq import Seq
myseq = Seq("AGTACACTGGT")
myseq.reverse_complement()

But, with @pyimport I try and fail with:

julia> using PyCall

julia> @pyimport Bio.Seq as seq

julia> seq.Seq("AGTACACTGGT")
ERROR: type: apply: expected Function, got PyObject

julia> seq.Seq.__init__("AGTACACTGGT")
ERROR: type PyObject has no field __init__

P.S.: I note I can do this, but looks like you can't create a Python Object, only used his methods:

julia> seq.reverse_complement("AGTACACTGGT")
"ACCAGTGTACT"

Maybe in @pyimport macros you could add some other sintaxis, for example if the user dont need to load all the package and only one function or more of it:

@pyimport from numpy import zeros, array, fromtxt
@pyimport from mayavi import mlab
@pyimport from scipy.ndimage import gaussian_filter as filter

When I type @pyimport math, I get this error message:

ERROR: could not load module /Library/Frameworks/Python.framework/Versions/Current/bin/python: dlopen(/Library/Frameworks/Python.framework/Versions/Current/bin/python.dylib, 9): image not found
 in pyinitialize at /Users/Celestrist/.julia/PyCall/src/PyCall.jl:365
 in pyimport at /Users/Celestrist/.julia/PyCall/src/PyCall.jl:105

Any idea what happens?

Only certain multidimensional Array types can currently be converted to Python as NumPy arrays. As a fallback, when this fails (or for multidimensional AbstractArray conversions), PyCall should just switch to creating a PyList (or a list of lists in the multidimensional case).

This is a great piece of code, now if I could only get it to work.
I have an Imac I5 with 16 Gbyte and OSX 10.8.2
I try to run the following example:

using PyCall
@pyimport pylab
x = linspace(0,2_pi,1000); y = sin(3_x + 4_cos(2_x));
pylab.plot(x, y, @pykw color="red" linewidth=2.0 linestyle="--")
pylab.show()

And get the following error:

ERROR: PyError(":(:PyImport_ImportModule)",PyObject <type 'exceptions.ImportError'>)
in pyerr_check at /Users/jim/.julia/PyCall/src/PyCall.jl:209
in pyimport at /Users/jim/.julia/PyCall/src/PyCall.jl:236
in include_from_node1 at loading.jl:89
in process_options at client.jl:245
in _start at client.jl:308
at /private/var/folders/c7/3shp57yj3fvcm57w07_302_00000gn/T/Cleanup At Startup/PycallEx-383846484.299.jl:385
logout

What am I doing wrong?

julia> @pyimport sympy
ERROR: syntax: invalid assignment location false
 in pywrap at .../.julia/PyCall/src/PyCall.jl:640
 in pywrap at .../.julia/PyCall/src/PyCall.jl:629

• using julia 0.2.0, PyCall 0.2.1, Python's SymPy 0.7.4
• other imports (@pyimport numpy, @pyimport scipy) work flawlessly

The README shows the example:

@pyimport pylab
x = linspace(0,2*pi,1000); y = sin(3*x + 4*cos(2*x));
pylab.plot(x, y, @pykw color="red" linewidth=2.0 linestyle="--")
pylab.show()

This doesn't work on my machine. I get:

julia> pylab.plot(x, y, @pykw color="red" linewidth=2.0 linestyle="--")
ERROR: PyError (PyObject_Call) <type 'exceptions.IndexError'>
IndexError('list index out of range',)
  File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 2460, in plot
    ax = gca()
  File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 701, in gca
    ax =  gcf().gca(**kwargs)
  File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 369, in gcf
    return figure()
  File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 343, in figure
    **kwargs)
  File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_tkagg.py", line 80, in new_figure_manager
    window = Tk.Tk()
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1707, in __init__
    baseName = os.path.basename(sys.argv[0])

 in pyerr_check at /home/simon/.julia/PyCall/src/exception.jl:58
 in pycall at /home/simon/.julia/PyCall/src/PyCall.jl:85
 in anonymous at /home/simon/.julia/PyCall/src/conversions.jl:136

and indeed, at this point:

julia> @pyimport sys

julia> sys["argv"]
PyObject []

However, in a new Julia session in which I haven't tried to use matplotlib before:

julia> @pyimport sys

julia> sys["argv"]
PyObject ['julia']

With the following incantation, matplotlib works fine:

@pyimport sys
dont_let_this_get_garbage_collected = sys["argv"]
@pyimport pylab
x = linspace(0,2*pi,1000); y = sin(3*x + 4*cos(2*x));
pylab.plot(x, y, @pykw color="red" linewidth=2.0 linestyle="--")
pylab.show()

We allow for docstring access using Julia's help function for a PyObject in these lines. However, a Python function imported through PyCall is treated as an actual Julia function and therefore doesn't get the same treatment as an official PyObject. I think we should allow easy access to a function's docstrings.

Right now trying to simply repeat the lines linked to above for a function I get the following:

julia>using PyCall

julia>@pyimport scipy.optimize as opt

julia> pycall(pybuiltin("help"), PyAny, opt.fmin)
Help on jl_Function object:

class jl_Function(jlwrap)
 |  Method resolution order:
 |      jl_Function
 |      jlwrap
 |      __builtin__.object
 |
 |  Methods defined here:
 |
 |  __call__(...)
 |      x.__call__(...) <==> x(...)
 |
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |
 |  __new__ = <built-in method __new__ of type object>
 |      T.__new__(S, ...) -> a new object with type S, a subtype of T
 |
 |  ----------------------------------------------------------------------
 |  Methods inherited from jlwrap:
 |
 |  __hash__(...)
 |      x.__hash__() <==> hash(x)
 |
 |  __repr__(...)
 |      x.__repr__() <==> repr(x)
 |
 |  ----------------------------------------------------------------------
 |  Data descriptors inherited from jlwrap:
 |
 |  jl_value
 |      Julia jl_value_t* (Any object)

pyimport doesn't seem to be able to find python modules that are present in the local directory. Any reason for this? What's the best way to import user modules?

How can I do from sklearn.metrics import roc_auc_score on Julia's PyCall?

julia> @pyimport sklearn.metrics.roc_auc_score
ERROR: mname not defined

julia> @pyimport sklearn.metrics
ERROR: mname not defined

julia> @pyimport sklearn

julia> sklearn
__anon__

julia> sklearn["metrics"]
ERROR: no method getindex(Module,ASCIIString)

julia> sklearn[:metrics]
ERROR: no method getindex(Module,Symbol)

julia> sklearn.metrics
ERROR: metrics not defined

I was using the BeautifulSoup module to parse HTML and when I use the find_all method to search for all tags of a particular type I get an Any array with entries PyObject NULL returned. However, when I use the find method to get the first tag of a particular type I get a PyObject containing a bs4.element.Tag. Some simple code to demonstrate this:

using PyCall
@pyimport bs4

html_doc = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""

soup = bs4.BeautifulSoup(html_doc)
soup[:find_all](;name="a") # Find all anchor tags, returns Any array of PyObject NULL
soup[:find](;name="a)  # Find first anchor tag, returns a PyObject bs4.element.Tag

Thanks.

I'm able to write data to a file using pickle or cPickle, but I can't read it back again. I know PyCall succeeds in writing the data, because I can read it back using a python console, but if I try using Julia, I get the following error:

ERROR: PyError (PyObject_Call) <type 'exceptions.TypeError'>
TypeError("unhashable type: 'bytearray'",)
  File "/nfs/isd2/nhodas/anaconda/lib/python2.7/pickle.py", line 1378, in load
    return Unpickler(file).load()
  File "/nfs/isd2/nhodas/anaconda/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)

 in pyerr_check at /nfs/isd2/nhodas/.julia/PyCall/src/exception.jl:58
 in pycall at /nfs/isd2/nhodas/.julia/PyCall/src/PyCall.jl:85
 in fn at /nfs/isd2/nhodas/.julia/PyCall/src/conversions.jl:181


You can reproduce the code using this gist:
https://gist.github.com/nhodas/25349a9e3c3481993dcc

I'm trying to get the sympy package to work with PyCall. I get stuck with the call to pywrap:

julia> o = pyimport("sympy")
PyObject <module 'sympy' from '/Library/Python/2.7/site-packages/sympy/init.pyc'>

julia> pywrap(o)
ERROR: no method convert(Type{fn},Function)
in ##PyCall_PyWrapper#304 at no file
in pywrap at /Users/verzani/.julia/PyCall/src/PyCall.jl:487

The issue is this part of the initialization:

$(Expr(:call, tname, o,
[ :(convert(PyAny, $(members[i][2])))
for i = 1:length(members) ]...))

I tried a few things, but had no luck. Any pointers?

Hi,

I'm using Julia Version 0.2.0-2713.rf38c2b8d0 on a Mac, with OS 10.8.4.
I have Python 2.7, with a working pylab.
@pyimport pylab as pb still works, with the explicit blocking call to pb.show(), however @pylab as pb does not work. My julia prompt stops responding with

julia> pb.plot(x, y)
/Library/Python/2.7/site-packages/matplotlib/backends/backend_wx.py:1419:     wxPyDeprecationWarning: Using deprecated class PySimpleApp.
wxapp = wx.PySimpleApp()
^CSYSTEM: show(lasterr) caused an error^[[A^C^Cfatal: error thrown and no exception handler available.

and has to be Ctrl-C'd. How can I help debugging?

Edit: I have attempted @pyimport matplotlib as mpl, then making a call to mpl.use() and tried different backends- same problem.

What is wrong here?

In Python:

>>> import Bio.pairwise2 as py_aln
>>> import Bio.SubsMat.MatrixInfo as py_mat
>>> py_aln.align.globalds
<Bio.pairwise2.alignment_function instance at 0x7f0db3da7998>
>>> py_aln.align.globalds("AHHH","HHHA",py_mat.ident,-0.25,-0.5)
[('AHHH-', '-HHHA', 17.5, 0, 5)]

In Julia:

julia> using PyCall
julia> @pyimport Bio.pairwise2 as py_aln
julia> @pyimport Bio.SubsMat.MatrixInfo as py_mat
julia> py_aln.align[:globalds]
PyObject <Bio.pairwise2.alignment_function instance at 0xaf48560>
julia> py_aln.align[:globalds]("AHHH","HHHA",py_mat.ident,-0.25,-0.5)
ERROR: type: apply: expected Function, got PyObject

Hello,
I use Ubuntu 12.04, 64 bit. I installed numpy 1.7.1 with pip.

Importing numpy fails:

julia> using PyCall

julia> @pyimport numpy
ERROR: PyError (PyImport_ImportModule) <type 'exceptions.ImportError'>
ImportError('cannot import name scimath',)
File "/usr/local/lib/python2.7/dist-packages/numpy/init.py", line 153, in
from . import add_newdocs
File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in
from numpy.lib import add_newdoc
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/init.py", line 17, in
from . import scimath as emath

in pyerr_check at /home/ufechner/.julia/PyCall/src/exception.jl:58
in pyimport at /home/ufechner/.julia/PyCall/src/PyCall.jl:85

Importing numpy from python works fine, importing other packages from julia
also works.

Any idea?

Uwe Fechner

Hi,

I am trying to use mayavi from julia. For now, it seems to work just fine, but the interaction with the gui only works after mlab.show(), which blocks julia; so, I can't programatically manipulate the mayavi pipeline from julia and interact with the mayavi gui...
In ipython the same happens, unless it is invoked with --gui=wx... can the same be achieved in PyCall?

Thanks.

Miguel Gaspar

On d1d07cc I have:

julia> using PyCall
julia> @pyimport pylab as p
julia> 

But on b64f210 I get:

julia> using PyCall                                                                                                                    

julia> @pyimport pylab as p
ERROR: Couldn't find libpython (libpython2.7.so); try pyinitialize("/path/to/libpython2.7.so")
 in error at error.jl:21
 in libpython_name at /home/bana/.julia/PyCall/src/PyCall.jl:342
 in pyinitialize at /home/bana/.julia/PyCall/src/PyCall.jl:353
 in pyimport at /home/bana/.julia/PyCall/src/PyCall.jl:105

julia> pyinitialize("/usr/lib/x86_64-linux-gnu/libpython2.7.so")                                                                       
ERROR: could not start process `/usr/lib/x86_64-linux-gnu/libpython2.7.so -c 'import sys; print(sys.executable)'`: permission denied (EACCES)
 in test_success at process.jl:420
 in success at process.jl:428
 in readall at process.jl:393
 in pyinitialize at /home/bana/.julia/PyCall/src/PyCall.jl:357

And it looks like this is because:

$ ls -l /usr/lib/x86_64-linux-gnu/libpython*
lrwxrwxrwx 1 root root   51 Apr 19 13:22 /usr/lib/x86_64-linux-gnu/libpython2.7.a -> ../python2.7/config-x86_64-linux-gnu/libpython2.7.a
lrwxrwxrwx 1 root root   17 Apr 19 13:22 /usr/lib/x86_64-linux-gnu/libpython2.7.so -> libpython2.7.so.1
lrwxrwxrwx 1 root root   19 Aug 16 10:57 /usr/lib/x86_64-linux-gnu/libpython2.7.so.1 -> libpython2.7.so.1.0
-rw-r--r-- 1 root root 3.2M Apr 19 13:22 /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0

This is on Ubuntu 13.04 with bleeding edge Julia.

If a Python object supports one or more of the IPython display methods, it would be good to automatically implement the corresponding writemime method so that the object displays graphically in IJulia (etc.).

Will also need to overload mimewritable for PyObject, since in this case checking method_exists(writemime, ...) won't be the correct thing to do.

See also this julia-dev discussion.

cc: @fperez, @JeffBezanson

The conversion into Complex doesn't work with sympy objects:

julia> using PyCall
julia> @pyimport sympy
julia> z = sympy.sqrt(-1)
PyObject I
julia> convert(Complex, z)
ERROR: PyError (PyComplex_RealAsDouble) <type 'exceptions.TypeError'>
TypeError("can't convert complex to float",)
File "/Library/Python/2.7/site-packages/sympy/core/expr.py", line 240, in float
raise TypeError("can't convert complex to float")

in pyerr_check at /Users/verzani/.julia/PyCall/src/exception.jl:58
in convert at /Users/verzani/.julia/PyCall/src/conversions.jl:62

Any thoughts on a workaround?

Hi,
Reporting a crash during any call of a python function.
Config:

• Win 7 64 bits
• Julia Version 0.3.0-prerelease+2809 (2014-04-28 22:41 UTC) Commit d1095bb* (8 days old master) x86_64-w64-mingw32 (latest snapshot installer)
• PyCall.jl version 0.4.2
• Python version 2.7.6 from Enthought Canopy 64 bits (the only python installed on the machine) (comes with matplotlib and dependancies)

Steps to reproduce:

julia> using PyCall
julia> @pyimport math

Error message:
ImportError: No module named site

Other infos:

• python works fine from within the Windows cmd:

C:\Users\cmey\Downloads\Julia 0.3.0-prerelease>python
Enthought Canopy Python 2.7.6 | 64-bit | (default, Apr 11 2014, 20:31:44) [MSC v
.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> from math import *
>>> sin(math.pi / 4) - sin(pi / 4)  # returns 0.0
0.0

• I launch Julia from within the same Windows cmd as Python
• PYTHONHOME and PYTHONPATH are unset

Might be related to #42, but my case differs at least at the OS level.

As a workaround, may I ask for a different and recomended python distribution that works with PyPlot on Win 7 64 ?

@pyimport is not able to find libpython2.7.so:

simonp@ubuntu:~$ julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.2.0-1796.rbac428f9
 _/ |\__'_|_|_|\__'_|  |  Commit bac428f990 2013-06-03 15:11:03
|__/                   |


julia> using PyCall

julia> @pyimport math
ERROR: could not load module libpython2.7.so: libpython2.7.so: cannot open shared object file: No such file or directory
 in pyinitialize at /home/simonp/.julia/PyCall/src/PyCall.jl:320
 in pyimport at /home/simonp/.julia/PyCall/src/PyCall.jl:105
...

I looked around for similarly named libraries and found the following:

/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1
/usr/lib/python2.7/config-x86_64-linux-gnu/libpython2.7.so
/usr/share/doc/libpython2.7-minimal
/usr/share/doc/libpython2.7-stdlib
/usr/share/doc/libpython2.7
/usr/share/lintian/overrides/libpython2.7-minimal
/usr/share/lintian/overrides/libpython2.7-stdlib
/usr/share/lintian/overrides/libpython2.7
...

Any suggestions? I've used mostly Windows in the past, so I'm not sure what to do at this point.

Thanks,
Peter

INFO: Installing PyCall v0.4.5
INFO: Package database updated
ERROR: test failed: collect(PyObject([1,3,5])) == [1,3,5]
 in error at error.jl:21
 in default_handler at test.jl:19
 in do_test at test.jl:39
 in include at boot.jl:244
 in include_from_node1 at loading.jl:128
while loading /home/idunning/pkgtest/.julia/v0.3/PyCall/test/runtests.jl, in expression starting on line 66
INFO: Package database updated

Just so it doesn't get forgotten.

I think that PyCall is great and was really excited to see I could use pyplot from it. However, I've hit a small problem.

I am using Python 2.7.2 and numpy 1.8_dev_3abd869 installed with the ScipySuperpack on OSX 10.8. In the npyinitialize function on line 55 of numpy.jl the attempt to find the function pointer to the numpy multiarray _ARRAY_API as a PyCapsule returns a NULL pointer and thus anything that used numpy arrays failed, for example the pyplot example. I found that on my system numpy.core.multiarray._ARRAY_API is a PyCObject so I tried the code on line 51 of numpy.jl and a non-zero function pointer was returned. I hacked the Python 2.7.2 case to try the PyCapsule version and if a NULL pointer is returned then to try the PyCObject version. Now the pyplot example works as do numpy functions that use arrays.

I'm not sure what the right way is to handle this, but I just wanted to bring it to your attention.

Thanks.

Currently, keyboard interrupts are deferred during execution of Python code, since CPython is not interrupt-safe. It would be nice to catch these interrupts and set a KeyboardInterrupt Python exception so that Python code is halted gracefully.

Currently, PyObject(s::Symbol) produces a Python string, and convert(Symbol, o) converts a Python string object back into a Symbol. However, automatic type conversion does not work, i.e. convert(PyAny, PyObject(:Foo)) produces "Foo" not :Foo.

One way to fix this might be to define a new Python string subtype, and instantiate this type from PyObject(s::Symbol). That way, it will be usable as a string in Python but convert(PyAny, ...) will be able to detect that it is "really" a symbol and convert it appropriately.

I am trying to use IJulia and PyPlot. It causes a kernel restart as soon as I enter "using pyplot". I tracked it down to what looks like a basic failure of PyCall.

I am using Anaconda Python 2.7.5 on Windows 7.

Also I'm using 64 bit Julia but 32 bit Python. I wonder if it matters.

c:\julia-05c6461b55>conda -V
conda 2.3.0

c:\julia-05c6461b55>where python
C:\Anaconda\python.exe

c:\julia-05c6461b55>python -V
Python 2.7.5 :: Anaconda 1.8.0 (32-bit)

c:\julia-05c6461b55>julia -v
julia version 0.2.0

c:\julia-05c6461b55>julia -e "using PyCall; @pyimport math"
Traceback (most recent call last):
  File "C:\Anaconda\lib\site.py", line 548, in <module>
    main()
  File "C:\Anaconda\lib\site.py", line 530, in main
    known_paths = addusersitepackages(known_paths)
  File "C:\Anaconda\lib\site.py", line 266, in addusersitepackages
    user_site = getusersitepackages()
  File "C:\Anaconda\lib\site.py", line 241, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "C:\Anaconda\lib\site.py", line 231, in getuserbase
    USER_BASE = get_config_var('userbase')
  File "C:\Anaconda\lib\sysconfig.py", line 516, in get_config_var
    return get_config_vars().get(name)
  File "C:\Anaconda\lib\sysconfig.py", line 449, in get_config_vars
    import re
  File "C:\Anaconda\lib\re.py", line 105, in <module>
    import sre_compile
  File "C:\Anaconda\lib\sre_compile.py", line 14, in <module>
    import sre_parse
  File "C:\Anaconda\lib\sre_parse.py", line 17, in <module>
    from sre_constants import *
  File "C:\Anaconda\lib\sre_constants.py", line 18, in <module>
    from _sre import MAXREPEAT
ImportError: cannot import name MAXREPEAT

This would eliminate the NumPy dependency (and the ugly parsing of NumPy header files). See PEP 3118 and the Python buffer docs.

cc: @jakevdp, @StefanKarpinski

I have multiple versions of python, and switch among them using virtualenv. However, PyCall try to detect default python using the system call

bash -lc "which python"

which will always return the python set in the .bash_profile. Why not omit the -l option in the bash command, so that whatever python I am using now can be detected, and the appropriate python module can be imported?

Using python3.2 on Debian Wheezy, if I reload a script using PyCall the first time it works fine, but the second time I get an error. This does not happen with python2.7

 julia> require("ex.jl")

julia> reload("ex.jl")
ERROR: PyError (PyObject_Call) <class 'TypeError'>
TypeError("object of type 'NoneType' has no len()",)
  File "/usr/lib/python3.2/inspect.py", line 266, in getmembers
    for key in dir(object):

 in pyerr_check at /home/sam/.julia/PyCall/src/exception.jl:58
 in pycall at /home/sam/.julia/PyCall/src/PyCall.jl:85
 in pywrap at /home/sam/.julia/PyCall/src/PyCall.jl:518
 in include_from_node1 at loading.jl:91
 in reload_path at loading.jl:114
 in reload at loading.jl:59
at /media/ntfsShared/otherNoSynch/tmp/ex.jl:562

Test case, $ cat ex.jl

using PyCall
pyinitialize("python3")
@pyimport numpy as np
x = np.arange(10)

Since Pypy is a faster (JIT compiler) version of Python. Is possible to call Pypy instead of CPython ? How can be do it ?
Best,

I am a Python developer and have multiple working versions of Python on my machine.

I cannot, however, get PyCall to work after simply following the instructions from the readme and doing Pkg.add("PyCall"). This is what a sample session looks like:

julia> using PyCall

julia> @pyimport scipy.optimize as so
ERROR: could not load module /libpython2.7.dylib: dlopen(/libpython2.7.dylib.dylib, 9): image not found
 in pyinitialize at /Users/sglyon/.julia/PyCall/src/PyCall.jl:339
 in pyimport at /Users/sglyon/.julia/PyCall/src/PyCall.jl:105

The error message is fairly clear at explaining what is happening, but I have no idea why PyCall is looking for python at / or how I tell it where my Python is. I would like to use the python at ~/anaconda/bin/python. I tried to be more explicit and set $PYTHONHOME to various values near that directory. I have tried all of the below values:

• export PYTHONHOME="/Users/sglyon/anaconda/lib/python2.7"
• export PYTHONHOME="/Users/sglyon/anaconda/bin"
• export PYTHONHOME="/Users/sglyon/anaconda/lib/"
• export PYTHONHOME="/Users/sglyon/anaconda/"

For each of those values I get the following printout (it is the same for any of the listed values):

julia> using PyCall

julia> @pyimport scipy.optimize as so
ImportError: No module named site
ERROR: failed process: Process(`python -c "import distutils.sysconfig; print(distutils.sysconfig.get_config_var('LDLIBRARY'))"`, ProcessExited(1)) [1]
 in error at error.jl:22
 in pipeline_error at process.jl:436
 in readall at process.jl:394
 in libpython_name at /Users/sglyon/.julia/PyCall/src/PyCall.jl:328
 in pyinitialize at /Users/sglyon/.julia/PyCall/src/PyCall.jl:339
 in pyimport at /Users/sglyon/.julia/PyCall/src/PyCall.jl:105

What other configuration do I need to do to get this to work with the python of my choosing? (Note that the anaconda python is the first on my$PATH and is executing when typing python from the command line.)