Comments (11)
This would be helpful to have the traceback, but I'm sure I will be able to reproduce this.
dill.source.getsource
uses readline
to get a function from history. @ivanov mentioned that ipython
uses it's own history buffer and not readline
, so I'm guessing that you are seeing a NameError
… and your code would work in IPython
-- just not the notebook. @vinc456: is that correct?
from dill.
Actually it's an "IOError: could not get source code" error.
The follow getsource() call works as expected in an IPython session running from the Windows command line.
In [1]: def doubler(f):
...: def func(*args, **kwds):
...: return 2 * f(*args, **kwds)
...: return func
...:
In [2]: @doubler
...:
...: def double_squared(x):
...: return x**2
...:
In [3]: double_squared(5)
Out[3]: 50
In [4]: import dill
In [5]: print dill.source.getsource(double_squared) # returns the expected result
%%javascript
from ctypes import CDLL
# This will crash a linux system; equivalent calls can be made on Windows or Mac
libc = CDLL("libc.so.6")
libc.time(-1) # BOOM!!
from time import sleep
sleep(2)
print 'test'
from time import sleep
sleep(2)
print 'hi'
%load soln/load.py
import dill
%clear
import dill
>>> def foo(x):
However, when running in the IPython Notebook or the IPython QT Console there is the following IOError.
print dill.source.getsource(double_squared)
---------------------------------------------------------------------------
IOError Traceback (most recent call last)
<ipython-input-5-a2b1f49c66be> in <module>()
----> 1 print dill.source.getsource(double_squared)
C:\Users\vinc456\Documents\GitHub\dill\dill\source.pyc in getsource(object, alias, lstrip, enclosing, force, builtin)
324 # get source lines; if fail, try to 'force' an import
325 try: # fails for builtins, and other assorted object types
--> 326 lines, lnum = getsourcelines(object, enclosing=enclosing)
327 except (TypeError, IOError): # failed to get source, resort to import hooks
328 if not force: # don't try to get types that findsource can't get
C:\Users\vinc456\Documents\GitHub\dill\dill\source.pyc in getsourcelines(object, lstrip, enclosing)
295 If lstrip=True, ensure there is no indentation in the first line of code.
296 If enclosing=True, then also return any enclosing code."""
--> 297 code, n = getblocks(object, lstrip=lstrip, enclosing=enclosing, locate=True)
298 return code[-1], n[-1]
299
C:\Users\vinc456\Documents\GitHub\dill\dill\source.pyc in getblocks(object, lstrip, enclosing, locate)
221 DEPRECATED: use 'getsourcelines' instead
222 """
--> 223 lines, lnum = findsource(object)
224
225 if ismodule(object):
C:\Users\vinc456\Documents\GitHub\dill\dill\source.pyc in findsource(object)
125
126 if not lines:
--> 127 raise IOError('could not get source code')
128
129 #FIXME: all below may fail if exec used (i.e. exec('f = lambda x:x') )
IOError: could not get source code
from dill.
I'll clean up the code and send you the IPython notebook file I was working during my layover today.
from dill.
Could you add triple backticks before and after the code, so that github doesn't interpret it as md.
from dill.
@vinc456: thanks. It's hard to parse your comment above with the traceback. Can you edit the markup as suggested?
from dill.
I've cleaned up the Markdown.
By the way, the Dill Primer I was working on can be found at https://dl.dropboxusercontent.com/u/1047492/Dill%20Primer.ipynb
There's probably a better way to share IPython Notebook files but I emailed a copy to John Tyree, who was going to have a look and maybe convert it to a .rst file. It's based on the typescript demo session Mike showed me at the first day of the SciPy 2014 Dill Sprint.
from dill.
Actually now that I have a closer look it seems like dill.source.getsource(double_squared) is not returning the expected result even in the case of the IPython terminal. I'm not sure what is going on here.
In [5]: print dill.source.getsource(double_squared) # not sure what is going on here
%%javascript
from ctypes import CDLL
# This will crash a linux system; equivalent calls can be made on Windows or Mac
libc = CDLL("libc.so.6")
libc.time(-1) # BOOM!!
from time import sleep
sleep(2)
print 'test'
from time import sleep
sleep(2)
print 'hi'
%load soln/load.py
import dill
%clear
import dil
But the same code will return the following results in a python terminal.
>>> print dill.source.getsource(double_squared, enclosing=True) # why should this be any different when called from the Python terminal?
def doubler(f):
def func(*args, **kwds):
return 2 * f(*args, **kwds)
return func
from dill.
@vinc456: Like I said, IPython
is not using readline
directly, and dill.source
assumes using readline
for history. IPython
apparently has it's own history mechanism, and it's different for both the terminal and the notebook. So, we'll need to figure out what exactly they do.
When dill.source
looks at readline
history, it converts the buffer into a big string, as it it had been read from a file… and then a good bit of dill.source
basically works like inspect
does on a file. The part with readline
can be abstracted out to a function (it's two lines), and I should be able to swap in whatever IPython
needs.
from dill.
@vinc456: I'm assuming this was a suitable conclusion…? or would you prefer a ticket with an option to remove IPython content so it looks like the same you'd get from raw python?
from dill.
Actually it's an "IOError: could not get source code" error.
The follow getsource() call works as expected in an IPython session running from the Windows command line.
In [1]: def doubler(f): ...: def func(*args, **kwds): ...: return 2 * f(*args, **kwds) ...: return func ...: In [2]: @doubler ...: ...: def double_squared(x): ...: return x**2 ...: In [3]: double_squared(5) Out[3]: 50 In [4]: import dill In [5]: print dill.source.getsource(double_squared) # returns the expected result %%javascript from ctypes import CDLL # This will crash a linux system; equivalent calls can be made on Windows or Mac libc = CDLL("libc.so.6") libc.time(-1) # BOOM!! from time import sleep sleep(2) print 'test' from time import sleep sleep(2) print 'hi' %load soln/load.py import dill %clear import dill >>> def foo(x):However, when running in the IPython Notebook or the IPython QT Console there is the following IOError.
print dill.source.getsource(double_squared) --------------------------------------------------------------------------- IOError Traceback (most recent call last) <ipython-input-5-a2b1f49c66be> in <module>() ----> 1 print dill.source.getsource(double_squared) C:\Users\vinc456\Documents\GitHub\dill\dill\source.pyc in getsource(object, alias, lstrip, enclosing, force, builtin) 324 # get source lines; if fail, try to 'force' an import 325 try: # fails for builtins, and other assorted object types --> 326 lines, lnum = getsourcelines(object, enclosing=enclosing) 327 except (TypeError, IOError): # failed to get source, resort to import hooks 328 if not force: # don't try to get types that findsource can't get C:\Users\vinc456\Documents\GitHub\dill\dill\source.pyc in getsourcelines(object, lstrip, enclosing) 295 If lstrip=True, ensure there is no indentation in the first line of code. 296 If enclosing=True, then also return any enclosing code.""" --> 297 code, n = getblocks(object, lstrip=lstrip, enclosing=enclosing, locate=True) 298 return code[-1], n[-1] 299 C:\Users\vinc456\Documents\GitHub\dill\dill\source.pyc in getblocks(object, lstrip, enclosing, locate) 221 DEPRECATED: use 'getsourcelines' instead 222 """ --> 223 lines, lnum = findsource(object) 224 225 if ismodule(object): C:\Users\vinc456\Documents\GitHub\dill\dill\source.pyc in findsource(object) 125 126 if not lines: --> 127 raise IOError('could not get source code') 128 129 #FIXME: all below may fail if exec used (i.e. exec('f = lambda x:x') ) IOError: could not get source code
this is still an issue for me, getsource
still fails in ipython (jupyter notebook)
from dill.
@pcko1: are you seeing the exact error above with the exact same code, or what?
from dill.
Related Issues (20)
- Cannot pickle TFAutomodels HOT 1
- dill.load - object has no attribute in custom class HOT 2
- Python 3.12: ModuleNotFoundError: No module named 'setuptools' HOT 5
- How to ensure the same functions serialize to the same bytes?
- Cannot use callable that was pickled within pytest HOT 14
- dill.source.getsource applied on lambda function returns many cached lines
- Class attributes changes at load if the object is in a list or tuple. HOT 1
- nan type drift for np.nan HOT 3
- dill.source.findsource fails when in asyncio REPL
- `save_function()` can't save function in a submodule that has the same name as an attribute of the parent module HOT 2
- bytes length not a multiple of item size
- dill.source.getsource returns decorated function instead of function HOT 2
- 0.3.7 incorrectly pickles the class definition for module/class with the same name HOT 7
- Exception: --- Logging error --- HOT 3
- Use dump_module() in jupyter notebook with remote kernel HOT 1
- UnpicklingError using dill but not stdlib pickle HOT 2
- Compatibility with Pyodide HOT 6
- tfp.math.psd_kernels.ExponentiatedQuadratic no longer serialiases with latest versions
- `dill.load_session()` raises `NameError: name 'UnpicklingError' is not defined` HOT 5
- pydantic>=2.5 classes can't be serialized HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dill.