zhuyifei1999 / guppy3 Goto Github PK
View Code? Open in Web Editor NEWguppy / heapy ported to Python3. It works for real!
Home Page: https://zhuyifei1999.github.io/guppy3/
License: MIT License
guppy / heapy ported to Python3. It works for real!
Home Page: https://zhuyifei1999.github.io/guppy3/
License: MIT License
Hi @zhuyifei1999,
Happened in Windows OS.
Installed Official Dropbox SDK from https://pypi.org/project/dropbox/.
pip install dropbox
Ran Python Interpreter
python
Python 3.8.10 (default, May 19 2021, 13:12:57) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
import dropbox
import guppy
hp = guppy.hpy()
before = hp.heap()
Windows thrown out exception:
Python has stopped working.
Problem signature:
Problem Event Name: APPCRASH
Application Name: python.exe
Application Version: 3.8.10150.1013
Application Timestamp: 60a5556d
Fault Module Name: python38.dll
Fault Module Version: 3.8.10150.1013
Fault Module Timestamp: 60a55542
Exception Code: c0000005
Exception Offset: 000000000006dc5b
Locale ID: 17417
Additional Information 1: b59d
Additional Information 2: b59df1129b1415b7f11d657440546bce
Additional Information 3: 53eb
Additional Information 4: 53eb508f924550baeb8a6a8bb1af95dd
If skip importing Dropbox SDK or hpy().heap(), there is no issue.
It is a showstopper issue to me as I'm looking forward for both dropbox and guppy module to work together.
Right now it just saves all the rows in the stats. That's someone one could use .all
and just save the string.
sets directory is empty
Unable to import
from guppy.sets.setsc import BitSet # base bitset type
from guppy.sets.setsc import ImmBitSet # immutable bitset type
from guppy.sets.setsc import immbit # immutable bitset singleton constructor
from guppy.sets.setsc import immbitrange # immutable bitset range constructor
from guppy.sets.setsc import immbitset # immutable bitset constructor
from guppy.sets.setsc import MutBitSet # mutable bitset
from guppy.sets.setsc import NodeSet # base nodeset type
from guppy.sets.setsc import ImmNodeSet # immmutable nodeset type
from guppy.sets.setsc import MutNodeSet # mutable nodeset type
Per-interpreter GIL might be coming to Python 3.9 [1]. We have zero thread-safety and rely on GIL to make sure other threads don't modify stuffs when we are accessing them. With per-interpreter GIL, traversing across interpreter boundary is unsafe... ideas?
I'm trying to use Guppy but as soon as I import it, python crashes with a segfault:
root@4738adfde8d2:/usr/src/app# ipython
Python 3.7.1 (default, Nov 16 2018, 22:26:09)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.8.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: from guppy import hpy
Segmentation fault
Linux 4738adfde8d2 4.9.125-linuxkit #1 SMP Fri Sep 7 08:20:28 UTC 2018 x86_64 GNU/Linux
Not sure what information I can provide to help?
To reproduce, with just a dummy tracer:
$ python -c 'trace = lambda frame, event, arg: trace; __import__("sys").settrace(trace); __import__("guppy").hpy().test()'
[...]
imported: guppy.heapy.test.test_Classifiers
test_base_classes (guppy.heapy.test.test_Classifiers.BaseCase) ... GC hook object was referred to from somebody!
ok
test_fancy_list_args (guppy.heapy.test.test_Classifiers.BaseCase) ... GC hook object was referred to from somebody!
ok
test_fancy_type_conversions (guppy.heapy.test.test_Classifiers.BaseCase) ... GC hook object was referred to from somebody!
ok
test_invalid_operations (guppy.heapy.test.test_Classifiers.BaseCase) ... GC hook object was referred to from somebody!
ok
test_minmax (guppy.heapy.test.test_Classifiers.BaseCase) ... GC hook object was referred to from somebody!
ok
----------------------------------------------------------------------
Ran 5 tests in 0.182s
OK
test_classification (guppy.heapy.test.test_Classifiers.ClassificationCase) ... GC hook object was referred to from somebody!
ok
test_selection (guppy.heapy.test.test_Classifiers.ClassificationCase) ... GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
ok
----------------------------------------------------------------------
Ran 2 tests in 19.371s
OK
test_laws (guppy.heapy.test.test_Classifiers.LawsCase) ... GC hook object was referred to from somebody!
ok
----------------------------------------------------------------------
Ran 1 test in 0.872s
OK
test_rendering (guppy.heapy.test.test_Classifiers.RenderCase) ... GC hook object was referred to from somebody!
ok
----------------------------------------------------------------------
Ran 1 test in 0.190s
OK
test_1 (guppy.heapy.test.test_Classifiers.SpecialCases) ... GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
GC hook object was referred to from somebody!
ok
test_2 (guppy.heapy.test.test_Classifiers.SpecialCases) ... GC hook object was referred to from somebody!
ok
test_alt_retclaset (guppy.heapy.test.test_Classifiers.SpecialCases) ... GC hook object was referred to from somebody!
ok
test_dictowner (guppy.heapy.test.test_Classifiers.SpecialCases) ... GC hook object was referred to from somebody!
ok
test_retclaset (guppy.heapy.test.test_Classifiers.SpecialCases) ... GC hook object was referred to from somebody!
aseq: Expected: b = 8
Got actually : a = 22
FAIL
test_via (guppy.heapy.test.test_Classifiers.SpecialCases) ... GC hook object was referred to from somebody!
FAIL
======================================================================
FAIL: test_retclaset (guppy.heapy.test.test_Classifiers.SpecialCases)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/test_Classifiers.py", line 347, in test_retclaset
self.aseq(grc(C1), rcC1) # (A)
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/support.py", line 249, in aseq
self.assertTrue(0)
AssertionError: 0 is not true
======================================================================
FAIL: test_via (guppy.heapy.test.test_Classifiers.SpecialCases)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/test_Classifiers.py", line 425, in test_via
s.byvia.kind == hp.Via("_.f_locals['v']", "_[('k',)]", "_[('v',)]", '_.keys()[0]'))
AssertionError: False is not true
----------------------------------------------------------------------
Ran 6 tests in 2.720s
FAILED (failures=2)
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/zhuyifei1999/guppy3/guppy/heapy/Use.py", line 363, in test
self._parent.test.test_all.test_main(debug)
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/test_all.py", line 35, in test_main
f(debug=debug)
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/test_Classifiers.py", line 959, in test_main
support.run_unittest(SpecialCases, debug)
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/support.py", line 186, in run_unittest
run_suite(suite, testclass)
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/support.py", line 176, in run_suite
raise TestFailed(msg)
guppy.heapy.test.support.TestFailed: errors occurred in guppy.heapy.test.test_Classifiers.SpecialCases
The reference chains seem to contain unexpected values.
If the tracer / profiler is completely C then it works fine (try using profile
and cProfile
to profile the tests).
@svenil found that this is probably related to the use of PyFrame_FastToLocalsWithError in call_trampoline, and test.test_sys.SysModuleTest.test_refcount & test.support.refcount_test may explain the reason.
Quote the thesis:
A producer profile classifies cells by the program components that created them.
Of these, the producer profile would require special instrumentation of the Python interpreter to record the call stack when allocating objects, and is therefore outside the current scope of Heapy.
But we no longer need special instrumentation now that Python 3 supports it natively:
$ python -X tracemalloc=10
Python 3.7.4 (default, Aug 26 2019, 23:27:06)
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tracemalloc
>>> for f in tracemalloc.get_object_traceback(tracemalloc.get_object_traceback).format(): print(f)
...
File "<stdin>", line 1
File "<frozen importlib._bootstrap>", line 983
File "<frozen importlib._bootstrap>", line 967
File "<frozen importlib._bootstrap>", line 677
File "<frozen importlib._bootstrap_external>", line 728
File "<frozen importlib._bootstrap>", line 219
File "/usr/lib/python3.7/tracemalloc.py", line 235
def get_object_traceback(obj):
We could probably somehow use tracemalloc instead of hook into python memory allocator to avoid reinventing the wheel. However, this has a non-negligible memory overhead so we probably don't want to enable it as default and show a message if no object would be classified this way.
Similarly, it would be nice if we could get the trace of a single object with some convenient attribute access on an identityset. Though, admittedly, reference graphs are far more useful.
How should this classifier be called? byprod? bytrace? byalloc?
Not sure if this is fixable on our side, but I should document this:
$ python -i -c 'hp = __import__("guppy").hpy()'
>>> h = hp.heap()
>>> h.[TAB][TAB][HANGS]
GDB:
gef➤ py-bt
Traceback (most recent call first):
File "/home/zhuyifei1999/guppy3/guppy/heapy/Classifiers.py", line 620, in owners
self.mod.hv.update_dictowners(drg)
File "/home/zhuyifei1999/guppy3/guppy/heapy/Classifiers.py", line 717, in owners
return self.d.owners(X)
File "/home/zhuyifei1999/guppy3/guppy/heapy/UniSet.py", line 1692, in get_owners
return self.mod.Use.Clodo.classifier.owners(a)
File "/home/zhuyifei1999/guppy3/guppy/heapy/UniSet.py", line 521, in <lambda>
owners = property(lambda self: self.fam.get_owners(self), doc="""\
<built-in method getattr of module object at remote 0x7f05681d4d10>
File "/usr/lib/python3.7/rlcompleter.py", line 173, in attr_matches
val = getattr(thisobject, word)
File "/usr/lib/python3.7/rlcompleter.py", line 89, in complete
self.matches = self.attr_matches(text)
gef➤ py-up
#11 Frame 0x559685b8fd90, for file /home/zhuyifei1999/guppy3/guppy/heapy/Classifiers.py, line 717, in owners (self=<ByClassOrDictOwner(mod=<Interface(_share=<Share(module=<module at remote 0x7f0567dead70>, parent=<Share(module=<module at remote 0x7f0567eda8f0>, parent=<Share(module=<module at remote 0x7f0568079f50>, parent=<Share(module=None, parent=<...>, name='', setable=(), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': None, 'guppy': <...>, 'types': <Share(module=<module at remote 0x7f0568092110>, parent=<...>, name='types', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': <module at remote 0x7f0568092110>, 'ModuleType': <type at remote 0x7f05692bc620>, 'CodeType': <type at remote 0x7f05692c1180>, 'FrameType': <type at remote 0x7f05692bee00>}, owners={}, ispackage=None, recursion=0, preload=(...)) at remote 0x7f0567e45ed0>, 'io': <Share(module=<...(truncated)
return self.d.owners(X)
gef➤
#15 Frame 0x559685a99280, for file /home/zhuyifei1999/guppy3/guppy/heapy/UniSet.py, line 1692, in get_owners (self=<IdentitySetFamily(mod=<Interface(_share=<Share(module=<module at remote 0x7f0567e700b0>, parent=<Share(module=<module at remote 0x7f0567eda8f0>, parent=<Share(module=<module at remote 0x7f0568079f50>, parent=<Share(module=None, parent=<...>, name='', setable=(), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': None, 'guppy': <...>, 'types': <Share(module=<module at remote 0x7f0568092110>, parent=<...>, name='types', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': <module at remote 0x7f0568092110>, 'ModuleType': <type at remote 0x7f05692bc620>, 'CodeType': <type at remote 0x7f05692c1180>, 'FrameType': <type at remote 0x7f05692bee00>}, owners={}, ispackage=None, recursion=0, preload=(...)) at remote 0x7f0567e45ed0>, 'io': <Share(module=<m...(truncated)
return self.mod.Use.Clodo.classifier.owners(a)
gef➤
#19 Frame 0x559685b90bb0, for file /home/zhuyifei1999/guppy3/guppy/heapy/UniSet.py, line 521, in <lambda> (self=<IdentitySetMulti at remote 0x7f0567ee4230>)
owners = property(lambda self: self.fam.get_owners(self), doc="""\
gef➤
#29 <built-in method getattr of module object at remote 0x7f05681d4d10>
gef➤
#31 Frame 0x559685d6e970, for file /usr/lib/python3.7/rlcompleter.py, line 173, in attr_matches (self=<Completer(use_main_ns=1, namespace={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <type at remote 0x5596859dbe50>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7f05681d4d10>, 'hp': <_GLUECLAMP_(_share=<Share(module=<module at remote 0x7f0567edaad0>, parent=<Share(module=<module at remote 0x7f0567eda8f0>, parent=<Share(module=<module at remote 0x7f0568079f50>, parent=<Share(module=None, parent=<...>, name='', setable=(), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': None, 'guppy': <...>, 'types': <Share(module=<module at remote 0x7f0568092110>, parent=<...>, name='types', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': <module at remote 0x7f0568092110>, 'ModuleType': <type at remote 0x7...(truncated)
val = getattr(thisobject, word)
gef➤ py-print thisobject
local 'thisobject' = <IdentitySetMulti at remote 0x7f0567ee4230>
gef➤ py-print word
local 'word' = 'owners'
gef➤ py-print words
local 'words' = {'__doc__', '__new__', '_doc_nodes', 'owners', '__reduce__', '__getattr__', '__or__', '__subclasshook__', '__dir__', '__bool__', '__xor__', '__str__', '__delattr__', '__mul__', '__getattribute__', '__len__', 'pathsout', '__ge__', '__eq__', '__rand__', '__rxor__', '_partition', 'size', '__call__', 'fam', '__le__', '__and__', '_help_url_', 'test_contains', 'referrers', 'er', '__lshift__', '__hash__', '__sub__', '__class__', '__iter__', '_hiding_tag_', '__module__', 'more', '_instahelp_', 'bysize', '__contains__', '__format__', '_er', '__sizeof__', 'get_examples', 'biper', 'get_rp', 'byid', '__lt__', 'bytype', 'by', 'brief', 'byunity', 'parts', 'stat', '_origin_', 'nodes', 'partition', 'disjoint', 'doc', '_get_help', 'byclodo', '__gt__', '__repr__', '_derive_origin_', 'dump', 'kind', 'rp', 'dictof', '__rsub__', '__getitem__', 'get_render', 'shpaths', 'count', 'sp', 'indisize', '__setattr__', '__slots__', 'maprox', 'get_shpaths', 'referents', 'byrcs', 'byidset', 'get_ckc', '__init_subclass__', '__ne__', 'pathsin'...(truncated)
gef➤ py-list
168 for word in words:
169 if (word[:n] == attr and
170 not (noprefix and word[:n+1] == noprefix)):
171 match = "%s.%s" % (expr, word)
172 try:
>173 val = getattr(thisobject, word)
174 except Exception:
175 pass # Include even if attribute not set
176 else:
177 match = self._callable_postfix(val, match)
178 matches.append(match)
This is https://github.com/python/cpython/blob/77abf23c67c1a465a8899666c69f6bcd6930e003/Lib/rlcompleter.py#L173, it wants to know whether each of out attributes are callable.
I would like to dive deeper into the memory consumption of a program using JAX's numpy. However, with guppy I am unfortunately not able to get out any information since the formatting of the result fails during stringifying and printing the result. One can easily reproduce the error using the following simple program:
from jax import numpy as jnp
from guppy import hpy
a = jnp.linspace(0.0, 1.0)
h = hpy() # succeeds
print(h.heap())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.9/site-packages/guppy/heapy/UniSet.py", line 349, in __str__
return self.fam.c_str(self)
File "/usr/lib/python3.9/site-packages/guppy/heapy/UniSet.py", line 1616, in c_str
return a.more._oh_printer.get_str_of_top()
File "/usr/lib/python3.9/site-packages/guppy/heapy/UniSet.py", line 74, in __getattr__
return self.fam.mod.View.enter(lambda: self.fam.c_getattr(self, other))
File "/usr/lib/python3.9/site-packages/guppy/heapy/View.py", line 256, in enter
retval = func()
File "/usr/lib/python3.9/site-packages/guppy/heapy/UniSet.py", line 74, in <lambda>
return self.fam.mod.View.enter(lambda: self.fam.c_getattr(self, other))
File "/usr/lib/python3.9/site-packages/guppy/heapy/UniSet.py", line 800, in c_getattr
return self.c_getattr2(a, b)
File "/usr/lib/python3.9/site-packages/guppy/heapy/UniSet.py", line 803, in c_getattr2
raise AttributeError(b)
AttributeError: more
I know it might be quite a stretch to ask guppy to support custom libraries like JAX so I would already be happy about insights into how to resolve it myself or at least how to work-around it.
Tracing some relevant functions:
test_horizon (guppy.heapy.test.test_View.FirstCase) ... [pid 2010953] horizon_new(0x7ffff6990940 <NyHorizon_Type>, (<guppy.heapy.heapyc.HeapView at remote 0x7ffff606c898>,), 0x0) = <unfinished ...>
[pid 2010953] horizon_patch_dealloc(0x7ffff6990c80 <NyRootState_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d4eb40 <PyDict_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d48860 <PyModule_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d4ff00 <PyList_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d4f840 <PyCFunction_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d4ad00 <PyFrame_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d48520 <_PyNone_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d4cf00 <PyCode_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d4f360 <PyBaseObject_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d4f500 <PyType_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d500a0 <PyTuple_Type>) = 0x0
[...]
[pid 2010953] horizon_patch_dealloc(0x7ffff7d5d640 <PyBufferedReader_Type>) = 0x0
[pid 2010953] horizon_patch_dealloc(0x7ffff7d5b920 <ZipImporter_Type>) = 0x0
[pid 2010953] horizon_patched_dealloc([]) = void
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.MutNodeSet at remote 0x7ffff6452fc0>) = <unfinished ...>
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmBitSet at remote 0x5555559cbe10>) = void
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmBitSet at remote 0x555555b109f0>) = void
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmBitSet at remote 0x555555af9ad0>) = void
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmBitSet at remote 0x5555559bb520>) = void
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmBitSet at remote 0x555555cc95d0>) = void
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmBitSet at remote 0x555555afff30>) = void
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmBitSet at remote 0x555555b02760>) = void
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmBitSet at remote 0x5555559664c0>) = void
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmBitSet at remote 0x555555b1b7c0>) = void
[pid 2010953] <... horizon_patched_dealloc resumed> = void
[pid 2010953] horizon_patch_dealloc(0x7ffff6990940 <NyHorizon_Type>) = 0x0
[pid 2010953] <... horizon_new resumed> = <guppy.heapy.heapyc.Horizon at remote 0x7ffff63430b0>
[pid 2010953] horizon_patched_dealloc((<guppy.heapy.heapyc.HeapView at remote 0x7ffff606c898>,)) = void
[pid 2010953] horizon_patched_dealloc(Frame 0x555555ae6898, for file /home/zhuyifei1999/guppy3/guppy/heapy/View.py, line 15, in __init__ (self=<Horizon(mod=<Interface(_share=<Share(module=<module at remote 0x7ffff69c89a8>, parent=<Share(module=<module at remote 0x7ffff6a578b8>, parent=<Share(module=<module at remote 0x7ffff6a577c8>, parent=<Share(module=None, parent=<...>, name='', setable=(), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': None, 'guppy': <...>, 'types': <Share(module=<module at remote 0x7ffff6ac24a8>, parent=<...>, name='types', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': <module at remote 0x7ffff6ac24a8>}, owners={}, ispackage=None, recursion=0, preload=(...)) at remote 0x7ffff5cddcf8>, 'sys': <Share(module=<module at remote 0x7ffff6c1eef8>, parent=<...>, name='sys', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={...(truncated)) = void
[...]
[pid 2010953] horizon_patched_dealloc(Frame 0x7ffff6246930, for file /home/zhuyifei1999/guppy3/guppy/heapy/View.py, line 19, in news (self=<Horizon(mod=<Interface(_share=<Share(module=<module at remote 0x7ffff69c89a8>, parent=<Share(module=<module at remote 0x7ffff6a578b8>, parent=<Share(module=<module at remote 0x7ffff6a577c8>, parent=<Share(module=None, parent=<...>, name='', setable=(), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': None, 'guppy': <...>, 'types': <Share(module=<module at remote 0x7ffff6ac24a8>, parent=<...>, name='types', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': <module at remote 0x7ffff6ac24a8>}, owners={}, ispackage=None, recursion=0, preload=(...)) at remote 0x7ffff5cddcf8>, 'sys': <Share(module=<module at remote 0x7ffff6c1eef8>, parent=<...>, name='sys', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_pa...(truncated)) = void
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.MutNodeSet at remote 0x7ffff5f37630>) = <unfinished ...>
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmBitSet at remote 0x7ffff696bb30>) = void
[pid 2010953] <... horizon_patched_dealloc resumed> = void
[pid 2010953] horizon_patched_dealloc(((), [])) = void
[pid 2010953] horizon_patched_dealloc(Frame 0x7ffff6122ac8, for file /home/zhuyifei1999/guppy3/guppy/heapy/ImpSet.py, line 20, in immnodeset (self=<Interface(_share=<Share(module=<module at remote 0x7ffff61371d8>, parent=<Share(module=<module at remote 0x7ffff6a578b8>, parent=<Share(module=<module at remote 0x7ffff6a577c8>, parent=<Share(module=None, parent=<...>, name='', setable=(), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': None, 'guppy': <...>, 'types': <Share(module=<module at remote 0x7ffff6ac24a8>, parent=<...>, name='types', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': <module at remote 0x7ffff6ac24a8>}, owners={}, ispackage=None, recursion=0, preload=(...)) at remote 0x7ffff5cddcf8>, 'sys': <Share(module=<module at remote 0x7ffff6c1eef8>, parent=<...>, name='sys', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent'...(truncated)) = void
[pid 2010953] horizon_patched_dealloc(Frame 0x7ffff61ab788, for file /home/zhuyifei1999/guppy3/guppy/heapy/UniSet.py, line 1516, in _cons (self=<IdentitySetFamily(mod=<Interface(_share=<Share(module=<module at remote 0x7ffff6137638>, parent=<Share(module=<module at remote 0x7ffff6a578b8>, parent=<Share(module=<module at remote 0x7ffff6a577c8>, parent=<Share(module=None, parent=<...>, name='', setable=(), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': None, 'guppy': <...>, 'types': <Share(module=<module at remote 0x7ffff6ac24a8>, parent=<...>, name='types', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': <module at remote 0x7ffff6ac24a8>}, owners={}, ispackage=None, recursion=0, preload=(...)) at remote 0x7ffff5cddcf8>, 'sys': <Share(module=<module at remote 0x7ffff6c1eef8>, parent=<...>, name='sys', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrappin...(truncated)) = <unfinished ...>
[pid 2010953] horizon_patched_dealloc(<guppy.sets.setsc.ImmNodeSet at remote 0x7ffff5f37f30>) = void
[pid 2010953] <... horizon_patched_dealloc resumed> = void
[pid 2010953] horizon_patched_dealloc(<method at remote 0x7ffff5c9f488>) = void
[pid 2010953] horizon_patched_dealloc(((),)) = void
[pid 2010953] horizon_patched_dealloc(Frame 0x7ffff611fcc0, for file /home/zhuyifei1999/guppy3/guppy/heapy/UniSet.py, line 1510, in __call__ (self=<IdentitySetFamily(mod=<Interface(_share=<Share(module=<module at remote 0x7ffff6137638>, parent=<Share(module=<module at remote 0x7ffff6a578b8>, parent=<Share(module=<module at remote 0x7ffff6a577c8>, parent=<Share(module=None, parent=<...>, name='', setable=(), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': None, 'guppy': <...>, 'types': <Share(module=<module at remote 0x7ffff6ac24a8>, parent=<...>, name='types', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': <module at remote 0x7ffff6ac24a8>}, owners={}, ispackage=None, recursion=0, preload=(...)) at remote 0x7ffff5cddcf8>, 'sys': <Share(module=<module at remote 0x7ffff6c1eef8>, parent=<...>, name='sys', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrap...(truncated)) = <unfinished ...>
[pid 2010953] horizon_patched_dealloc({}) = void
[pid 2010953] <... horizon_patched_dealloc resumed> = void
[pid 2010953] horizon_patched_dealloc(<method at remote 0x7ffff5c9fe48>) = void
[pid 2010953] horizon_patched_dealloc(<method at remote 0x7ffff5cfef08>) = void
[pid 2010953] horizon_patched_dealloc(Frame 0x555555ae1898, for file /home/zhuyifei1999/guppy3/guppy/heapy/UniSet.py, line 82, in __le__ (self=<IdentitySetMulti at remote 0x7ffff5ee7888>, other=<...>)) = void
[pid 2010953] horizon_patched_dealloc((<IdentitySetMulti at remote 0x7ffff5ee7888>,)) = void
[pid 2010953] horizon_patched_dealloc(<method at remote 0x7ffff5c9f488>) = void
[pid 2010953] horizon_patched_dealloc(Frame 0x7ffff5ff6d68, for file /home/zhuyifei1999/guppy3/guppy/heapy/UniSet.py, line 58, in __ge__ (self=<IdentitySetMulti at remote 0x7ffff5ee7888>, other=<...>)) = void
[pid 2010953] horizon_patched_dealloc((<IdentitySetMulti at remote 0x7ffff5ee7888>,)) = void
[pid 2010953] horizon_patched_dealloc(<method at remote 0x7ffff5c9f488>) = void
[pid 2010953] horizon_patched_dealloc(Frame 0x7ffff617d408, for file /home/zhuyifei1999/guppy3/guppy/heapy/UniSet.py, line 37, in __eq__ (self=<IdentitySetMulti at remote 0x7ffff5ee7888>, other=<...>)) = void
[pid 2010953] horizon_patched_dealloc((<IdentitySetMulti at remote 0x7ffff5ee7888>,)) = void
[pid 2010953] horizon_patched_dealloc(<method at remote 0x7ffff5c9fe48>) = void
[pid 2010953] horizon_patched_dealloc(Frame 0x555555899f78, for file /home/zhuyifei1999/guppy3/guppy/heapy/UniSet.py, line 149, in __ne__ (self=<IdentitySetMulti at remote 0x7ffff5ee7888>, other=<...>)) = void
[pid 2010953] horizon_patched_dealloc((<IdentitySetMulti at remote 0x7ffff5ee7888>,)) = void
[pid 2010953] horizon_patched_dealloc(<method at remote 0x7ffff5cfef08>) = void
[pid 2010953] horizon_patched_dealloc(Frame 0x555555885948, for file /home/zhuyifei1999/guppy3/guppy/heapy/test/support.py, line 103, in aseq (self=<FirstCase(_testMethodName='test_horizon', _outcome=<_Outcome(expecting_failure=False, result=<TextTestResult(failfast=False, failures=[], errors=[], testsRun=4, skipped=[], expectedFailures=[], unexpectedSuccesses=[], shouldStop=False, buffer=False, tb_locals=False, _stdout_buffer=None, _stderr_buffer=None, _original_stdout=<_io.TextIOWrapper at remote 0x7ffff6c40630>, _original_stderr=<_io.TextIOWrapper at remote 0x7ffff6c40708>, _mirrorOutput=False, stream=<_WritelnDecorator(stream=<_io.TextIOWrapper at remote 0x7ffff6c40630>) at remote 0x7ffff6909470>, showAll=True, dots=False, descriptions=True, _testRunEntered=True, _moduleSetUpFailed=False, _previousTestClass=<type at remote 0x5555559831f8>) at remote 0x7ffff5f93be0>, result_supports_subtests=True, success=True, skipped=[], expectedFailure=None, errors=[(<...>, None)]) at remote 0x7ffff5f50eb8>, _testMethodDoc=None, _cleanups=[], _subtest=None,...(truncated)) = void
[pid 2010953] horizon_patched_dealloc(Frame 0x55555590f5b8, for file /home/zhuyifei1999/guppy3/guppy/heapy/test/test_View.py, line 73, in test_horizon (self=<FirstCase(_testMethodName='test_horizon', _outcome=<_Outcome(expecting_failure=False, result=<TextTestResult(failfast=False, failures=[], errors=[], testsRun=4, skipped=[], expectedFailures=[], unexpectedSuccesses=[], shouldStop=False, buffer=False, tb_locals=False, _stdout_buffer=None, _stderr_buffer=None, _original_stdout=<_io.TextIOWrapper at remote 0x7ffff6c40630>, _original_stderr=<_io.TextIOWrapper at remote 0x7ffff6c40708>, _mirrorOutput=False, stream=<_WritelnDecorator(stream=<_io.TextIOWrapper at remote 0x7ffff6c40630>) at remote 0x7ffff6909470>, showAll=True, dots=False, descriptions=True, _testRunEntered=True, _moduleSetUpFailed=False, _previousTestClass=<type at remote 0x5555559831f8>) at remote 0x7ffff5f93be0>, result_supports_subtests=True, success=True, skipped=[], expectedFailure=None, errors=[(<...>, None)]) at remote 0x7ffff5f50eb8>, _testMethodDoc=None, _cleanups=[], _subt...(truncated)) = <unfinished ...>
[pid 2010953] horizon_patched_dealloc({'mod': <Interface(_share=<Share(module=<module at remote 0x7ffff69c89a8>, parent=<Share(module=<module at remote 0x7ffff6a578b8>, parent=<Share(module=<module at remote 0x7ffff6a577c8>, parent=<Share(module=None, parent=<...>, name='', setable=(), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': None, 'guppy': <...>, 'types': <Share(module=<module at remote 0x7ffff6ac24a8>, parent=<...>, name='types', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': <module at remote 0x7ffff6ac24a8>}, owners={}, ispackage=None, recursion=0, preload=(...)) at remote 0x7ffff5cddcf8>, 'sys': <Share(module=<module at remote 0x7ffff6c1eef8>, parent=<...>, name='sys', setable=(...), chgable=(...), importedfrom={}, nowrap=(...), wrapattr=None, wrapping=0, data={'_parent': <...>, '_root': <...>, '_module': <module at remote 0x7ffff6c1eef8>, 'exc_info': <built-in method e...(truncated)) = <unfinished ...>
[pid 2010953] horizon_patched_dealloc(<guppy.heapy.heapyc.Horizon at remote 0x7ffff63430b0>) = <unfinished ...>
[pid 2010953] horizon_remove(0x7ffff63430b0) = void
[pid 2010953] <... horizon_patched_dealloc resumed> = void
[pid 2010953] <... horizon_patched_dealloc resumed> = void
[pid 2010953] horizon_patched_dealloc(<Horizon at remote 0x7ffff5d08278>) =
Thread 1 "python" received signal SIGSEGV, Segmentation fault.
PyDict_GetItem (op=0x0, key=<type at remote 0x7ffff7d4f360>) at /usr/src/debug/dev-lang/python-3.6.9/Python-3.6.9/Objects/dictobject.c:1411
1411 if (!PyDict_Check(op))
gef➤ bt
#0 PyDict_GetItem (op=0x0, key=<type at remote 0x7ffff7d4f360>) at /usr/src/debug/dev-lang/python-3.6.9/Python-3.6.9/Objects/dictobject.c:1411
#1 0x00007ffff697c72f in horizon_get_org_dealloc (t=0x7ffff7d4f360 <PyBaseObject_Type>) at /home/zhuyifei1999/guppy3/src/heapy/horizon.c:38
#2 0x00007ffff697ca43 in horizon_patched_dealloc (v=<Horizon at remote 0x7ffff5d08278>) at /home/zhuyifei1999/guppy3/src/heapy/horizon.c:98
#3 0x00007ffff7c031a3 in subtype_dealloc (self=<Horizon at remote 0x7ffff5d08278>) at /usr/src/debug/dev-lang/python-3.6.9/Python-3.6.9/Objects/typeobject.c:1222
#4 0x00007ffff7c5041b in frame_dealloc (f=Frame 0x55555590f5b8, for file /home/zhuyifei1999/guppy3/guppy/heapy/test/test_View.py, line 73, in test_horizon (hn=<IdentitySetMulti at remote 0x7ffff5ee7888>)) at /usr/src/debug/dev-lang/python-3.6.9/Python-3.6.9/Objects/frameobject.c:462
#5 0x00007ffff697ca4c in horizon_patched_dealloc (v=Frame 0x55555590f5b8, for file /home/zhuyifei1999/guppy3/guppy/heapy/test/test_View.py, line 73, in test_horizon (hn=<IdentitySetMulti at remote 0x7ffff5ee7888>)) at /home/zhuyifei1999/guppy3/src/heapy/horizon.c:98
#6 0x00007ffff7b8115f in _PyFunction_FastCall (co=<optimized out>, args=<optimized out>, nargs=0x1, globals=<optimized out>) at /usr/src/debug/dev-lang/python-3.6.9/Python-3.6.9/Python/ceval.c:4936
#7 0x00007ffff7b82de2 in fast_function (func=<optimized out>, stack=<optimized out>, nargs=<optimized out>, kwnames=<optimized out>) at /usr/src/debug/dev-lang/python-3.6.9/Python-3.6.9/Python/ceval.c:4975
#8 0x00007ffff7b82ee5 in call_function (pp_stack=0x7fffffffb008, oparg=<optimized out>, kwnames=<optimized out>) at /usr/src/debug/dev-lang/python-3.6.9/Python-3.6.9/Python/ceval.c:4872
#9 0x00007ffff7b87ef4 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at /usr/src/debug/dev-lang/python-3.6.9/Python-3.6.9/Python/ceval.c:3335
#10 0x00007ffff7b82a41 in PyEval_EvalFrameEx (throwflag=0x0, f=Frame 0x5555558bab38, for file /usr/lib64/python3.6/unittest/case.py, line 605, in run (self=<FirstCase(_testMethodName='test_horizon', _outcome=<_Outcome(expecting_failure=False, result=<TextTestResult(failfast=False, failures=[], errors=[], testsRun=4, skipped=[], expectedFailures=[], unexpectedSuccesses=[], shouldStop=False, buffer=False, tb_locals=False, _stdout_buffer=None, _stderr_buffer=None, _original_stdout=<_io.TextIOWrapper at remote 0x7ffff6c40630>, _original_stderr=<_io.TextIOWrapper at remote 0x7ffff6c40708>, _mirrorOutput=False, stream=<_WritelnDecorator(stream=<_io.TextIOWrapper at remote 0x7ffff6c40630>) at remote 0x7ffff6909470>, showAll=True, dots=False, descriptions=True, _testRunEntered=True, _moduleSetUpFailed=False, _previousTestClass=<type at remote 0x5555559831f8>) at remote 0x7ffff5f93be0>, result_supports_subtests=True, success=True, skipped=[], expectedFailure=None, errors=[(<...>, None)]) at remote 0x7ffff5f50eb8>, _testMethodDoc=None, _cleanups=[], _subtest=None, _type_equality_f...(truncated)) at /usr/src/debug/dev-lang/python-3.6.9/Python-3.6.9/Python/ceval.c:754
[...]
gef➤ py-bt
Traceback (most recent call first):
File "/usr/lib64/python3.6/unittest/case.py", line 605, in run
testMethod()
File "/usr/lib64/python3.6/unittest/case.py", line 653, in __call__
return self.run(*args, **kwds)
File "/usr/lib64/python3.6/unittest/suite.py", line 122, in run
test(result)
File "/usr/lib64/python3.6/unittest/suite.py", line 84, in __call__
return self.run(*args, **kwds)
File "/usr/lib64/python3.6/unittest/runner.py", line 176, in run
test(result)
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/support.py", line 51, in run_suite
result = runner.run(suite)
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/support.py", line 73, in run_unittest
run_suite(suite, testclass)
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/test_View.py", line 275, in test_main
support.run_unittest(FirstCase, debug)
File "/home/zhuyifei1999/guppy3/guppy/heapy/test/test_all.py", line 36, in test_main
f(debug=debug)
File "/home/zhuyifei1999/guppy3/guppy/heapy/Use.py", line 363, in test
self._parent.test.test_all.test_main(debug)
File "test.py", line 8, in <module>
hpy().test()
How could this work?
Hi, I'm using python 3.10 in one of my projects and wanted to use the remote monitoring mode from guppy3. However, after adding
import guppy.heapy.RM
I get an error saying AttributeError: module 'guppy.heapy.heapyc' has no attribute 'interpreter'
.
After some digging, I found that the interpreter is not available for python version above 3.8 (found here)
Is there any specific reason for this ? In the Readme, it's specified that the package works with python 3.9, 3.10 and 3.11.
Any help here would be appreciated :)
Thank you for porting guppy to the latest and greatest Py3. I am wondering how to analyze guppy heap files that has been dump to files with some UI/console tools?
On aarch64, pip install guppy3 builds the wheels from source code and then install it. It requires user to have development environment installed on his system. also, it take some time to build the wheels than downloading and extracting the wheels from pypi.
On aarch64, pip install guppy3 should download the wheels from pypi
@zhuyifei1999 , Please let me know your interest on releasing aarch64 wheels. I can help in this.
Hi. I'm trying to execute ">>> x.shpaths" for some x, which results in the following stack trace.
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/OutputHandling.py", line 386, in reprfunc
return self.printer.get_str_of_top()
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/OutputHandling.py", line 272, in get_str_of_top
return self.get_str(self, True)
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/OutputHandling.py", line 339, in get_str
return self.mod.View.enter(lambda: f())
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/View.py", line 256, in enter
retval = func()
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/OutputHandling.py", line 339, in <lambda>
return self.mod.View.enter(lambda: f())
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/OutputHandling.py", line 307, in f
nxt = next(it)
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/OutputHandling.py", line 232, in lines_from
yield self.line_at(idx)
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/OutputHandling.py", line 212, in line_at
li = next(self.line_iter)
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/Path.py", line 368, in _oh_get_line_iter
for el in self:
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/Path.py", line 289, in __iter__
return self.iter()
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/Path.py", line 292, in iter
return PathsIter(self, start, stop)
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/Path.py", line 176, in __init__
self.reset(start)
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/Path.py", line 209, in reset
sr = self.mod.sortedrels(self.paths.IG, src)
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/Path.py", line 462, in sortedrels
t.sort(key=lambda x: x[0])
File "/home/pino/projects/krrez_9_venv/lib/python3.9/site-packages/guppy/heapy/Path.py", line 94, in __lt__
return self.r < other.r
TypeError: '<' not supported between instances of 'weakref' and 'weakref'
I am trying to understand the inner workings of guppy. I am curious how the following method in IdentitySetSingleton
actually works to return the actual Python object:
def _get_theone(self):
return self._node
Looking through the C extension code in hv.c, I see in hv_heap()
function that it calls hv_heap_rec()
which in turn calls NyNodeSet_setobj()
. In NyNodeSet_setobj()
, Py_INCREF()
is called to increase the reference count of each obj
. But in the code, I don't see where obj
is stored in a node?
Thank you for your explanation.
Example:
>>> from guppy import hpy
>>> h=hpy()
>>> x=h.idset(range(100)).byid
>>> print(repr(x.more))
Index Size % Cumulative % Value
10 14 1.0 154 11.0 11
11 14 1.0 168 12.0 12
12 14 1.0 182 13.0 13
13 14 1.0 196 14.0 14
14 14 1.0 210 15.0 15
15 14 1.0 224 16.0 16
16 14 1.0 238 17.0 17
17 14 1.0 252 18.0 18
18 14 1.0 266 19.0 19
19 14 1.0 280 20.0 20
<80 more rows. Type e.g. '_.more' to view.>
>>> print(str(x.more))
Set of 100 <int> objects. Total size = 1398 bytes.
Index Size % Cumulative % Value
0 14 1.0 14 1.0 1
1 14 1.0 28 2.0 2
2 14 1.0 42 3.0 3
3 14 1.0 56 4.0 4
4 14 1.0 70 5.0 5
5 14 1.0 84 6.0 6
6 14 1.0 98 7.0 7
7 14 1.0 112 8.0 8
8 14 1.0 126 9.0 9
9 14 1.0 140 10.0 10
<90 more rows. Type e.g. '_.more' to view.>
>>>
`
This bug was first reported at:
https://sourceforge.net/p/guppy-pe/bugs/20/
Can we do something about it? It is probably a stack overflow. It would be better if we could check for stack size and report an exception instead of trigging a segmentation fault.
#!/usr/bin/env python
from guppy import hpy
class node(object):
def __init__(self, value, _next=None):
self.value = value
self.next = _next
llist = node(0)
curnode = llist
for i in range(int(1e6)):
curnode.next = node(0)
curnode = curnode.next
heap = hpy().heap()
I'm doing:
from guppy import hpy
hpy().pb()
and I get:
Traceback (most recent call last):
File "", line 1, in
File "/home/rlippmann/git/homeassistant/core/venv/lib/python3.10/site-packages/guppy/heapy/Prof.py", line 2998, in pb
pa.new_profile_browser(filename)
File "/home/rlippmann/git/homeassistant/core/venv/lib/python3.10/site-packages/guppy/heapy/Prof.py", line 1670, in new_profile_browser
return ProfileBrowser(self, filename)
File "/home/rlippmann/git/homeassistant/core/venv/lib/python3.10/site-packages/guppy/heapy/Prof.py", line 2418, in init
self.display = Display(self.disptab,
File "/home/rlippmann/git/homeassistant/core/venv/lib/python3.10/site-packages/guppy/heapy/Prof.py", line 821, in init
self.var_xgrid = BooleanVar(xgrid)
File "/usr/lib/python3.10/tkinter/init.py", line 610, in init
Variable.init(self, master, value, name)
File "/usr/lib/python3.10/tkinter/init.py", line 372, in init
self._root = master._root()
AttributeError: 'bool' object has no attribute '_root'
I get a window to pop up, but I can't do anything.
Am I doing something wrong?
When doing the following we get an exception. It seems Python3 needs the next method for iterator not the "_ next _" method. When I define the next method in PathsIter in guppy/heapy/Path.py it works. I have tested in the Guppy3-3.0.3 release as well as latest trunk as of today 10 sep 2019.
Python 3.6.8 (default, Jan 14 2019, 11:02:34)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
from guppy import hpy
h=hpy()
hp=h.heap()
hp
Partition of a set of 34117 objects. Total size = 2248255 bytes.
Index Count % Size % Cumulative % Kind (class / dict of class)
0 9003 26 595755 26 595755 26 str
1 8437 25 303296 13 899051 40 tuple
2 4369 13 229084 10 1128135 50 bytes
3 2187 6 183708 8 1311843 58 types.CodeType
4 402 1 177108 8 1488951 66 type
5 2141 6 145588 6 1634539 73 function
6 402 1 120156 5 1754695 78 dict of type
7 93 0 82324 4 1837019 82 dict of module
8 189 1 70004 3 1907023 85 dict (no owner)
9 396 1 44352 2 1951375 87 set
<93 more rows. Type e.g. '_.more' to view.>
hp.sp
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.6/dist-packages/guppy3-3.0.3-py3.6-linux-i686.egg/guppy/heapy/Path.py", line 330, in repr
self.pp(output=f)
File "/usr/local/lib/python3.6/dist-packages/guppy3-3.0.3-py3.6-linux-i686.egg/guppy/heapy/Path.py", line 412, in pp
self.more(start, output=output)
File "/usr/local/lib/python3.6/dist-packages/guppy3-3.0.3-py3.6-linux-i686.egg/guppy/heapy/Path.py", line 258, in call
self.printiter(it, output)
File "/usr/local/lib/python3.6/dist-packages/guppy3-3.0.3-py3.6-linux-i686.egg/guppy/heapy/Path.py", line 433, in printiter
it.next().pp(output=output)
AttributeError: 'PathsIter' object has no attribute 'next'
This is my dump from Heapy:
Partition of a set of 1235525 objects. Total size = 153363236 bytes.
Index Count % Size % Cumulative % Kind (class / dict of class)
0 260111 21 34847420 23 34847420 23 str
1 82607 7 16193248 11 51040668 33 dict (no owner)
2 102994 8 14936792 10 65977460 43 bytes
3 189724 15 14334888 9 80312348 52 tuple
4 73463 6 9990968 7 90303316 59 function
5 48013 4 8660794 6 98964110 65 types.CodeType
6 64567 5 5171896 3 104136006 68 list
7 4888 0 4896088 3 109032094 71 type
8 2817 0 4368312 3 113400406 74 dict of module
9 4888 0 2434624 2 115835030 76 dict of type
<1421 more rows. Type e.g. '_.more' to view.>
Numbers like 153363236 bytes
are difficult to read. 153,363,236 bytes
would be easier.
Getting exception output after this:
from guppy import hpy
h=hpy().heap()
print(h)
This is the exception:
Traceback (most recent call last):
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\etc\Glue.py", line 216, in getattr2
x = self.data[name]
KeyError: 'hv'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\etc\Glue.py", line 281, in getattr3
x = getattr(Clamp, name)
AttributeError: type object '_GLUECLAMP_' has no attribute 'hv'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\juan0\Documents\GitHub\VMIwithDRL\VMIwithDRL\src\implementation\VMImodel.py", line 2, in <module>
from agent_model.training_agent import TrainingAgent
File "C:\Users\juan0\Documents\GitHub\VMIwithDRL\VMIwithDRL\src\agent_model\training_agent.py", line 8, in <module>
h=hpy().heap()
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\heapy\Use.py", line 181, in heap
h = self.View.heap()
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\heapy\View.py", line 353, in heap
self.root = objs
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\etc\Glue.py", line 53, in __setattr__
return self._share.setattr(self, name, value)
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\etc\Glue.py", line 371, in setattr
im(inter, value)
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\heapy\View.py", line 191, in _set_root
self.hv.root = root
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\etc\Glue.py", line 50, in __getattr__
return self._share.getattr(self, name)
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\etc\Glue.py", line 209, in getattr
d = self.getattr2(inter, dct, owner, name)
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\etc\Glue.py", line 227, in getattr2
x = self.getattr3(inter, name)
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\etc\Glue.py", line 321, in getattr3
x = f()
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\heapy\View.py", line 162, in _get_hv
is_hiding_calling_interpreter=self.is_hiding_calling_interpreter)
File "C:\Users\juan0\Anaconda3\envs\tensor-cpu\lib\site-packages\guppy\heapy\View.py", line 394, in new_hv
hv = self.heapyc.HeapView(root, heapdefs)
TypeError: heapdefs must be a capsule object
guppy3==3.1.0
Using Windows 7. I ran this:
h = guppy.hpy()
he = h.heap()
It waited for a few seconds and then the process crashed. Here are the error details from the dialog:
Problem signature:
Problem Event Name: APPCRASH
Application Name: python.exe
Application Version: 3.8.1150.1013
Application Timestamp: 5dfab277
Fault Module Name: python38.dll
Fault Module Version: 3.8.1150.1013
Fault Module Timestamp: 5dfab24b
Exception Code: c0000005
Exception Offset: 000000000002feaf
OS Version: 6.1.7601.2.1.0.256.1
Locale ID: 1033
Additional Information 1: e8ad
Additional Information 2: e8adaaf2e9f8209565e4a118f3c1cb38
Additional Information 3: 359a
Additional Information 4: 359a43a120d45186052c5a12b0526d2d
Hello @zhuyifei1999,
I wanted to learn if I can use guppy to monitor an external process? My use case is that a python process is already running. I want to monitor the heap size over hours/days. It is a daemon, a long-running process, and based on how to use the guppy3
, it seems I need to be inside that process to query the heap, this is not desirable for my use case.
Could you help to share your thoughts with me?
Thank you
Hi,
I got this question after checking out #24 .
In fact, I have some memory issues with my Django server, but the online environment is not quite reachable for some reason.
Therefore, I wonder whether guppy3 provides any approaches to dump the memory/heap snapshot so that I can retrieve the snapshot via an API and do the rest of the analysis offline.
Thanks.
This occured in commit 6373998 which was apparently after V3.0.6 was released.
In hv_cli_dictof_update_new_method in src/hv_cli_dictof.c, the call to hv_get_objects was replaced with a call to gc_get_objects. This was to be faster because of an easier way to find objects and they were also fewer. However, we didn't get non-gc-collected objects such as simple dicts with non-gc-collected members. This made the hv_cli_dictof_update method be called each time we tried to classify such dicts, from hv_cli_dictof_classify, because we didn't store any information about those dicts in the nodegraph self->owners. The classifier thought it needed more information each time.
Code classifying an idset() of a list of 5000 empty dicts went from about 0.3 seconds (with the change reverted) to 9 seconds in Python 3 with the modification mentioned.
# dictofno.py
from guppy import hpy
h=hpy()
d=[{} for i in range(5000)]
print (h.idset(d))
Checking in the debugger revealed that the update function was called many times.
It has been suggested (by the author of Guppy 3, YiFei Zhu) that the hv_cli_dictof_classify function should call the update method only once.
I presume this means only once during the lifetime of the classifier. I think this would work if the classifier is allocated together with the set it classifies, because those sets are immutable so the classification would not change. I think this is the current situation.
However, if we would turn to have memoized classifiers in the Heapy guppy.hpy() structure, (to speed things up) then there might be the case that new dicts are created that should require to update the dict ownership graph.
Then it would require many calls to update during the lifetime of the classifier. I don't know if this will be a reality but it may be something to consider.
I'm about to do a release, and this happened:
$ python setup.py sdist
running sdist
running egg_info
writing guppy3.egg-info/PKG-INFO
writing dependency_links to guppy3.egg-info/dependency_links.txt
writing top-level names to guppy3.egg-info/top_level.txt
reading manifest file 'guppy3.egg-info/SOURCES.txt'
writing manifest file 'guppy3.egg-info/SOURCES.txt'
running check
creating guppy3-3.0.6
creating guppy3-3.0.6/guppy
creating guppy3-3.0.6/guppy/etc
creating guppy3-3.0.6/guppy/gsl
creating guppy3-3.0.6/guppy/heapy
creating guppy3-3.0.6/guppy/heapy/test
creating guppy3-3.0.6/guppy/sets
creating guppy3-3.0.6/guppy3.egg-info
creating guppy3-3.0.6/src
creating guppy3-3.0.6/src/heapy
creating guppy3-3.0.6/src/sets
copying files to guppy3-3.0.6...
copying README.md -> guppy3-3.0.6
copying setup.py -> guppy3-3.0.6
copying guppy/__init__.py -> guppy3-3.0.6/guppy
copying guppy/etc/Cat.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/Code.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/Descriptor.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/FSA.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/Glue.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/Help.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/IterPermute.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/KanExtension.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/KnuthBendix.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/RE.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/RE_Rect.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/__init__.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/cmd.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/etc.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/textView.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/tkcursors.py -> guppy3-3.0.6/guppy/etc
copying guppy/etc/xterm.py -> guppy3-3.0.6/guppy/etc
copying guppy/gsl/Document.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/DottedTree.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/Exceptions.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/FileIO.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/Filer.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/Gsml.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/Help.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/Html.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/Latex.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/Main.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/SpecNodes.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/Tester.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/Text.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/XHTML.py -> guppy3-3.0.6/guppy/gsl
copying guppy/gsl/__init__.py -> guppy3-3.0.6/guppy/gsl
copying guppy/heapy/AbstractAlgebra.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Classifiers.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Console.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Doc.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/ImpSet.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Monitor.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/OutputHandling.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Part.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Path.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Prof.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/RM.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/RefPat.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Remote.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/RemoteConstants.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Spec.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Target.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/UniSet.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/Use.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/View.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/__init__.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/pbhelp.py -> guppy3-3.0.6/guppy/heapy
copying guppy/heapy/test/__init__.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/support.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_Classifiers.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_ER.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_OutputHandling.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_Part.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_Path.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_RefPat.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_RetaGraph.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_Spec.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_UniSet.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_View.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_all.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_dependencies.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_gsl.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_heapyc.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_menuleak.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/heapy/test/test_sf.py -> guppy3-3.0.6/guppy/heapy/test
copying guppy/sets/__init__.py -> guppy3-3.0.6/guppy/sets
copying guppy/sets/test.py -> guppy3-3.0.6/guppy/sets
copying guppy3.egg-info/PKG-INFO -> guppy3-3.0.6/guppy3.egg-info
copying guppy3.egg-info/SOURCES.txt -> guppy3-3.0.6/guppy3.egg-info
copying guppy3.egg-info/dependency_links.txt -> guppy3-3.0.6/guppy3.egg-info
copying guppy3.egg-info/top_level.txt -> guppy3-3.0.6/guppy3.egg-info
copying src/heapy/heapyc.c -> guppy3-3.0.6/src/heapy
copying src/heapy/stdtypes.c -> guppy3-3.0.6/src/heapy
copying src/sets/bitset.c -> guppy3-3.0.6/src/sets
copying src/sets/nodeset.c -> guppy3-3.0.6/src/sets
copying src/sets/sets.c -> guppy3-3.0.6/src/sets
Writing guppy3-3.0.6/setup.cfg
Creating tar archive
removing 'guppy3-3.0.6' (and everything under it)
A lot of files under src/ is missing.
Failure runs:
Seems to be the refpat sometimes terminates prematurely with [R]
,
Attempts to reproduce by a Windows development machine fails.
Attempts to reproduce by ssh-ing to GitHub Actions fails, and the test passes post-ssh: https://github.com/zhuyifei1999/guppy3/actions/runs/1663126825
Hi,
I pulled the last checkin after v3.0.6 from Master, installed it and tested it. Unfortunately, it seems to be something wrong with how the test skips the test in main() of SpecNodes.py.
There is a check for file not found in SpecNodes.py:
if not root.os.path.exists(main_dt_name):
print('%s does not exist, skipping test' % main_dt_name)
However, it seems to be missing a return statement there. I get:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.6/dist-packages/guppy3-3.0.6-py3.6-linux-i686.egg/guppy/gsl/../../specs/docexample.gsl'
I put a return after the print above and then the test was skipped and it could continue with the other tests.
As a side note, would it be a bad idea to install the specs/*.gsl files?
I see that I am also not installing those files in Guppy-PE/0.1.11.
But maybe we could consider installing them?
Regards
Sverker
Cannot get this package to build on Python 3.11:
Upstream: home-assistant/core#74650
Build output:
Building wheel for guppy3 (setup.py): started
Building wheel for guppy3 (setup.py): finished with status 'error'
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [96 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/guppy
copying guppy/_version.py -> build/lib.linux-x86_64-cpython-311/guppy
copying guppy/__init__.py -> build/lib.linux-x86_64-cpython-311/guppy
creating build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/etc.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/xterm.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/IterPermute.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/Cat.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
Running setup.py clean for guppy3
copying guppy/etc/RE.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/KanExtension.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/textView.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/Glue.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/FSA.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/RE_Rect.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/Help.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/KnuthBendix.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/Code.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/Descriptor.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/cmd.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/__init__.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
copying guppy/etc/tkcursors.py -> build/lib.linux-x86_64-cpython-311/guppy/etc
creating build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/Exceptions.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/Document.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/Main.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/Latex.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/Html.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/SpecNodes.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/FileIO.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/Filer.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/Tester.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/Help.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/DottedTree.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/Text.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/XHTML.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/Gsml.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
copying guppy/gsl/__init__.py -> build/lib.linux-x86_64-cpython-311/guppy/gsl
creating build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Part.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/pbhelp.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Classifiers.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/RefPat.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/UniSet.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Spec.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Doc.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Use.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Console.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Remote.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Monitor.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Prof.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/OutputHandling.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/RM.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/View.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Path.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/RemoteConstants.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/ImpSet.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/Target.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
copying guppy/heapy/__init__.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy
creating build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_menuleak.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_Classifiers.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_all.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_View.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_OutputHandling.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_gsl.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_RetaGraph.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_heapyc.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_ER.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_Path.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_RefPat.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_Spec.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_dependencies.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_UniSet.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_sf.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/test_Part.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/support.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
copying guppy/heapy/test/__init__.py -> build/lib.linux-x86_64-cpython-311/guppy/heapy/test
creating build/lib.linux-x86_64-cpython-311/guppy/sets
copying guppy/sets/test.py -> build/lib.linux-x86_64-cpython-311/guppy/sets
copying guppy/sets/__init__.py -> build/lib.linux-x86_64-cpython-311/guppy/sets
running build_ext
building 'guppy.sets.setsc' extension
creating build/temp.linux-x86_64-cpython-311
creating build/temp.linux-x86_64-cpython-311/src
creating build/temp.linux-x86_64-cpython-311/src/sets
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/runner/work/core/core/venv/include -I/opt/hostedtoolcache/Python/3.11.0-rc.2/x64/include/python3.11 -c src/sets/bitset.c -o build/temp.linux-x86_64-cpython-311/src/sets/bitset.o
src/sets/bitset.c:7:10: fatal error: longintrepr.h: No such file or directory
7 | #include "longintrepr.h"
| ^~~~~~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
Ref: cython/cython#4461
Hi
When trying to print heap, I got an AttributeError. I tweaked the code a bit in UniSet.py to report the types of self and a as well as b:
File "/home/r_pang_apple_com/.local/lib/python3.8/site-packages/guppy/heapy/UniSet.py", line 336, in getitem
def getitem(self, idx): return self.fam.c_getitem(self, idx)
File "/home/r_pang_apple_com/.local/lib/python3.8/site-packages/guppy/heapy/UniSet.py", line 1613, in c_getitem
return a.partition.get_set(idx)
File "/home/r_pang_apple_com/.local/lib/python3.8/site-packages/guppy/heapy/UniSet.py", line 74, in getattr
return self.fam.mod.View.enter(lambda: self.fam.c_getattr(self, other))
File "/home/r_pang_apple_com/.local/lib/python3.8/site-packages/guppy/heapy/View.py", line 256, in enter
retval = func()
File "/home/r_pang_apple_com/.local/lib/python3.8/site-packages/guppy/heapy/UniSet.py", line 74, in
return self.fam.mod.View.enter(lambda: self.fam.c_getattr(self, other))
File "/home/r_pang_apple_com/.local/lib/python3.8/site-packages/guppy/heapy/UniSet.py", line 800, in c_getattr
return self.c_getattr2(a, b)
File "/home/r_pang_apple_com/.local/lib/python3.8/site-packages/guppy/heapy/UniSet.py", line 803, in c_getattr2
raise AttributeError(f"{type(self)}.{type(a)}.{b}")
AttributeError: <class 'guppy.heapy.UniSet.IdentitySetFamily'>.<class 'guppy.heapy.UniSet.IdentitySetMulti'>.partition
Is this expected?
Thanks.
Hi! Thanks for this tool. I'm trying to figure out why some Django Gunicorn processes are so large.
I'd like to either log the heap or even possibly make the heap into a view in Django so I can see the heap -- but I can't figure out how to get a textual representation of the output instead of the interactive one.
Is this possible?
hpy() raises an AttributeError when coverage is running code that uses hpy, because the CoverageDebugFile module has no dict attribute:
.tox/py37/lib/python3.7/site-packages/guppy/__init__.py:36: in hpy
return r.guppy.heapy.Use
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:50: in __getattr__
return self._share.getattr(self, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:218: in getattr
d = self.getattr2(inter, cache, owner, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:235: in getattr2
x = self.getattr_package(inter, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:283: in getattr_package
x = self.makeModule(x, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:349: in makeModule
return Share(module, self, module.__name__, Clamp)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:207: in __init__
getattr(inter, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:50: in __getattr__
return self._share.getattr(self, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:218: in getattr
d = self.getattr2(inter, cache, owner, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:237: in getattr2
x = self.getattr3(inter, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:322: in getattr3
pa = getattr(pa, at)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:50: in __getattr__
return self._share.getattr(self, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:218: in getattr
d = self.getattr2(inter, cache, owner, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:235: in getattr2
x = self.getattr_package(inter, name)
.tox/py37/lib/python3.7/site-packages/guppy/etc/Glue.py:272: in getattr_package
x = __import__(self.makeName(name))
.tox/py37/lib/python3.7/site-packages/guppy/heapy/View.py:571: in <module>
prime_builtin_types()
.tox/py37/lib/python3.7/site-packages/guppy/heapy/View.py:551: in prime_builtin_types
for t in list(mod.__dict__.values()):
E AttributeError: 'tuple' object has no attribute '__dict__'
The offending module is a tuple:
(<coverage.debug.DebugOutputFile object at 0x7fec9fb98668>, False)
which has no dict attribute.
Would you accept a PR to look for the dict attribute in View.py and skip anything that doesn't have that attribute?
Thanks
Suppose we have the following file structure:
files/
a.py
b.py
main.py
And each file contains codes like:
# a.py
class A():
pass
# b.py
from .a import A
class B():
self.some_attrs = A()
# main.py
from files import B
obj = B()
Is this possible to replace the imported class A with another class using guppy3? And then when execute obj = B()
, the new class will be constructed and attached to self.some_attrs
This assignment that will be a check is dubious. It is the same for the two cases. Suggest to fix and also have a test case that catches it, if possible. Does the code in nodetuple_richcompare
work as intended? I don't understand it after these years.
hv_cli_and.c, lines 235-236
vi = (Py_ssize_t)vt->ob_item[i];
**wi = (Py_ssize_t)vt->ob_item[i];**
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.