[Question] How does `theone` return the Python object? about guppy3 HOT 8 CLOSED

I am aware that nodes[1] is just a pointer that points to the second element of nodes, not an array of size 1

Don't understand your question. I'm referring to the declaration PyObject *nodes[1]; being an array of indefinite size. The literal declaration of this is "nodes is an array of 1 element, of a pointer to PyObject". What I'm saying is, rather than that, it is "nodes is an array of indefinite size, of pointers to PyObject".

In other words, the assumption "NyNodeSetObject holds the pointer to nodes[1]" is incorrect.

This might help you: https://cdecl.org/?q=int+*nodes%5B1%5D%3B

Edit: I resized I was skipping too fast in my thoughts in my answer above. Sorry,

from guppy3.

My bad! It all makes sense! I need to brush up on my C syntax! Thank you!

from guppy3.

Hmm... so the things such as NyNodeSet is just, think of it like a python set, except instead of equality being governed by the object's defined __eq__, it's equality based on the objects identity (a.k.a. memory address). Almost everything else from the API standpoint should be similar to that of a python set. You can see the test cases: https://github.com/zhuyifei1999/guppy3/blob/master/guppy/sets/test.py#L1327

Because nodeset contains objects, it has to keep the objects alive. This is where Py_INCREF you mentioned comes in.

Things like IdentitySet is more of a Python wrapper around the C NyNodeSet that provides the lots of methods and printers and stuffs to better interact with and visualize the sets.

Since you're looking at hv_heap, I assume you want to see all the way from how nodesets gets created, so let's assume somehow the heap contains only one object, so you did hpy().heap().theone:

Use.heap():

View.heap():

View.enter is for limiting the visibility of guppy-internal frames during reference traversal, so the important part is self.idset(self.hv.heap())

hv.heap() is your mentioned hv_heap:

The important thing here is that it creates and returns a mutnodeset (a mutable nodeset) that contains everything it saw during the traversal.

Then View.idset() is an import / alias:

Use.idset():

UniSet.idset():

Here immnodeset first creates a copy of the mutnodeset, but now it's immutable, then it does IdentitySetFamily._cons():

If the size of the set is 1, then it gets the single element of the set via tuple(arg)[0], and calls the constructor of IdentitySetSingleton with it

And IdentitySetSingleton's constructor sets self._node with the given element:

So tl;dr: in hpy().heap().theone, heap() calls hv_heap() to create a mutnodeset, and then IdentitySetFamily._cons() sees that it's a set of size 1 and calls IdentitySetSingleton with the single element it contains, and the constructor sets the _node attribute.

Does this answer how it works?

from guppy3.

Hi @zhuyifei1999,

Thank you for your thorough explanation. This is definitely very helpful! I was able to follow the exact control flow when debugging with PyCharm.

Because nodeset contains objects, it has to keep the objects alive. This is where Py_INCREF you mentioned comes in.

This makes sense! To confirm: when adding an object by calling NyNodeSet_setobj():

When retrieving the object from theone, I am assuming this method:

Thank you!

from guppy3.

where it adds the object's adjusted address to the bitset (or nodes, depending on whether the set is immutable or not) within the NyNodeSetObject struct:

Immutable one cannot be added. NyMutNodeSet_Check(v) fails and you get ValueError: mutable nodeset required.

When retrieving the object from theone, I am assuming this method:
somehow calls some C code to get the object from the address? I am sorry, this is where I get a bit fuzzy. I presume the following C function is triggered by some function(s) when self._node in the code above is called?

No, custom C code is completely uninvolved here.

This corresponds to

In __init__:

self._node = node

In _get_theone:

return self._node 

How __init__ was called with the element / node was explained above. The argument to __init__ is the object returned by theone. Nothing fancy involved.

from guppy3.

OK. I think I see what is going on here. I believe, correct me if I am wrong, that the "trick" here is:

I guess my last question is, what is stored in nodes[0] since the NyNodeSetObject holds the pointer to nodes[1] instead of simply nodes?

Thanks!

from guppy3.

I guess my last question is, what is stored in nodes[0] since the NyNodeSetObject holds the pointer to nodes[1] instead of simply nodes?

This you can answer with GDB:

$ find build guppy -name '*.so' -delete; pip install --global-option build --global-option --debug -e .
Obtaining file:///home/zhuyifei1999/guppy3
Installing collected packages: guppy3
  Attempting uninstall: guppy3
    Found existing installation: guppy3 3.1.0
    Uninstalling guppy3-3.1.0:
      Successfully uninstalled guppy3-3.1.0
  Running setup.py develop for guppy3
Successfully installed guppy3-3.1.0
$ gdb python
GNU gdb (Gentoo 10.2 vanilla) 10.2
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
[...]
(gdb) r
Starting program: /home/zhuyifei1999/guppy3/venv/bin/python 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Python 3.9.5 (default, Jun  2 2021, 13:52:24) 
[GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import guppy.sets.setsc
>>> def a():
...  b = object()
...  print(b)
...  c = guppy.sets.setsc.ImmNodeSet([b])
...  while True: pass
... 
>>> a()
<object object at 0x7ffff6bec3a0>
^C
Program received signal SIGINT, Interrupt.
0x00007ffff7c2d1cc in _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at /usr/src/debug/dev-lang/python-3.9.5_p2/Python-3.9.5/Python/ceval.c:3256
3256	            DISPATCH();
(gdb) py-locals
b = <object at remote 0x7ffff6bec3a0>
c = <guppy.sets.setsc.ImmNodeSet at remote 0x7ffff6ad8ae0>
(gdb) p *(NyNodeSetObject *)0x7ffff6ad8ae0
$1 = {
  ob_base = {
    ob_base = {
      ob_refcnt = 0x1,
      ob_type = 0x7ffff6abd5c0 <NyImmNodeSet_Type>
    },
    ob_size = 0x1
  },
  flags = 0x1,
  _hiding_tag_ = 0x0,
  u = {
    bitset = <object at remote 0x7ffff6bec3a0>,
    nodes = {<object at remote 0x7ffff6bec3a0>}
  }
}
(gdb) 

The nodes[1] doesn't really mean it's an array of size 1, it's a C array of indefinite size. The array contains whatever objects the nodeset have. This is similar to how CPython implements tuples:
https://github.com/python/cpython/blob/769d7d0c66c5b86e2dd29b9ce67ac2daaab1bb38/Include/cpython/tupleobject.h#L5-L11

typedef struct {
    PyObject_VAR_HEAD
    /* ob_item contains space for 'ob_size' elements.
       Items must normally not be NULL, except during construction when
       the tuple is not yet visible outside the function that builds it. */
    PyObject *ob_item[1];
} PyTupleObject;

from guppy3.

I am not sure if you actually answered my question. I am aware that nodes[1] is just a pointer that points to the second element of nodes, not an array of size 1. But why does it skip the first element? Why does bitset not do the same (i.e., bitset[1])?

from guppy3.