Giter Club home page Giter Club logo

acquisition's Introduction

Environmental Acquisiton

This package implements "environmental acquisiton" for Python, as proposed in the OOPSLA96 paper by Joseph Gil and David H. Lorenz:

We propose a new programming paradigm, environmental acquisition in the context of object aggregation, in which objects acquire behaviour from their current containers at runtime. The key idea is that the behaviour of a component may depend upon its enclosing composite(s). In particular, we propose a form of feature sharing in which an object "inherits" features from the classes of objects in its environment. By examining the declaration of classes, it is possible to determine which kinds of classes may contain a component, and which components must be contained in a given kind of composite. These relationships are the basis for language constructs that supports acquisition.

Introductory Example

Zope implements acquisition with "Extension Class" mix-in classes. To use acquisition your classes must inherit from an acquisition base class. For example:

>>> import ExtensionClass, Acquisition

>>> class C(ExtensionClass.Base):
...     color = 'red'

>>> class A(Acquisition.Implicit):
...     def report(self):
...         print(self.color)
...
>>> a = A()
>>> c = C()
>>> c.a = a

>>> c.a.report()
red

>>> d = C()
>>> d.color = 'green'
>>> d.a = a

>>> d.a.report()
green

>>> try:
...     a.report()
... except AttributeError:
...     pass
... else:
...     raise AssertionError('AttributeError not raised.')

The class A inherits acquisition behavior from Acquisition.Implicit. The object, a, "has" the color of objects c and d when it is accessed through them, but it has no color by itself. The object a obtains attributes from its environment, where its environment is defined by the access path used to reach a.

Acquisition Wrappers

When an object that supports acquisition is accessed through an extension class instance, a special object, called an acquisition wrapper, is returned. In the example above, the expression c.a returns an acquisition wrapper that contains references to both c and a. It is this wrapper that performs attribute lookup in c when an attribute cannot be found in a.

Acquisition wrappers provide access to the wrapped objects through the attributes aq_parent, aq_self, aq_base. Continue the example from above:

>>> c.a.aq_parent is c
True
>>> c.a.aq_self is a
True

Explicit and Implicit Acquisition

Two styles of acquisition are supported: implicit and explicit acquisition.

Implicit acquisition

Implicit acquisition is so named because it searches for attributes from the environment automatically whenever an attribute cannot be obtained directly from an object or through inheritance.

An attribute can be implicitly acquired if its name does not begin with an underscore.

To support implicit acquisition, your class should inherit from the mix-in class Acquisition.Implicit.

Explicit Acquisition

When explicit acquisition is used, attributes are not automatically obtained from the environment. Instead, the method aq_acquire must be used. For example:

>>> print(c.a.aq_acquire('color'))
red

To support explicit acquisition, your class should inherit from the mix-in class Acquisition.Explicit.

Controlling Acquisition

A class (or instance) can provide attribute by attribute control over acquisition. You should subclass from Acquisition.Explicit, and set all attributes that should be acquired to the special value Acquisition.Acquired. Setting an attribute to this value also allows inherited attributes to be overridden with acquired ones. For example:

>>> class C(Acquisition.Explicit):
...     id = 1
...     secret = 2
...     color = Acquisition.Acquired
...     __roles__ = Acquisition.Acquired

The only attributes that are automatically acquired from containing objects are color, and __roles__. Note that the __roles__ attribute is acquired even though its name begins with an underscore. In fact, the special Acquisition.Acquired value can be used in Acquisition.Implicit objects to implicitly acquire selected objects that smell like private objects.

Sometimes, you want to dynamically make an implicitly acquiring object acquire explicitly. You can do this by getting the object's aq_explicit attribute. This attribute provides the object with an explicit wrapper that replaces the original implicit wrapper.

Filtered Acquisition

The acquisition method, aq_acquire, accepts two optional arguments. The first of the additional arguments is a "filtering" function that is used when considering whether to acquire an object. The second of the additional arguments is an object that is passed as extra data when calling the filtering function and which defaults to None. The filter function is called with five arguments:

  • The object that the aq_acquire method was called on,
  • The object where an object was found,
  • The name of the object, as passed to aq_acquire,
  • The object found, and
  • The extra data passed to aq_acquire.

If the filter returns a true object that the object found is returned, otherwise, the acquisition search continues.

Here's an example:

>>> from Acquisition import Explicit

>>> class HandyForTesting(object):
...     def __init__(self, name):
...         self.name = name
...     def __str__(self):
...         return "%s(%s)" % (self.name, self.__class__.__name__)
...     __repr__=__str__
...
>>> class E(Explicit, HandyForTesting): pass
...
>>> class Nice(HandyForTesting):
...     isNice = 1
...     def __str__(self):
...         return HandyForTesting.__str__(self)+' and I am nice!'
...     __repr__ = __str__
...
>>> a = E('a')
>>> a.b = E('b')
>>> a.b.c = E('c')
>>> a.p = Nice('spam')
>>> a.b.p = E('p')

>>> def find_nice(self, ancestor, name, object, extra):
...     return hasattr(object,'isNice') and object.isNice

>>> print(a.b.c.aq_acquire('p', find_nice))
spam(Nice) and I am nice!

The filtered acquisition in the last line skips over the first attribute it finds with the name p, because the attribute doesn't satisfy the condition given in the filter.

Filtered acquisition is rarely used in Zope.

Acquiring from Context

Normally acquisition allows objects to acquire data from their containers. However an object can acquire from objects that aren't its containers.

Most of the examples we've seen so far show establishing of an acquisition context using getattr semantics. For example, a.b is a reference to b in the context of a.

You can also manually set acquisition context using the __of__ method. For example:

>>> from Acquisition import Implicit
>>> class C(Implicit): pass
...
>>> a = C()
>>> b = C()
>>> a.color = "red"
>>> print(b.__of__(a).color)
red

In this case, a does not contain b, but it is put in b's context using the __of__ method.

Here's another subtler example that shows how you can construct an acquisition context that includes non-container objects:

>>> from Acquisition import Implicit

>>> class C(Implicit):
...     def __init__(self, name):
...         self.name = name

>>> a = C("a")
>>> a.b = C("b")
>>> a.b.color = "red"
>>> a.x = C("x")

>>> print(a.b.x.color)
red

Even though b does not contain x, x can acquire the color attribute from b. This works because in this case, x is accessed in the context of b even though it is not contained by b.

Here acquisition context is defined by the objects used to access another object.

Containment Before Context

If in the example above suppose both a and b have an color attribute:

>>> a = C("a")
>>> a.color = "green"
>>> a.b = C("b")
>>> a.b.color = "red"
>>> a.x = C("x")

>>> print(a.b.x.color)
green

Why does a.b.x.color acquire color from a and not from b? The answer is that an object acquires from its containers before non-containers in its context.

To see why consider this example in terms of expressions using the __of__ method:

a.x -> x.__of__(a)

a.b -> b.__of__(a)

a.b.x -> x.__of__(a).__of__(b.__of__(a))

Keep in mind that attribute lookup in a wrapper is done by trying to look up the attribute in the wrapped object first and then in the parent object. So in the expressions above proceeds from left to right.

The upshot of these rules is that attributes are looked up by containment before context.

This rule holds true also for more complex examples. For example, a.b.c.d.e.f.g.attribute would search for attribute in g and all its containers first. (Containers are searched in order from the innermost parent to the outermost container.) If the attribute is not found in g or any of its containers, then the search moves to f and all its containers, and so on.

Additional Attributes and Methods

You can use the special method aq_inner to access an object wrapped only by containment. So in the example above, a.b.x.aq_inner is equivalent to a.x.

You can find out the acquisition context of an object using the aq_chain method like so:

>>> [obj.name for obj in a.b.x.aq_chain] ['x', 'b', 'a']

You can find out if an object is in the containment context of another object using the aq_inContextOf method. For example:

>>> a.b.aq_inContextOf(a) True

Acquisition Module Functions

In addition to using acquisition attributes and methods directly on objects you can use similar functions defined in the Acquisition module. These functions have the advantage that you don't need to check to make sure that the object has the method or attribute before calling it.

aq_acquire(object, name [, filter, extra, explicit, default, containment])

Acquires an object with the given name.

This function can be used to explictly acquire when using explicit acquisition and to acquire names that wouldn't normally be acquired.

The function accepts a number of optional arguments:

filter

A callable filter object that is used to decide if an object should be acquired.

The filter is called with five arguments:

  • The object that the aq_acquire method was called on,
  • The object where an object was found,
  • The name of the object, as passed to aq_acquire,
  • The object found, and
  • The extra argument passed to aq_acquire.

If the filter returns a true object that the object found is returned, otherwise, the acquisition search continues.

extra

Extra data to be passed as the last argument to the filter.

explicit

A flag (boolean value) indicating whether explicit acquisition should be used. The default value is true. If the flag is true, then acquisition will proceed regardless of whether wrappers encountered in the search of the acquisition hierarchy are explicit or implicit wrappers. If the flag is false, then parents of explicit wrappers are not searched.

This argument is useful if you want to apply a filter without overriding explicit wrappers.

default

A default value to return if no value can be acquired.

containment

A flag indicating whether the search should be limited to the containment hierarchy.

In addition, arguments can be provided as keywords.

aq_base(object)

Return the object with all wrapping removed.

aq_chain(object [, containment])

Return a list containing the object and it's acquisition parents. The optional argument, containment, controls whether the containment or access hierarchy is used.

aq_get(object, name [, default, containment])

Acquire an attribute, name. A default value can be provided, as can a flag that limits search to the containment hierarchy.

aq_inner(object)

Return the object with all but the innermost layer of wrapping removed.

aq_parent(object)

Return the acquisition parent of the object or None if the object is unwrapped.

aq_self(object)

Return the object with one layer of wrapping removed, unless the object is unwrapped, in which case the object is returned.

In most cases it is more convenient to use these module functions instead of the acquisition attributes and methods directly.

Acquisition and Methods

Python methods of objects that support acquisition can use acquired attributes. When a Python method is called on an object that is wrapped by an acquisition wrapper, the wrapper is passed to the method as the first argument. This rule also applies to user-defined method types and to C methods defined in pure mix-in classes.

Unfortunately, C methods defined in extension base classes that define their own data structures, cannot use aquired attributes at this time. This is because wrapper objects do not conform to the data structures expected by these methods. In practice, you will seldom find this a problem.

Conclusion

Acquisition provides a powerful way to dynamically share information between objects. Zope uses acquisition for a number of its key features including security, object publishing, and DTML variable lookup. Acquisition also provides an elegant solution to the problem of circular references for many classes of problems. While acquisition is powerful, you should take care when using acquisition in your applications. The details can get complex, especially with the differences between acquiring from context and acquiring from containment.

acquisition's People

Contributors

d-maurer avatar dataflake avatar davisagli avatar fafhrd91 avatar freddrake avatar hannosch avatar icemac avatar jamadden avatar jugmac00 avatar lrowe avatar mgedmin avatar msabramo avatar philikon avatar sallner avatar sidnei avatar stephan-hof avatar strichter avatar tseaver avatar vernans avatar witsch avatar yuseitahara avatar zopyx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

acquisition's Issues

aq_acquire ignores the 'default' parameter.

As follows:

>>> from Acquisition import aq_acquire, aq_get
>>> aq_acquire(object(), 'nonesuch', default='abc')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: 'object' object has no attribute 'nonesuch'

PURE_PYTHON: bad access to attributes with acquisition wrapper value

The problem is demonstrated by the following code:

>>> from Acquisition import Implicit
>>> class I(Implicit):
...   def add(self, id, o):
...     setattr(self, id, o)
...     return getattr(self, id)
... 
>>> top = I()
>>> f = top.add("f", I())
>>> 
>>> i = I()
>>> i.add("f", f)
<__main__.I object at 0x7f7b8acbf668>
>>> i.f.aq_self is f
False

Works in the "C" version.

The problem is caused by Acquisition._Wrapper.__of__:

    def __of__(self, parent):
        # Based on __of__ in the C code;
        # simplify a layer of wrapping.

        # We have to call the raw __of__ method or we recurse on our
        # own lookup (the C code does not have this issue, it can use
        # the wrapped __of__ method because it gets here via the
        # descriptor code path)...
        wrapper = self._obj.__of__(parent)
        if (not isinstance(wrapper, _Wrapper) or
                not isinstance(wrapper._container, _Wrapper)):
            return wrapper
        # but the returned wrapper should be based on this object's
        # wrapping chain
        wrapper._obj = self

        while (isinstance(wrapper._obj, _Wrapper) and
               (wrapper._obj._container is wrapper._container._obj)):
            # Since we mutate the wrapper as we walk up, we must copy
            # XXX: This comes from the C implementation. Do we really need to
            # copy?
            wrapper = type(wrapper)(wrapper._obj, wrapper._container)
            wrapper._obj = wrapper._obj._obj
        return wrapper

It fails, if parent is not a wrapper.

segfault in _Acquisition.so's `Wrapper_findattr`; object lifetime issue?

Recently on 64-bit Linux with Python 2.7.8 we've been seeing a web application crash the python interpreter with segmentation faults like this:

segfault at 7f8619c3e054 ip 00007f864a5debb5 sp 00007fff347f37b0 error 4 in _Acquisition.so[7f864a5da000+8000]

With the base address of 0x7f864a5da000 and the instruction pointer of 0x7f864a5debb5, that gives us our crashing offset into the binary _Acquisiton.so of 0x4bb5. gdb and addr2line identify that as line 487 in _Acquisition.c:

  if ((*name=='a' && name[1]=='q' && name[2]=='_') ||

Looking at the disassembly backs that up:

   0x0000000000004ba6 <+86>:    cmovne %rdx,%rbx
   0x0000000000004baa <+90>:    test   $0x10000000,%eax
   0x0000000000004baf <+95>:    jne    0x52c0 <Wrapper_findattr+1904>
   0x0000000000004bb5 <+101>:   movzbl (%rbx),%eax
   0x0000000000004bb8 <+104>:   cmp    $0x61,%al
   0x0000000000004bba <+106>:   je     0x4de0 <Wrapper_findattr+656>

In other words, the crash is on dereferencing name itself (because the next instruction is to compare it with 'a'). (FWIW, this is in Acquisition 4.2, but I compared the disassembly and compilation with a build of Acquisition 4.0.3 from November and they are the same.)

Where did name come from? Here's the lines leading up to 487, the very first lines in the function:

  PyObject *r, *v, *tb, *tmp;
  char *name="";

  if (PyString_Check(oname)) name=PyString_AS_STRING(oname);
  if (PyUnicode_Check(oname)) {
    tmp=PyUnicode_AsASCIIString(oname);
    if (tmp==NULL) return NULL;
    name=PyString_AS_STRING(tmp);
    Py_DECREF(tmp);
  }
  if ((*name=='a' && name[1]=='q' && name[2]=='_') ||

Now, from the error message, we know that name isn't a null pointer. We also know that it's not the constant value that name is initialized to (because it's out of the address space that the .so occupies).

If oname had been a str object, name would directly be pointing to the oname->ob_sval buffer, because that's what PyString_AS_STRING does; As the documentation says, "The pointer refers to the internal buffer of string, not a copy." Presumably that buffer isn't invalid (could it be?), and there's no way for that to change during this function.

On the other hand, if oname was a unicode object, a temporary string is allocated with PyUnicode_AsASCIIString. This returns a new reference, and the code then decrements the reference, presumably releasing the string. But at that point, name is a pointer to the ob_sval of a now-released string.

The only hypothesis I have to explain the crash is this dangling pointer. Depending on the internal state of the Python (arena) memory allocator, the page that the unicode string was on may have been munmap or simply freed back to the operating system. Most of the time it works fine because there's nothing that would cause that memory to get reused (corrupting the string) or unmapped, but sometimes, under certain conditions (memory pressure?), the dangling pointer crashes.

Does this sound plausible? If so, is the fix simply to copy the string before dereferencing the temporary string? I can submit a PR for that.

Please release a new egg

#22 may be more widespread, IMHO it will affect all Acquisition-heavy sites since at least Acquisition 2.13.11, which ships with Zope 2.13.26. That's the oldest version I tested.

The symptoms include frequent Python crashes with segfaults pointing to _Acquisition.so, signal 6 (ABORT) crashes with stack traces pointing to GC garbage collection and signal 7 (bus error) crashes.

Please release a new version and eggs, I'm currently relying on a locally-built egg for a customer. Zope 2.13.26 appears to run fine with the current Acquisition fix, ExtensionClass 4.3.0 and as dependency BTreeFolder 3.0.

I would do it myself, but the plethora of binary eggs on PiPI suggests that there's some ready-made rig somewhere to produce them. I can only do source eggs locally.

Acquisition wrappers do not respect equality/ordering of the base object

Unless a class defines __cmp__ (which is unusual for Python 3), acquisition wrappers ignore equality and ordering definitions for the base object and implement them based on the base object's addresses.

>>> from Acquisition.tests import I
>>> class RichOrdered(I):
...   def __eq__(self, other): return self.id == other.id
...   def __lt__(self, other): return self.id < other.id
... 
>>> i1 = RichOrdered(""); i2 = RichOrdered("")
>>> top = I(""); top.i1 = i1; top.i2 = i2
>>> i1 == i2
True
>>> top.i1 == top.i2
False
>>> i2.id = "i2"
>>> i1 < i2
True
>>> top.i1 < top.i2
False

Build / publish wheels on Appveyor

Checklist from zopefoundation/zope.proxy#24:

  • a copy of appveyor.yml from zope.interface or similar
  • someone with an Appveyor account (e.g. @mgedmin ) to enable this project
  • encrypted PyPI upload credentials in the appveyor.yml (no change required from the copy in zope.interface if @mgedmin is the one who enables the Appveyor account)
  • grant zope.wheelbuilder PyPI rights
  • push the appveyor.yml commit to master
  • (optionally) a push of a temporary branch that builds and uploads Acquisition 4.4.2 wheels

Missing C fields?

I've been looking over tp_flags and what these imply in different Python versions.

It looks like we declare Py_TPFLAGS_DEFAULT, which implies Py_TPFLAGS_HAVE_INPLACEOPS.

That in turn says PySequenceMethods should have the sq_inplace_concat and sq_inplace_repeat fields. Our method definition ends with sq_contains. Our PyNumberMethods defines all the inplace fields. The inplace operators were added in Python 2.0.

Py_TPFLAGS_DEFAULT also implies Py_TPFLAGS_HAVE_CLASS, which wants a whole host of fields. We now define all of them up to tp_new, but miss the rest of them:

tp_free,
tp_is_gc,
tp_bases,
tp_mro,
tp_cache,
tp_subclasses,
tp_weaklist

Looking over the tp_flags for Python 3.6, there's only one new flag worth mentioning Py_TPFLAGS_HAVE_FINALIZE. This is a new addition as of Python 3.4, where they reworked how destructors worked.

In Python 3.5 the tp_compare flag got repurposed as tp_as_async, but we luckily have set that one to 0.

I'm not quite sure what to do about these. I think we can ignore the new opt-in finalize and async flags.

Do we need to do something about the new-style classes additions (Py_TPFLAGS_HAVE_CLASS)?

The inplace sequence methods seem like something we maybe should wrap and add to both the Python and C code.

@stephan-hof Thoughts?

PURE_PYTHON: bad (maybe redundant) "special method"s implementation

The pure Python version implements the _Wrapper special methods wrongly:
consider the following case:

from Acquisition import Implicit
class I(Implicit):
  def add(self, id):
    o = I(); o.id = id
    setattr(self, id, o)
    return getattr(self, id)

top = I()
f = top.add("f")
f2 = f.f

Calling bool(f2) causes an infinite loop:

>>> bool(f2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dieter/tmp/ec/Acquisition/src/Acquisition/__init__.py", line 545, in __nonzero__
    return bool(nonzero(self))  # Py3 is strict about the return type
  File "/home/dieter/tmp/ec/Acquisition/src/Acquisition/__init__.py", line 545, in __nonzero__
    return bool(nonzero(self))  # Py3 is strict about the return type
  File "/home/dieter/tmp/ec/Acquisition/src/Acquisition/__init__.py", line 545, in __nonzero__
    return bool(nonzero(self))  # Py3 is strict about the return type
  [Previous line repeated 988 more times]
  File "/home/dieter/tmp/ec/Acquisition/src/Acquisition/__init__.py", line 535, in __nonzero__
    aq_self = self._obj
  File "/home/dieter/tmp/ec/Acquisition/src/Acquisition/__init__.py", line 412, in __getattribute__
    return _OGA(self, name)
RecursionError: maximum recursion depth exceeded while calling a Python object

Additional debugging information for Acquisition error

_Acquisition.c has specific code for recognising when an Acquisition-wrapped item is being fed in to the Python Pickler, and it throws TypeError: "Can't pickle objects in acquisition wrappers." when this occurs.

The error is very opaque and difficult to track down what, actually, is being erroneously Acquisition wrapped and fed in to the Pickler. A sample (not Production-quality) improvement for this would be something like:

--- Acquisition-4.7/setup.py	2020-10-07 22:44:28.000000000 +1300
+++ Acquisition-4.7a/setup.py	2022-04-06 18:29:55.235552924 +1200
@@ -36,7 +36,7 @@
                   include_dirs=['include', 'src']),
     ]
 
-version = '4.7'
+version = '4.7a'
 
 setup(
     name='Acquisition',
diff -ru Acquisition-4.7/src/Acquisition/_Acquisition.c Acquisition-4.7a/src/Acquisition/_Acquisition.c
--- Acquisition-4.7/src/Acquisition/_Acquisition.c	2020-10-07 22:44:28.000000000 +1300
+++ Acquisition-4.7a/src/Acquisition/_Acquisition.c	2022-04-07 00:14:47.942013701 +1200
@@ -1477,8 +1477,16 @@
 PyObject *
 Wrappers_are_not_picklable(PyObject *wrapper, PyObject *args)
 {
+    PyObject *obj;
+    PyObject *repr;
+    /* C progammers disease... */
+    char msg[1024];
+    /* Unwrap wrapper completely -> obj. */
+    obj = get_base(wrapper);
+    repr = PyObject_Repr((PyObject *)obj);
+    snprintf(msg, 1024, "Can't pickle objects in acquisition wrappers - %s", (char *)PyBytes_AS_STRING(repr));
     PyErr_SetString(PyExc_TypeError,
-                    "Can't pickle objects in acquisition wrappers.");
+                    msg);
     return NULL;
 }

PyPy Support: Interest and Feasibility

We have a project we're interested in running on PyPy, and one of our current blockers for that is Acquisition. Our project doesn't use all the features of Acquisition and I'm not an Acquisition expert, but as I started noodling around with it over at https://github.com/Nextthought/Acquisition, it started to seem like it should be possible to implement almost all of Acquisition in pure Python in a similar fashion to zope.proxy. (A few hours work has about half the test cases passing, although this does seem like a clear place where the 80/20 rule applies.)

I think I have the time to work on such a port now, but before I get too much further I wanted to see if there was any community interest in such a port, and whether such a port is likely to get reviewed/merged. If there is, I'll press ahead with trying to make as complete an implementation as possible (although I'm sure I'll have some questions first), otherwise I'll just focus on small stubs for the things I know our project needs. Of course, if someone already knows of reasons why a port is doomed to failure (I didn't find much traffic about the topic via Google), I'd appreciate hearing that too :)

Converting ImplicitAcquisitionWrapper to bytes fails

Calling the bytes built-in on a ImplicitAcquisitionWrapper fails with AttributeError: __index__, see the following example:

class A(Implicit):

    def __bytes__(self):
        return b'my bytes'

a = A()
a.b = A()
wrapper = Acquisition.ImplicitAcquisitionWrapper(a.b, a)
bytes(wrapper)

The result is AttributeError: __index__ in the last line. (But when using str and __str__ the return value of the __str__ method is returned.)

This happens on Python 3.

Port C extension to Python 3

After we have a port of the ExtensionClass C extension to Python 3, it should be possible to also port Acquisition's C extension.

@stephan-hof had shown interest in the past but other are most welcome to work on it ;)

Pure-Python Wrapper objects break `object.__getattribute__` in methods of the wrapped object

Certain "base" classes make direct use of object.__getattribute__ in their method implementations. (i.e., outside of their own __getattribute__ implementation). They may do this for performance reasons, or to simplify their implementation of __getattribute__. One prominent example is persistent.Persistent, but there are undoubtedly others.

If classes like this are wrapped with a pure-Python acquisition wrapper, those methods fail, because the self they have is not the self they expect (it's the wrapper), and directly calling object.__getattribute__ bypasses the wrapper's __getattribute__ implementation.

Here's an example test that fails with pure-Python, but works under the C implementation:

    def test_object_getattribute_in_rebound_method(self):

        class Persistent(object):
            __slots__ = ('__flags')
            def __init__(self):
                self.__flags = 42

            def get_flags(self):
                return object.__getattribute__(self, '_Persistent__flags')

        wrapped = Persistent()
        wrapper = Acquisition.ImplicitAcquisitionWrapper(wrapped, None)

        self.assertEqual(wrapped.get_flags(), wrapper.get_flags())

It fails like so:

  File "/Acquisition/tests.py", line 3171, in test_object_getattribute_in_rebound_method
    self.assertEqual(wrapped.get_flags(), wrapper.get_flags())
  File "/Acquisition/tests.py", line 3166, in get_flags
    return object.__getattribute__(self, '_Persistent__flags')
AttributeError: 'ImplicitAcquisitionWrapper' object has no attribute '_Persistent__flags'

I'm trying to figure out how to fix this.

Build breaks on Python 3.11

What I did:

Run the tests for Python 3.11.0a2.

What I expect to happen:

Successful test run.

What actually happened:

$ tox -epy311
py311 create: /.../Acquisition/.tox/py311
py311 develop-inst: /.../Acquisition
ERROR: invocation failed (exit code 1), logfile: /.../Acquisition/.tox/py311/log/py311-1.log
======================================================================= log start ========================================================================
Looking in indexes: https://pypi.org/simple, https://mihowitz:****@development.verdi.de/devpi/verdi/prod/+simple/
Obtaining file:///.../Acquisition
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting ExtensionClass>=4.2.0
  Using cached https://development.verdi.de/devpi/verdi/prod/%2Bf/662/b2dc300520a7a/ExtensionClass-4.5.1.tar.gz (34 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting zope.interface
  Using cached zope.interface-5.4.0-cp311-cp311-macosx_11_0_x86_64.whl
Collecting zope.testrunner
  Using cached https://development.verdi.de/devpi/verdi/prod/%2Bf/ae7/fbeb862a36083/zope.testrunner-5.4.0-py2.py3-none-any.whl (216 kB)
Requirement already satisfied: setuptools in ./.tox/py311/lib/python3.11/site-packages (from zope.interface->Acquisition==4.10.dev0) (58.3.0)
Collecting zope.exceptions
  Using cached https://development.verdi.de/devpi/verdi/prod/%2Bf/bb9/8cc07e90ebe59/zope.exceptions-4.4-py2.py3-none-any.whl (18 kB)
Collecting six
  Using cached https://development.verdi.de/devpi/root/pypi/%2Bf/8ab/b2f1d86890a2d/six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: ExtensionClass
  Building wheel for ExtensionClass (setup.py): started
  Building wheel for ExtensionClass (setup.py): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /.../Acquisition/.tox/py311/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-install-ld_8nlao/extensionclass_61c21a0fe7954764a661895b303a5cd0/setup.py'"'"'; __file__='"'"'/private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-install-ld_8nlao/extensionclass_61c21a0fe7954764a661895b303a5cd0/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-wheel-y9_17peb
       cwd: /private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-install-ld_8nlao/extensionclass_61c21a0fe7954764a661895b303a5cd0/
  Complete output (53 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-11.0-x86_64-3.11
  creating build/lib.macosx-11.0-x86_64-3.11/ComputedAttribute
  copying src/ComputedAttribute/__init__.py -> build/lib.macosx-11.0-x86_64-3.11/ComputedAttribute
  copying src/ComputedAttribute/tests.py -> build/lib.macosx-11.0-x86_64-3.11/ComputedAttribute
  creating build/lib.macosx-11.0-x86_64-3.11/MethodObject
  copying src/MethodObject/__init__.py -> build/lib.macosx-11.0-x86_64-3.11/MethodObject
  copying src/MethodObject/tests.py -> build/lib.macosx-11.0-x86_64-3.11/MethodObject
  creating build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
  copying src/ExtensionClass/__init__.py -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
  copying src/ExtensionClass/tests.py -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
  running egg_info
  writing src/ExtensionClass.egg-info/PKG-INFO
  writing dependency_links to src/ExtensionClass.egg-info/dependency_links.txt
  writing requirements to src/ExtensionClass.egg-info/requires.txt
  writing top-level names to src/ExtensionClass.egg-info/top_level.txt
  reading manifest file 'src/ExtensionClass.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE.txt'
  writing manifest file 'src/ExtensionClass.egg-info/SOURCES.txt'
  copying src/ComputedAttribute/_ComputedAttribute.c -> build/lib.macosx-11.0-x86_64-3.11/ComputedAttribute
  copying src/MethodObject/_MethodObject.c -> build/lib.macosx-11.0-x86_64-3.11/MethodObject
  copying src/ExtensionClass/ExtensionClass.h -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
  copying src/ExtensionClass/_ExtensionClass.c -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
  copying src/ExtensionClass/_compat.h -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
  creating build/lib.macosx-11.0-x86_64-3.11/ExtensionClass/pickle
  copying src/ExtensionClass/pickle/pickle.c -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass/pickle
  running build_ext
  building 'ExtensionClass._ExtensionClass' extension
  creating build/temp.macosx-11.0-x86_64-3.11
  creating build/temp.macosx-11.0-x86_64-3.11/src
  creating build/temp.macosx-11.0-x86_64-3.11/src/ExtensionClass
  /usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -Isrc -I/.../Acquisition/.tox/py311/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c src/ExtensionClass/_ExtensionClass.c -o build/temp.macosx-11.0-x86_64-3.11/src/ExtensionClass/_ExtensionClass.o
  src/ExtensionClass/_ExtensionClass.c:840:16: error: expression is not assignable
    Py_TYPE(typ) = ECExtensionClassType;
    ~~~~~~~~~~~~ ^
  src/ExtensionClass/_ExtensionClass.c:800:33: warning: comparison of integers of different signs: 'Py_ssize_t' (aka 'long') and 'unsigned long' [-Wsign-compare]
            if (typ->tp_basicsize <= sizeof(_emptyobject))
                ~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~
  src/ExtensionClass/_ExtensionClass.c:985:32: error: expression is not assignable
    Py_TYPE(&ExtensionClassType) = &PyType_Type;
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
  src/ExtensionClass/_ExtensionClass.c:995:22: error: expression is not assignable
    Py_TYPE(&BaseType) = &ExtensionClassType;
    ~~~~~~~~~~~~~~~~~~ ^
  src/ExtensionClass/_ExtensionClass.c:1003:42: error: expression is not assignable
    Py_TYPE(&NoInstanceDictionaryBaseType) = &ExtensionClassType;
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
  1 warning and 4 errors generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for ExtensionClass
  Running setup.py clean for ExtensionClass
Failed to build ExtensionClass
Installing collected packages: zope.interface, zope.exceptions, six, ExtensionClass, zope.testrunner, Acquisition
    Running setup.py install for ExtensionClass: started
    Running setup.py install for ExtensionClass: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /.../Acquisition/.tox/py311/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-install-ld_8nlao/extensionclass_61c21a0fe7954764a661895b303a5cd0/setup.py'"'"'; __file__='"'"'/private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-install-ld_8nlao/extensionclass_61c21a0fe7954764a661895b303a5cd0/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-record-8278nq5v/install-record.txt --single-version-externally-managed --compile --install-headers /.../Acquisition/.tox/py311/include/site/python3.11/ExtensionClass
         cwd: /private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-install-ld_8nlao/extensionclass_61c21a0fe7954764a661895b303a5cd0/
    Complete output (55 lines):
    running install
    /.../Acquisition/.tox/py311/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running build
    running build_py
    creating build
    creating build/lib.macosx-11.0-x86_64-3.11
    creating build/lib.macosx-11.0-x86_64-3.11/ComputedAttribute
    copying src/ComputedAttribute/__init__.py -> build/lib.macosx-11.0-x86_64-3.11/ComputedAttribute
    copying src/ComputedAttribute/tests.py -> build/lib.macosx-11.0-x86_64-3.11/ComputedAttribute
    creating build/lib.macosx-11.0-x86_64-3.11/MethodObject
    copying src/MethodObject/__init__.py -> build/lib.macosx-11.0-x86_64-3.11/MethodObject
    copying src/MethodObject/tests.py -> build/lib.macosx-11.0-x86_64-3.11/MethodObject
    creating build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
    copying src/ExtensionClass/__init__.py -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
    copying src/ExtensionClass/tests.py -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
    running egg_info
    writing src/ExtensionClass.egg-info/PKG-INFO
    writing dependency_links to src/ExtensionClass.egg-info/dependency_links.txt
    writing requirements to src/ExtensionClass.egg-info/requires.txt
    writing top-level names to src/ExtensionClass.egg-info/top_level.txt
    reading manifest file 'src/ExtensionClass.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    adding license file 'LICENSE.txt'
    writing manifest file 'src/ExtensionClass.egg-info/SOURCES.txt'
    copying src/ComputedAttribute/_ComputedAttribute.c -> build/lib.macosx-11.0-x86_64-3.11/ComputedAttribute
    copying src/MethodObject/_MethodObject.c -> build/lib.macosx-11.0-x86_64-3.11/MethodObject
    copying src/ExtensionClass/ExtensionClass.h -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
    copying src/ExtensionClass/_ExtensionClass.c -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
    copying src/ExtensionClass/_compat.h -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass
    creating build/lib.macosx-11.0-x86_64-3.11/ExtensionClass/pickle
    copying src/ExtensionClass/pickle/pickle.c -> build/lib.macosx-11.0-x86_64-3.11/ExtensionClass/pickle
    running build_ext
    building 'ExtensionClass._ExtensionClass' extension
    creating build/temp.macosx-11.0-x86_64-3.11
    creating build/temp.macosx-11.0-x86_64-3.11/src
    creating build/temp.macosx-11.0-x86_64-3.11/src/ExtensionClass
    /usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -Isrc -I/.../Acquisition/.tox/py311/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c src/ExtensionClass/_ExtensionClass.c -o build/temp.macosx-11.0-x86_64-3.11/src/ExtensionClass/_ExtensionClass.o
    src/ExtensionClass/_ExtensionClass.c:840:16: error: expression is not assignable
      Py_TYPE(typ) = ECExtensionClassType;
      ~~~~~~~~~~~~ ^
    src/ExtensionClass/_ExtensionClass.c:800:33: warning: comparison of integers of different signs: 'Py_ssize_t' (aka 'long') and 'unsigned long' [-Wsign-compare]
              if (typ->tp_basicsize <= sizeof(_emptyobject))
                  ~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~
    src/ExtensionClass/_ExtensionClass.c:985:32: error: expression is not assignable
      Py_TYPE(&ExtensionClassType) = &PyType_Type;
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
    src/ExtensionClass/_ExtensionClass.c:995:22: error: expression is not assignable
      Py_TYPE(&BaseType) = &ExtensionClassType;
      ~~~~~~~~~~~~~~~~~~ ^
    src/ExtensionClass/_ExtensionClass.c:1003:42: error: expression is not assignable
      Py_TYPE(&NoInstanceDictionaryBaseType) = &ExtensionClassType;
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
    1 warning and 4 errors generated.
    error: command '/usr/bin/clang' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /.../Acquisition/.tox/py311/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-install-ld_8nlao/extensionclass_61c21a0fe7954764a661895b303a5cd0/setup.py'"'"'; __file__='"'"'/private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-install-ld_8nlao/extensionclass_61c21a0fe7954764a661895b303a5cd0/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/dn/l6fksjj91v78hz2503q68vr40000h1/T/pip-record-8278nq5v/install-record.txt --single-version-externally-managed --compile --install-headers /.../Acquisition/.tox/py311/include/site/python3.11/ExtensionClass Check the logs for full command output.

======================================================================== log end =========================================================================
________________________________________________________________________ summary _________________________________________________________________________
ERROR:   py311: InvocationError for command /.../Acquisition/.tox/py311/bin/python -m pip install --exists-action w -e '/.../Acquisition[test]' (exited with code 1)

What version of Python and Zope/Addons I am using:

Python 3.11.0a2
Acquisition: master

Uses deprecated API PyEval_CallObjectWithKeywords, raises warnings under 3.10

Deprecation Warning during installation

What I did:

I was running the tests of Zope with tox in a fresh environment with no pre-compiled eggs.

What I expect to happen:

Installation without any warnings.

What actually happened:

Getting distribution for 'Acquisition==4.8'.
WARNING: The easy_install command is deprecated and will be removed in a future version.
src/Acquisition/_Acquisition.c: In function ‘CallMethod’:
src/Acquisition/_Acquisition.c:133:5: warning: ‘PyEval_CallObjectWithKeywords’ is deprecated [-Wdeprecated-declarations]
  133 |     result = PyEval_CallObjectWithKeywords(callable, args, kwargs);
      |     ^~~~~~
In file included from /usr/include/python3.10/Python.h:144,
                 from include/ExtensionClass/ExtensionClass.h:83,
                 from src/Acquisition/_Acquisition.c:15:
/usr/include/python3.10/ceval.h:17:43: note: declared here
   17 | Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyEval_CallObjectWithKeywords(
      |                                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Got Acquisition 4.8.

What version of Python and Zope/Addons I am using:

Linux
Python 3.10 via tox
Zope on this commit https://github.com/zopefoundation/Zope/tree/314683f66cc705d162a35436b1c7178ddb5f409d

Fix process of reference count during GC

When we subtract reference counter of an object, the destructor is automatically called if the value of counter is 0.
At the same time an error occurs, as the GC executed while the destructor is still in running phase.
To eliminate the error, we have added below program which exclude objects from GC in the destructor in line 4.

static void
Wrapper_dealloc(Wrapper *self)
{
PyObject_GC_UnTrack((PyObject )self); //Add this line to eliminate self from GC
Wrapper_clear(self);
self->ob_type->tp_free((PyObject
)self);
}

Patch: Acquisition.zip

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.