Giter Club home page Giter Club logo

custom_inherit's Introduction

custom_inherit

Automated tests status PyPi version Conda Version Python version support

Contents

Overview

The Python package custom_inherit provides convenient, light-weight tools for inheriting docstrings in customizeable ways.

Features

  • A metaclass that instructs children to inherit docstrings for their attributes from their parents, using custom docstring inheritance styles. This works for all varieties of methods (instance, static, class) and properties, including abstract ones.
  • A decorator that merges a string/docstring with the docstring of the decorated object using custom styles. This can decorate functions as well as all varieties of class attributes.
  • Built-in docstring merging styles for popular docstring specifications:
  • A simple interface for using your own docstring inheritance style.

Implementation Notes

  • These tools are compatible with Sphinx - the inherited docstrings will be rendered by this package.
  • These tools do not obfuscate function signatures or error traceback messages, nor do they affect performance beyond the initial construction process.
  • No dependencies.

Projects That Use custom_inherit

Basic Usage

Inheriting Docstrings Using a Metaclass

custom_inherit exposes a metaclass, DocInheritMeta(), that, when derived from by a class, will automatically mediate docstring inheritance for all subsequent derived classes of that parent. Thus a child's attributes (methods, classmethods, staticmethods, properties, and their abstract counterparts) will inherit documentation from its parent's attribute, and the resulting docstring is synthesized according to a customizable style.

The style of the inheritance scheme can be specified explicitly when passing DocInheritMeta its arguments. Here is a simple usage example using the built-in "numpy" style of inheritance:

from custom_inherit import DocInheritMeta

class Parent(metaclass=DocInheritMeta(style="numpy")):
   def meth(self, x, y=None):
       """ Parameters
           ----------
           x: int
              blah-x

           y: Optional[int]
              blah-y

           Raises
           ------
           NotImplementedError"""
       raise NotImplementedError

class Child(Parent):
    def meth(self, x, y=None):
        """ Method description

            Returns
            -------
            int

            Notes
            -----
            Some notes here."""
        return 0

Because we specified style="numpy" in DocInheritMeta, the inherited docstring of Child.meth will be:

  """ Method description

      Parameters
      ----------
      x: int
         blah-x

      y: Optional[int]
         blah-y

      Returns
      -------
      int

      Notes
      -----
      Some notes here."""

(note the special case where the "Raises" section of the parent's method is left out, because the child class implements a "Returns" section instead. Jump ahead for a detailed description of the "numpy" style)

Keep in mind that the syntax for deriving from a meta class is slightly different in Python 2:

from custom_inherit import DocInheritMeta

class Parent(object):
   __metaclass__ = DocInheritMeta(style="numpy")
   ...

Inheriting Docstrings Using a Decorator

custom_inherit also exposes a decorator capable of mediating docstring inheritance on an individual function (or property, method, etc.) level. In this example, we provide our own custom inheritance style-function on the fly (rather than using a built-in style):

from custom_inherit import doc_inherit

def my_style(prnt_doc, child_doc): return "\n-----".join([prnt_doc, child_doc])

def parent():  # parent can be any object with a docstring, or simply a string itself
   """ docstring to inherit from"""

@doc_inherit(parent, style=my_style)
def child():
   """ docstring to inherit into"""

Given the customized (albeit stupid) inheritance style specified in this example, the inherited docsting of child, in this instance, will be:

"""docstring to inherit from
  -----
  docstring to inherit into"""

Advanced Usage

A very natural, but more advanced use case for docstring inheritance is to define an abstract base class that has detailed docstrings for its abstract methods/properties. This class can be passed DocInheritMeta(abstract_base_class=True), and it will have inherited from abc.ABCMeta, plus all of its derived classes will inherit the docstrings for the methods/properties that they implement:

# Parent is now an abstract base class
class Parent(metaclass=DocInheritMeta(style="numpy", abstract_base_class=True)):
   ...

For the "numpy", "google", and "napoleon_numpy" inheritance styles, one then only needs to specify the "Returns" or "Yields" section in the derived class' attribute docstring for it to have a fully-detailed docstring.

Another option is to be able to decide whether to include all special methods, meaning methods that start and end by "__" such as "init" method, or not in the doctstring inheritance process. Such an option can be pass to the DocInheritMeta metaclass constructor:

# Each child class will also merge the special methods' docstring of its parent class
class Parent(metaclass=DocInheritMeta(style="numpy", include_special_methods=True)):
   ...

Special methods are not included by default.

Built-in Styles

Utilize a built-in style by specifying any of the following names (as a string), wherever the style parameter is to be specified. The built-in styles are:

  • "parent": Wherever the docstring for a child-class' attribute (or for the class itself) is None, inherit the corresponding docstring from the parent. (Deprecated in Python 3.5)

  • "numpy": NumPy-styled docstrings from the parent and child are merged gracefully with nice formatting. The child's docstring sections take precedence in the case of overlap.

  • "numpy_with_merge": Behaves identically to the "numpy" style, but also merges - with de-duplication - sections that overlap, instead of only keeping the child's section. The sections that are merged are "Attributes", "Parameters", "Methods", "Other Parameters", and "Keyword Arguments".

  • "google": Google-styled docstrings from the parent and child are merged gracefully with nice formatting. The child's docstring sections take precedence in the case of overlap. This adheres to the napoleon specification for the Google style.

  • "google_with_merge": Behaves identically to the "google" style, but also merges - with de-duplication - sections that overlap, instead of only keeping the child's section. The sections that are merged are "Attributes", "Parameters", "Methods", "Other Parameters", and "Keyword Arguments" (or their aliases).

  • "numpy_napoleon": NumPy-styled docstrings from the parent and child are merged gracefully with nice formatting. The child's docstring sections take precedence in the case of overlap. This adheres to the napoleon specification for the NumPy style.

  • "numpy_napoleon_with_merge": Behaves identically to the 'numpy_napoleon' style, but also merges - with de-duplication - sections that overlap, instead of only keeping the child's section. The sections that are merged are "Attributes", "Parameters", "Methods", "Other Parameters", and "Keyword Arguments" (or their aliases).

  • "reST": reST-styled docstrings from the parent and child are merged gracefully with nice formatting. Docstring sections are specified by reST section titles. The child's docstring sections take precedence in the case of overlap.

For the numpy, numpy_with_merge, numpy_napoleon, numpy_napoleon_with_merge, google and google_with_merge styles, if the parent's docstring contains a "Raises" section and the child's docstring implements a "Returns" or a "Yields" section instead, then the "Raises" section is not included in the resulting docstring. This is to accomodate for the relatively common use case in which an abstract method/property raises NotImplementedError. Child classes that implement this method/property clearly will not raise this. Of course, any "Raises" section that is explicitly included in the child's docstring will appear in the resulting docstring.

Detailed documentation and example cases for the default styles can be found here

Making New Inheritance Styles

Implementing your inheritance style is simple.

  • Provide an inheritance style on the fly wherever a style parameter is specified:

    • Supply any function of the form: func(prnt_doc: str, child_doc: str) -> str
  • Log an inheritance style, and refer to it by name wherever a style parameter is specified, using either:

    • custom_inherit.store["my_style"] = func
    • custom_inherit.add_style("my_style", func).

Installation and Getting Started

Install via pip:

    pip install custom_inherit

Install via conda:

    conda install -c conda-forge custom-inherit

or

Download/clone this repository, go to its directory, and install custom_inherit by typing in your command line:

    python setup.py install

If, instead, you want to install the package with links, so that edits you make to the code take effect immediately within your installed version of custom_inherit, type:

    python setup.py develop

and then get started with

from custom_inherit import DocInheritMeta, doc_inherit, store
# print(store) shows you the available styles

Documentation

Documentation is available via help(custom_inherit).

custom_inherit.DocInheritMeta(style="parent", abstract_base_class=False):
    """ A metaclass that merges the respective docstrings of a parent class and of its child, along with their
        properties, methods (including classmethod, staticmethod, decorated methods).

        Parameters
        ----------
        style: Union[Hashable, Callable[[str, str], str]], optional (default: "parent")
            A valid inheritance-scheme style ID or function that merges two docstrings.

        abstract_base_class: bool, optional(default: False)
            If True, the returned metaclass inherits from abc.ABCMeta.

            Thus a class that derives from DocInheritMeta(style="numpy", abstract_base_class=True)
            will be an abstract base class, whose derived classes will inherit docstrings
            using the numpy-style inheritance scheme.


        Returns
        -------
        custom_inherit.DocInheritorBase"""


custom_inherit.doc_inherit(parent, style="parent"):
    """ Returns a function/method decorator that, given `parent`, updates the docstring of the decorated
        function/method based on the specified style and the corresponding attribute of `parent`.

        Parameters
        ----------
        parent : Union[str, Any]
            The object whose docstring, is utilized as the parent docstring
	    during the docstring merge. Or, a string can be provided directly.

        style : Union[Hashable, Callable[[str, str], str]], optional (default: "parent")
            A valid inheritance-scheme style ID or function that merges two docstrings.

        Returns
        -------
        custom_inherit.DocInheritDecorator


        Notes
        -----
        `doc_inherit` should always be the inner-most decorator when being used in
        conjunction with other decorators, such as `@property`, `@staticmethod`, etc."""


custom_inherit.remove_style(style):
    """ Remove the specified style from the style store.

        Parameters
        ----------
        style: Hashable
            The inheritance-scheme style ID to be removed."""


custom_inherit.add_style(style_name, style_func):
    """ Make available a new function for merging a 'parent' and 'child' docstring.

        Parameters
        ----------
        style_name : Hashable
            The identifier of the style being logged
        style_func: Callable[[Optional[str], Optional[str]], Optional[str]]
            The style function that merges two docstrings into a single docstring."""

Go Back To:

custom_inherit's People

Contributors

abeelen avatar antoined avatar livinthelookingglass avatar nirum avatar rsokl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

custom_inherit's Issues

Documentation

I was copy'n'pasting the examples and found some mistakes:

  • the colons are sometimes missing:
    • class Parent(metaclass=DocInheritMeta(style="numpy"))should be class Parent(metaclass=DocInheritMeta(style="numpy")):
    • class Parent(object) as well

The indentation is not always correct:

  • the line with raise NotImplementedError to the left
  • the indentations are inconsistent (but valid) in the section inheriting Docstrings using a decorator

Cannot remove parameters

In my case I remove all arguments accepted in parent class in child's class method. It seems there is no way to also remove them from a docstring and it is automatically copied over? So I would like to remove the whole section. It seems I have to define an otherwise empty "Parameters" section.

(Working in numpy style.)

It does not work with multiple inheritance (mixins)

I have set of mixins to be used together with a base class:

from custom_inherit import DocInheritMeta

class Parent(metaclass=DocInheritMeta(style="numpy")):
    """
    Parent class.

    Attributes
    ------------
    foo : str
        Foo attribute.
    """

class Mixin:
    """
    This is mixin which does something.
    """

class Child(Mixin, Parent):
    """
    Child description.
    """
>>> Child.__doc__
'Child description.'

Instead of expected:

>>> Child.__doc__
'Child description.\n\nAttributes\n----------\nfoo : str\n    Foo attribute.'

Now, the workaround is to put Mixin after Parent in the inheritance, but this goes against the logic I need (mixin has to override some things from parent).

Whitespace stripped with numpy style causes trouble

Part of the inheritance merging (for numpy style) involves stripping as much whitespace as possible. This is expected behaviour from the docstrings

Any whitespace that can be uniformly removed from a docstring's second line and onwards is
removed. Sections will be separated by a single blank line.
and the tests
GrandChild.__doc__
== "Grand Child.\n\nMethods\n-------\nblah : does not much\nmeth: does something\n\nNotes\n-----\nBlah"

However, from the example README.md it's not expected (I've taken the liberty to shorten the example):

  """Method description

      Parameters
      ----------
      x: int
         blah-x
       ..."""

As this will be changed to

"Method description.\n\nParameters\n----------\nx: int\n blahx\n..."

However, this causes some trouble with other tools which use the whitespace differentiate between the parameter name and description to parse the docstrings. For instance, the parser in mkdocstrings and pytkdocs can no longer function correctly with the whitespace stripped.

Would the behaviour be able to be changed to keep the indentation? To a user this isn't expected from reading the package documentation or by assuming how it will behave.

I believe it might not be too hard a change to make but don't know this package well enough to be certain. I'm happy to make a PR if this is agreeable and you point me in the direction of where would be a good place to start

Multiple inheritance

When having multiple inheritance (without properly using ABC or mixins #20 I found a strange behavior :

from custom_inherit import DocInheritMeta


class Parent(metaclass=DocInheritMeta(style="numpy_with_merge")):
    """Parent.

    Notes
    -----
    Blah
    """

    def meth(self, x, y=None):
        """
        Raises
        ------
        NotImplementedError"""
        raise NotImplementedError


class Child1(Parent):
    """Child1. 

    Methods
    -------
    meth: does something
    """

    def meth(self, x, y=None):
        """Method description. """
        return 0


class Child2(Parent):
    """Child 2.

    Methods
    -------
    blah : does not much
    """
    def blah(self):
        """Method blah."""
        pass

class GrandChild(Child1, Child2):
    """Grand Child."""
    pass
>>> GrandChild.__doc__
'Grand Child.\n\nMethods\n-------\nblah : does not much\nmeth: does something\n\nNotes\n-----\nBlah\nBlah\nBlah\nBlah'

The doc of the grand parent class gets repeated. Is that the correct behavior ?

Inherit docstrings from Zope Interfaces

Feature request: inherit docstrings from zope.interface. For example:

from custom_inherit import DocInheritMeta
from zope.interface import implementer, Interface

class IClass(Interface):
    x = Attribute("Variable x.  Insert this description to class documentation.")
    def __init__(x):
        """Constructor.  Sets `x`"""
    def power(p):
        """Return `x**p`"""

@implementer(IClass)
class Class(metadata=DocInheritMeta(style="numpy")):
    def __init__(self, x):
        self.x = x
    def power(self, p):
        return self.x**p

c = Class(x=5)

Not sure if anything else is needed from an interface perspective: I think DocInheritMeta can probably just check to see if Class has the appropriate attributes (set by the implementer decorator) and then extract the documentation using duck typing.

If you think this fits with the design of custom_inherit, I can look at submitting a PR.

Conda-Forge Package

I'm curious if you would be interested in making this package available on Anaconda distributions via Conda-Forge. Seeing as this package has no dependencies, packaging it there would be a breeze, allowing for users to leverage it within Anaconda/Miniconda environments without relying on pip.

I can guide you through the process if this helps.

Merging a google docstring with a numpy docstring

Hey. Great tool, thanks a lot!

I was wondering if we can merge docstring of different kinds.

"""This is a test."""
import sklearn
import numpy
from typing import Optional
from custom_inherit import doc_inherit


class MyDecisionTreeClassifier(sklearn.tree.DecisionTreeClassifier):
    @doc_inherit(sklearn.tree.DecisionTreeClassifier.predict_proba, style="google_with_merge")
    def predict_proba(
        self,
        X: numpy.ndarray,
        check_input: Optional[bool] = True,
        my_option: Optional[bool] = False,
    ) -> numpy.ndarray:
        """Predict class probabilities of the input samples X.

        Args:
            my_option: If True, this is happening. If False, this other thing is happening and I
                need two lines to explain this option.

        Returns:
            numpy.ndarray: the result

        """
        print(my_option)
        super.predict_proba(X, check_input=check_input)

Here, I'd like to merge scikit docstring (which is numpy) with mine (which is google). It more or less works already:

Capture d’écran 2022-05-03 à 15 02 35

but you can see that there is a bad thing happening because my_option explanation is too long. More precisely, it cuts the line in two, and writes "option. (need two lines to explain this) –" as a fake new option.

I wonder if it can be fixed easily. If not, it is already a great tool thanks

Transferring ownership

Hi all - I am going to be transferring ownership of this project to my active GitHub account: @rsokl . I simply don't use this GitHub account anymore.

Allow docstring of __init__ to also inherit from the parent class

Hi all,

I noticed that, all methods starting and ending by "__" was kept off the docstring inheritance when the metaclass DocInheritMeta is used.

What is the reason for that? I would like the __init__ docstring also inherit from the parent class, could we imagine to have that as an option? is there any reason we should not always include it in the inheritance process?

Problem when merging with flake8

I don't know if this is more of an issue to do to flake8, but I try.

Still with the .py from #46:

"""This is a test."""
import sklearn
import numpy
from typing import Optional
from custom_inherit import doc_inherit


class MyDecisionTreeClassifier(sklearn.tree.DecisionTreeClassifier):
    @doc_inherit(sklearn.tree.DecisionTreeClassifier.predict_proba, style="google_with_merge")
    def predict_proba(
        self,
        X: numpy.ndarray,
        check_input: Optional[bool] = True,
        my_option: Optional[bool] = False,
    ) -> numpy.ndarray:
        """Predict class probabilities of the input samples X.

        Args:
            my_option: If True, this is happening. If False, this other thing is happening and I
                need two lines to explain this option.

        Returns:
            numpy.ndarray: the result

        """
        print(my_option)
        super.predict_proba(X, check_input=check_input)

When you flake8 it with flake8 src/concrete/ml/sklearn/d.py --config flake8_src.cfg with

[flake8]
max-line-length = 100
per-file-ignores = __init__.py:F401
extend-ignore = E203

I get

src/concrete/ml/sklearn/d.py:18:1: DAR101 Missing parameter(s) in Docstring: - X
src/concrete/ml/sklearn/d.py:18:1: DAR101 Missing parameter(s) in Docstring: - check_input
src/concrete/ml/sklearn/d.py:22:1: DAR202 Excess "Returns" in Docstring: + return

Then, I have to add a # noqa: DAR101 in the docstring, to make flake8 happy. But it is a bit ugly in the html then:

Capture d’écran 2022-05-03 à 15 14 25

class docstring with 'numpy_merge' section order

I found out that section order matters for 'numpy_merge' in class docstring :

from custom_inherit import DocInheritMeta

class Parent(metaclass=DocInheritMeta(style="numpy_with_merge")):
   """Parent summary.

   Methods
   -------
   meth
   """
   pass

class Child(Parent):
    """Child summary.
    
    Attributes
    ----------
    None: so far

    """
    pass

c = Child()
print(c.__doc__)

will return

Child summary.

Attributes
----------
None: so far

while, inverting the two sections

from custom_inherit import DocInheritMeta

class Parent(metaclass=DocInheritMeta(style="numpy_with_merge")):
   """Parent summary.

   Attributes
   ----------
   None: so far
   """
   pass

class Child(Parent):
    """Child summary.
    
    Methods
    -------
    meth
    """
    pass

c = Child()
print(c.__doc__)

will return the expected output :

Child summary.

Methods
-------
meth

Attributes
----------
None: so far

Is that the correct behavior ?

Within-Field Docstring Merging

I'm wondering if there is any way to handle a case where parameters or comments are composed from multiple sections.

It's always easier to understand with an example:

def some_operation(func):
    @doc_inherit(parent=func, style="numpy_napoleon")
    def wrapper( ... ):
        """
        Parameters
        ---------------
        x : str
          some string
        y : str
          some other string
        """
        < stuff happens >
        return
    return wrapper

@some_operation
def foo( ... ):
    """
    Parameters
    ---------------
    z : int
      some number
    """
    < stuff happens >

Then on inspection we get something resembling:

    Parameters
    ---------------
    x : str
      some string
    y : str
      some other string
    z : int
      some number

I can understand how technically challenging this could be but I'm wondering if this is a desired feature. Does this seem feasible?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.