Giter Club home page Giter Club logo

blacken-docs's Introduction

blacken-docs

image

image

image

image

pre-commit

Run Black on Python code blocks in documentation files.

Installation

Use pip:

python -m pip install blacken-docs

Python 3.8 to 3.12 supported.

Black 22.1.0+ supported.

pre-commit hook

You can also install blacken-docs as a pre-commit hook. Add the following to the repos section of your .pre-commit-config.yaml file (docs):

-   repo: https://github.com/adamchainz/blacken-docs
    rev: ""  # replace with latest tag on GitHub
    hooks:
    -   id: blacken-docs
        additional_dependencies:
        - black==22.12.0

Then, reformat your entire project:

pre-commit run blacken-docs --all-files

Since Black is a moving target, it’s best to pin it in additional_dependencies. Upgrade as appropriate.

Usage

blacken-docs is a command line tool that rewrites documentation files in place. It supports Markdown, reStructuredText, and LaTex files. Additionally, you can run it on Python files to reformat Markdown and reStructuredText within docstrings.

Run blacken-docs with the filenames to rewrite:

blacken-docs README.rst

If any file is modified, blacken-docs exits nonzero.

blacken-docs does not have any ability to recurse through directories. Use the pre-commit integration, globbing, or another technique for applying to many files. For example, with git ls-files | xargs_:

git ls-files -z -- '*.md' | xargs -0 blacken-docs

…or PowerShell’s ForEach-Object__:

git ls-files -- '*.md' | %{blacken-docs $_}

blacken-docs currently passes the following options through to Black:

It also has the below extra options:

  • --check - Don’t modify files but indicate when changes are necessary with a message and non-zero return code.
  • -E / --skip-errors - Don’t exit non-zero for errors from Black (normally syntax errors).
  • --rst-literal-blocks - Also format literal blocks in reStructuredText files (more below).

History

blacken-docs was created by Anthony Sottile in 2018. At the end of 2022, Adam Johnson took over maintenance.

Supported code block formats

blacken-docs formats code blocks matching the following patterns.

Markdown

In “python” blocks:

```python
def hello():
    print("hello world")
```

And “pycon” blocks:

```pycon

>>> def hello():
...     print("hello world")
...

```

Within Python files, docstrings that contain Markdown code blocks may be reformatted:

def f():
    """docstring here

    ```python
    print("hello world")
    ```
    """

reStructuredText

In “python” blocks:

.. code-block:: python

    def hello():
        print("hello world")

In “pycon” blocks:

.. code-block:: pycon

    >>> def hello():
    ...     print("hello world")
    ...

Use --rst-literal-blocks to also format literal blocks:

An example::

    def hello():
        print("hello world")

Literal blocks are marked with :: and can be any monospaced text by default. However Sphinx interprets them as Python code by default. If your project uses Sphinx and such a configuration, add --rst-literal-blocks to also format such blocks.

Within Python files, docstrings that contain reStructuredText code blocks may be reformatted:

def f():
    """docstring here

    .. code-block:: python

        print("hello world")
    """

LaTeX

In minted “python” blocks:

\begin{minted}{python}
def hello():
    print("hello world")
\end{minted}

In minted “pycon” blocks:

\begin{minted}{pycon}
>>> def hello():
...     print("hello world")
...
\end{minted}

In PythonTeX blocks:

\begin{pycode}
def hello():
    print("hello world")
\end{pycode}

blacken-docs's People

Contributors

adamchainz avatar andreablengino avatar aneeshusa avatar asottile avatar basnijholt avatar benjaoming avatar carltongibson avatar cmarqu avatar dependabot[bot] avatar harupy avatar jdufresne avatar jellezijlstra avatar joaquimesteves avatar lr4d avatar maxb2 avatar mxr avatar ndvanforeest avatar peterjc avatar philiptrauner avatar pre-commit-ci[bot] avatar proofit404 avatar spectre5 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

blacken-docs's Issues

--diff option

We have black checks in our CI to verify that any PRs/pushes are compliant. It would be nice to be able to include blacken-docs in these checks, but I don't want the CI steps to modify any code. Having a --diff option similar to black would be a nice feature that would show in our CI what needs to change.

Add option or switch default to use the same exit code behaviour as `black`

For my usage of blacken-docs I don't care whether or not any files were updated but I do want to be notified if any files could not be reformatted, whether that's due to a syntax error or anything else of that kind. This means that the --skip-errors flag doesn't work for my use case unfortunately.

Currently, blacken-docs exits with 1 if any files were reformatted or if there was an error processing the code block. It would be really helpful if there was an option to exit with 0 even if files were reformatted.

For context, the default black behaviour is to exit with 0 when files are reformatted and if there are any errors, it exits with 123.

Thanks!

Nested example failure

I was writing about using blacken-docs and it crashed like:

$ python -m blacken_docs example.rst
Traceback (most recent call last):
  File "/.../3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/.../3.10/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/.../site-packages/blacken_docs.py", line 248, in <module>
    exit(main())
  File "/.../site-packages/blacken_docs.py", line 243, in main
    retv |= format_file(filename, black_mode, skip_errors=args.skip_errors)
  File "/.../site-packages/blacken_docs.py", line 199, in format_file
    new_contents, errors = format_str(contents, black_mode)
  File "/.../site-packages/blacken_docs.py", line 186, in format_str
    src = RST_RE.sub(_rst_match, src)
  File "/.../site-packages/blacken_docs.py", line 106, in _rst_match
    min_indent = min(INDENT_RE.findall(match['code']))
ValueError: min() arg is an empty sequence

With this example.rst:

blacken-docs does changes like:

.. code-block:: diff

     .. code-block:: python

    -    name  = 'World'
    -    format_html('Hello <strong>{}</strong>',
    -        name
    -    )
    +    name = "World"
    +    format_html("Hello <strong>{}</strong>", name)

It seems blacken-docs picks up .. code-block:: python, but then INDENT_RE doesn't then due to diff indentation.

(I worked around this by changing the example to not include .. code-block:: python.)

Support pycon language in markdown.

It useful to write doctest inside markdown files.

    ```pycon

    >>> def foo ( bar,) : 
    ...     return ''

    ```

I saw it is supported for restructured text.

Maybe there is a relatively easy way to do the same for markdown?

Use Python 3.6+ features, e.g. Path

I see that this project can be upgraded to use latest Python 3.6+ features such as pathlib.Path instead of open (e.g. Path(..).read_text())

ignore jupyter magics

With jupytext one can keep notebooks as Markdown files.

Often one uses Jupyter magics (e.g., %%time), when running blacken-docs on the .md files I get many warnings like:

FILENAME_HERE.md:30: code block parse error Cannot parse: 1:0: %load_ext autoreload

Would it be possible to easily ignore those magics?

Take directories as arguments and search for files

In a CI context, it would be nice to be able to point the tool to folders and have it search for appropriate files itself while ignoring stuff in gitignore. Otherwise, in a CI scripting/tox/command runner context, you'll have to start making different commands for different shells recursively looking for files.

A `--check` mode would be useful

I like to run tests in CI against pull requests that fail if the user forgot to run Black or various other tools.

Black provides a useful --check option for this:

black --check .

It would be great if blacken-docs had the equivalent.

line number error with code-block:: python

I don't understand this one, with python code-block I get a line number error (using pre-commit) but if I remove the language the error goes away but I get no syntax highlighting? Example is here: https://github.com/sarnold/python-daemonizer

This generates an error:

.. code-block:: python

  from daemon import Daemon

  class pantalaimon(Daemon):
      def run(self):
          # Do stuff

but the same thing without the language does not:

.. code-block::

  from daemon import Daemon

  class pantalaimon(Daemon):
      def run(self):
          # Do stuff

The latter one highlights in sphinx docs but not on Github :(

UnboundLocalError: local variable 'indentation' referenced before assignment

$ blacken-docs t.rst 
Traceback (most recent call last):
  File "/tmp/c/venv/bin/blacken-docs", line 8, in <module>
    sys.exit(main())
  File "/tmp/c/venv/lib/python3.8/site-packages/blacken_docs.py", line 210, in main
    retv |= format_file(filename, black_mode, skip_errors=args.skip_errors)
  File "/tmp/c/venv/lib/python3.8/site-packages/blacken_docs.py", line 166, in format_file
    new_contents, errors = format_str(contents, black_mode)
  File "/tmp/c/venv/lib/python3.8/site-packages/blacken_docs.py", line 155, in format_str
    src = RST_PYCON_RE.sub(_rst_pycon_match, src)
  File "/tmp/c/venv/lib/python3.8/site-packages/blacken_docs.py", line 139, in _rst_pycon_match
    code += orig_line[indentation:] + '\n'
UnboundLocalError: local variable 'indentation' referenced before assignment
.. code-block:: pycon

    pass

This is clearly bad in that it's not actually a console session (no leading >>>), but it would be better to emit a warning and skip the block than to crash with an internal error.

pre-commit hook to ensure the pinned Black on blacken-docs' hook is the same as Black's rev

I wrote this for a project of mine:

    - repo: local
      hooks:
          - id: check-blacks-versions-matches-blacken-docs
            name: Check Black's versions matches blacken-doc's
            language: python
            entry: ./.git-hooks/check_blacks_versions_matches_blacken_docs.py
            additional_dependencies: [pyaml]
            files: ".pre-commit-config.yaml"
#!/usr/bin/env python3
import yaml


def find_first(f):
    def find_first_(list_):
        if not (
            found := next(
                (e for e in list_ if f(e)),
                None,
            )
        ):
            raise RuntimeError(f"No element matches with {f}")

        return found

    return find_first_


def has_entry(key, value):
    def has_entry_(dict_):
        return dict_.get(key, None) == value

    return has_entry_


def key(key):
    def key_(dict_):
        if not (found := dict_.get(key, None)):
            raise RuntimeError(f'No "{key}"')

        return found

    return key_


def pipe_value(v, *fs):
    a = v

    for i, f in enumerate(fs, 1):
        try:
            a = f(a)
        except Exception as e:
            raise RuntimeError(f"Exception on {i}th function") from e

    return a


with open(".pre-commit-config.yaml") as f:
    contents = yaml.load(f, Loader=yaml.FullLoader)

repos = key("repos")(contents)

black_rev = pipe_value(
    repos,
    find_first(has_entry("repo", "https://github.com/psf/black")),
    key("rev"),
)

pin_prefix = "black=="
blacks_pin = pipe_value(
    repos,
    find_first(has_entry("repo", "https://github.com/asottile/blacken-docs")),
    key("hooks"),
    find_first(has_entry("id", "blacken-docs")),
    key("additional_dependencies"),
    find_first(lambda e: e.startswith(pin_prefix)),
)

if blacks_pin != f"{pin_prefix}{black_rev}":
    raise RuntimeError(
        f"Black's pin in blackend-docs's additional_dependencies (`{blacks_pin}`) doesn't match black's repo rev (`{black_rev}`)."
    )

# All good, can't complain! 👍

This feels like something that most people would want. Maybe we want to get a variation of this in?

The code is a bunch unpythonic, maybe re-writing it is due.

If I use blacken-docs, does that mean I don't need to use black anymore

I see pre-committ-config.yaml is the configuration file where you can set the category of files that use blacken-docs

For example, the following file instructs blacken-docs to format files with the suffixes RST, md, py, markdown, Tex

repos:
  - repo: https://github.com/psf/black
    rev: stable  
    hooks:
      - id: black
        language_version: python3.7
  - repo: https://github.com/asottile/blacken-docs
    rev: master
    hooks:
    -   id: blacken-docs
        language: python
        files: '\.(rst|md|markdown|py|tex)$'

Does that mean I can remove black and just keep blacken-docs?
just like this:

repos:
  - repo: https://github.com/asottile/blacken-docs
    rev: master
    hooks:
    -   id: blacken-docs
        language: python
        files: '\.(rst|md|markdown|py|tex)$'

Support directory

black supports just giving a directory, and then automatically processes all files in that directory.

Would be nice if blacken-docs had that feature as well and behaved the same.

The docstring in the python file is not formatted by blacken-docs

Hi ,All
I have set up the format python file, and the docstring in the python file is not formatted by blacken-docs, why is that

This is my.pre-committ-config.yaml configuration file

repos:
  - repo: https://github.com/psf/black
    rev: stable  
    hooks:
      - id: black
        language_version: python3.7
        args: [--line-length=120]
        args: [--skip-string-normalization]

  - repo: https://github.com/asottile/blacken-docs
    rev: master
    hooks:
      - id: blacken-docs
        language: python
        args: [--line-length=120]
        args: [--skip-string-normalization]
        files: '\.(rst|md|markdown|py|tex)$'

  - repo: https://github.com/pre-commit/mirrors-isort
    rev: master  
    hooks:
      - id: isort

This is my Python file

#!/usr/bin/env python.exe
# -*- coding:utf-8 -*-
# @Author : huazai
# @Time : 2020/3/24 16:36
# @File : tt.py
# @Description :


import os
import re
import smtplib
import subprocess
import sys


def foo(a: str, b: str) -> str:
    c = None  # type:str
    c = a
    return c


class Zabbix_api():
    """Form a complex number.
    Keyword arguments:
    print(  len(  data))
    print(len(json.dumps(data)))
    aa= json.dumps(data)
    print(aa   )
    """

As you can see, after executing blacken-docs, the docstring in the Zabbix_api class is not formatted.

Thanks.

VS Code extension?

Thanks a lot for this project!

Is there any VS Code extension existing for that already? Otherwise, happy to dig into doing one!

Incompatible with black v22.1.0

I'm getting this after upgrading to black v22.1.0, using blacken-docs 1.12.0:

Traceback (most recent call last):
  File "/Users/xxx/.cache/pre-commit/repo2345cyqn/py_env-python3/bin/blacken-docs", line 8, in <module>
    sys.exit(main())
  File "/Users/xxx/.cache/pre-commit/repo2345cyqn/py_env-python3/lib/python3.8/site-packages/blacken_docs.py", line 238, in main
    black_mode = black.FileMode(
  File "<string>", line 3, in __init__
TypeError: set object expected; got list
Traceback (most recent call last):
  File "/Users/xxx/.cache/pre-commit/repo2345cyqn/py_env-python3/bin/blacken-docs", line 8, in <module>
    sys.exit(main())
  File "/Users/xxx/.cache/pre-commit/repo2345cyqn/py_env-python3/lib/python3.8/site-packages/blacken_docs.py", line 238, in main
    black_mode = black.FileMode(
  File "<string>", line 3, in __init__
TypeError: set object expected; got list

Add --check option

This is a great project!

I'd like my CI to fail if blacken-docs hasn't been run.

Black itself supports --check which behaves as follows:

--check      Don't write the files back, just return the
             status.  Return code 0 means nothing would
             change.  Return code 1 means some files
             would be reformatted. Return code 123 means
             there was an internal error

Would it be feasible to add something like this to blacken-docs?

Thanks!

Add option to format rST `::` literal code blocks

I have a project that uses Sphinx and we have a fair number of literal code blocks in our docs. I would rather not have to ask contributors to not use literal code blocks.

Would you be willing to add an option to format rST literal code blocks (or accept a PR adding such an option)?

--target-version results in KeyError: 'DOCS/INDEX.RST'

Tested 0.5.0 and master. Black 19.3b0.

When I run with the --target-version argument, I receive the following exception:

$ blacken-docs --target-version py33 docs/index.rst
Traceback (most recent call last):
  File ".../venv/bin/blacken-docs", line 11, in <module>
    load_entry_point('blacken-docs==0.5.0', 'console_scripts', 'blacken-docs')()
  File ".../venv/lib64/python3.7/site-packages/blacken_docs.py", line 114, in main
    args = parser.parse_args(argv)
  File "/usr/lib64/python3.7/argparse.py", line 1749, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib64/python3.7/argparse.py", line 1781, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib64/python3.7/argparse.py", line 1987, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/usr/lib64/python3.7/argparse.py", line 1927, in consume_optional
    take_action(action, args, option_string)
  File "/usr/lib64/python3.7/argparse.py", line 1839, in take_action
    argument_values = self._get_values(action, argument_strings)
  File "/usr/lib64/python3.7/argparse.py", line 2387, in _get_values
    value = [self._get_value(action, v) for v in arg_strings]
  File "/usr/lib64/python3.7/argparse.py", line 2387, in <listcomp>
    value = [self._get_value(action, v) for v in arg_strings]
  File "/usr/lib64/python3.7/argparse.py", line 2402, in _get_value
    result = type_func(arg_string)
  File ".../venv/lib64/python3.7/site-packages/blacken_docs.py", line 103, in <lambda>
    type=lambda v: black.TargetVersion[v.upper()],
  File "/usr/lib64/python3.7/enum.py", line 352, in __getitem__
    return cls._member_map_[name]
KeyError: 'DOCS/INDEX.RST'

When I run the same command without the --target-version argument, the command works as expected.

Issue with atypical Latex indentation

e.g. https://github.com/peterjc/biopython/blob/biopython-175/Doc/Tutorial/chapter_phenotype.tex

Reduced to a smaller test case, note the inconsistent indentation of lines 4 and 6 (which LaTeX does not care about):

\begin{description}
  \item[Well identifier]
    If you know the well identifier (row + column identifiers) you can access the desired well directly.
\begin{minted}{pycon}
    >>> record["A02"]
    \end{minted}

  \item[Well plate coordinates]
    The same well can be retrieved by using the row and columns numbers (0-based index).

%doctest examples lib:numpy
\begin{minted}{pycon}
>>> from Bio import phenotype
>>> record = list(phenotype.parse("Plates.csv", "pm-csv"))[-1]
>>> print(record[0, 1].id)
A02
\end{minted}

more text here...

\end{description}

The output:

\begin{description}
  \item[Well identifier]
    If you know the well identifier (row + column identifiers) you can access the desired well directly.
\begin{minted}{pycon}
>>> record["A02"]
\end{minted}

tem[Well plate coordinates]
The same well can be retrieved by using the row and columns numbers (0-based index).

test examples lib:numpy
in{minted}{pycon}
>>> from Bio import phenotype
>>> record = list(phenotype.parse("Plates.csv", "pm-csv"))[-1]
>>> print(record[0, 1].id)

\end{minted}

more text here...

\end{description}

Notice line 8 onwards (after the end of the code snippet) has lost the first few characters, it should still start \item

I suspect the issue is the two code blocks with text in between being treated as one code block?

Brakes docstrings when triple single quote (''') is used inside code block

Not sure whether this is an issue of blacken, but I found an interesting example that breaks my code. Basically, I have a function with a docstring that contains an example of a class you can pass to that function as an argument. The important thing is that inside this function I'm extracting the docstring of the class, so I also wanted to include a docstring in that example.

I guess you can see where this is going, here is an example:

def foo(data):
    """Docstring of foo...

    Example:
        ```python
        class Data:
            '''Example docstring of Data'''
            ...
        ```
    """

Blacken will try to format it and change the inner ''' to """, which breaks the outer docstring.

When I use ''' for the outer docstring, black won't change it to """ because it realizes that would break the string. So I guess that's the cleanest solution for now. Any better ideas?

Parse error for >>> prompt in code examples

Sphinx supports docs code examples with a prompt and output, as in the example from http://www.sphinx-doc.org/en/stable/markup/code.html

>>> 1 + 1
2

Such code blocks are present in most docs files in the projects I work on, and blacken-docs gives a "code block parse error" for those and I think skips them completely, so doesn't work for me (and I think many other people / projects) at the moment.

There must be some code in sphinx to parse those out, and to syntax highlight the rest of the code, so it should be possible to import and use that part of sphinx and only process the code after stripping the >>> out.

using black config file?

couldnt tell from documentation but does by default will be blacken-docs use projects black config within pyproject.toml example

# pyproject.toml
[tool.black]
line-length = 79

Tool doesn't honor black preferences

I've adopted blacken-docs as a pre-commit hook on my projects, but today blacken-docs gave me trouble by performing string normalization where I prefer it not be:

pip-run master $ git clone gh://jaraco/pip-run
Cloning into 'pip-run'...
remote: Enumerating objects: 466, done.
remote: Counting objects: 100% (466/466), done.
remote: Compressing objects: 100% (265/265), done.
remote: Total 1785 (delta 278), reused 369 (delta 193), pack-reused 1319
Receiving objects: 100% (1785/1785), 290.69 KiB | 4.21 MiB/s, done.
Resolving deltas: 100% (1050/1050), done.
pip-run master $ cd pip-run
pip-run master $ git checkout d17b4e8
Note: switching to 'd17b4e8'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at d17b4e8 Improve syntax in readme
pip-run HEAD $ pre-commit install
pre-commit installed at .git/hooks/pre-commit
pip-run HEAD $ pre-commit run
black................................................(no files to check)Skipped
blacken-docs.........................................(no files to check)Skipped
pip-run HEAD $ pre-commit run --all
black....................................................................Passed
blacken-docs.............................................................Failed
- hook id: blacken-docs
- exit code: 1
- files were modified by this hook

README.rst: Rewriting...

pip-run HEAD $ git diff
diff --git a/README.rst b/README.rst
index 87c34d9..6fa9b2e 100644
--- a/README.rst
+++ b/README.rst
@@ -177,11 +177,11 @@ First, add a ``__requires__`` directive at the head of the script:
 
     #!/usr/bin/env python
 
-    __requires__ = ['requests']
+    __requires__ = ["requests"]
 
     import requests
 
-    req = requests.get('https://pypi.org/project/pip-run')
+    req = requests.get("https://pypi.org/project/pip-run")
     print(req.status_code)
 
 Then, simply invoke that script with pip-run::
@@ -199,10 +199,11 @@ allowing a script to specify a custom package index:
 
     #!/usr/bin/env python
 
-    __requires__ = ['my_private_package']
-    __index_url__ = 'https://my.private.index/'
+    __requires__ = ["my_private_package"]
+    __index_url__ = "https://my.private.index/"
 
     import my_private_package
+
     ...
 
 Supplying parameters to Pip

pip-run HEAD $ head -n 6 pyproject.toml | tail -n 2
[tool.black]
skip-string-normalization = true

Is it by design that the configuration for black is different than that for blacken-docs? Is there a way that blacken-docs could be configured to honor the black config?

Internal assertion error on `...`-elided traceback blocks

$ blacken-docs t.rst
Traceback (most recent call last):
  File "/tmp/c/venv/bin/blacken-docs", line 8, in <module>
    sys.exit(main())
  File "/tmp/c/venv/lib/python3.8/site-packages/blacken_docs.py", line 210, in main
    retv |= format_file(filename, black_mode, skip_errors=args.skip_errors)
  File "/tmp/c/venv/lib/python3.8/site-packages/blacken_docs.py", line 166, in format_file
    new_contents, errors = format_str(contents, black_mode)
  File "/tmp/c/venv/lib/python3.8/site-packages/blacken_docs.py", line 155, in format_str
    src = RST_PYCON_RE.sub(_rst_pycon_match, src)
  File "/tmp/c/venv/lib/python3.8/site-packages/blacken_docs.py", line 131, in _rst_pycon_match
    assert fragment is not None
AssertionError
.. code-block:: pycon

    >>> 1/0
    Traceback (most recent call last):
        ...
    ZeroDivisionError: division by zero

https://github.com/asottile/blacken-docs/blob/master/blacken_docs.py#L130-L134

            if continuation_match:
                assert fragment is not None
                fragment += line[continuation_match.end():] + '\n'
            else:
                finish_fragment()

I think this could just translate to if continuation_match and fragment is not None: so that ... lines in output are not treated as continuations.

Support Markdown include syntax ({! myfile.py !})

The "include" syntax should be skipped instead of bailing out:

docs/howtos/authenticate-web-ui-users.md:71: code block parse error Cannot parse: 1:1: {! docs/howtos/aiohttp-oauth-redirect-example.py !}

Background: we "include" longer Python files (which are separately formatted with black) in Markdown with the mdx_include markdown extension.

My proposal would be to detect code blocks which just contain such include syntax ({! ... !}) and ignore them.

IPython shell examples

I tend to use IPython shell examples rather than the plain Python console, and I'd like blacken-docs to format them, e.g.

 .. code-block:: ipython

-    In [1]: print('Hello World')
+    In [1]: print("Hello World")
     Hello World

     In [2]: print(
-       ...: 'Hello World'
+       ...: "Hello World"
        ...: )
     Hello World

Would this be an accepted feature? Happy to give it a shot if so.

Handle or ignore interactive RST Python blocks with `>>>` and output

Sphinx allows interactive Python blocks to be displayed by using the >>> token to indicate user input and having the remaining output represent the response of the program. This is potentially rendered in RST. When running blacken-docs on RST files that have these interactive blocks, a parse error is when trying to parse the line. Easiest handling of this is probably to skip formatting these blocks if they have a >>> token.

Documentation on Sphinx feature: https://www.sphinx-doc.org/en/1.4.9/markup/code.html

Error with black==22.1.0

My pre-commit config:

  - repo: https://github.com/asottile/blacken-docs
    rev: v1.12.0
    hooks:
      - id: blacken-docs
        additional_dependencies: [ black==22.1.0 ]

Error:

blacken-docs.............................................................Failed
- hook id: blacken-docs
- exit code: 1

Traceback (most recent call last):
  File "/tmp/ppfeufer/.cache/pre-commit/repoh7brsv91/py_env-python3/bin/blacken-docs", line 8, in <module>
    sys.exit(main())
  File "/tmp/ppfeufer/.cache/pre-commit/repoh7brsv91/py_env-python3/lib/python3.9/site-packages/blacken_docs.py", line 238, in main
    black_mode = black.FileMode(
  File "<string>", line 3, in __init__
TypeError: set object expected; got list

Works perfectly fine with ``black==21.12b0`

pip show black:

Name: black
Version: 22.1.0
Summary: The uncompromising code formatter.
Home-page: https://github.com/psf/black
Author: Łukasz Langa
Author-email: [email protected]
License: MIT
Location: /mnt/sda1/Development/Python/AllianceAuth/venv-3.9/lib/python3.9/site-packages
Requires: click, mypy-extensions, pathspec, platformdirs, tomli, typing-extensions
Required-by: 

blacken-docs crashes with black==19.3b0

$ pip freeze | grep black
black==19.3b0
blacken-docs==0.4.0

$ cat example.md
image

$ blacken-docs example.md
Traceback (most recent call last):
  File "/Users/mxr/.pyenv/versions/py3.6.5/bin/blacken-docs", line 11, in <module>
    sys.exit(main())
  File "/Users/mxr/.pyenv/versions/3.6.5/envs/py3.6.5/lib/python3.6/site-packages/blacken_docs.py", line 109, in main
    'mode': black.FileMode.AUTO_DETECT,
AttributeError: type object 'FileMode' has no attribute 'AUTO_DETECT'

This looks to have changed in psf/black#618. Per the docs we should just not specify the mode to get auto-detect behavior (see here). I can try to take a stab at this in the next few days.

Poetry Refuses to add blacken-docs

I keep getting this:

>>> poetry add -D blacken-docs

Using version ^1.12.0 for blacken-docs

Updating dependencies
Resolving dependencies...

[TypeError]
unhashable type: 'VersionUnion'

async/await keywords not recognized in code block?

Hi,

My documentation looks like this:

.. code-block:: python

    async with aiohttp.ClientSession() as session:
        gh = GitHubAPI(session, "mariatta", oauth_token=os.getenv("GH_AUTH"))

When I run blacken-docs, I'm getting this error:

code block parse error Cannot parse: 1:6: async with aiohttp.ClientSession() as session:

But when I removed the async keyword, it works:

.. code-block:: python

    with aiohttp.ClientSession() as session:
        gh = GitHubAPI(session, "mariatta", oauth_token=os.getenv("GH_AUTH"))

Same problem when I have this type of code block:

.. code-block:: python

    await gh.patch(url, data={"state": "closed"})

Removing the await keyword works.
Any idea?

Thanks.

`blacken-docs` removes empty lines between methods.

Hello,

I hit an interesting behavior in markdown documents when formatting doctest examples in the pycon code blocks.

Let say we have this python code in the usage.py file:

class Robot:

    def __init__(self, servo, controller, settings):
        self.servo = servo
        self.controller = controller
        self.settings = settings

    def run(self):
        while True:
            events = self.accept_events()
            if not events:
                break
            self.process(events)

    def accept_events(self):
        # We can inject methods.
        return self.controller()

    def process(self, events):
        # We can inject dictionaries.
        max_point = self.settings["max_point"]
        for event in events:
            if event.x > max_point:
                # We can inject objects.
                self.servo.reverse("x")
            if event.y > max_point:
                self.servo.reverse("y")

Then I apply black to this file it will produce this change:

$ black usage.py
diff --git a/usage.py b/usage.py
index 1789079..b63d5ff 100644
--- a/usage.py
+++ b/usage.py
@@ -1,5 +1,4 @@
 class Robot:
-
     def __init__(self, servo, controller, settings):
         self.servo = servo
         self.controller = controller
@@ -25,4 +24,3 @@ class Robot:
                 self.servo.reverse("x")
             if event.y > max_point:
                 self.servo.reverse("y")
-

So, let's say we include the same code snippet in the markdown document usage.md:

```pycon

>>> class Robot:
...
...     def __init__(self, servo, controller, settings):
...         self.servo = servo
...         self.controller = controller
...         self.settings = settings
...
...     def run(self):
...         while True:
...             events = self.accept_events()
...             if not events:
...                 break
...             self.process(events)
...
...     def accept_events(self):
...         # We can inject methods.
...         return self.controller()
...
...     def process(self, events):
...         # We can inject dictionaries.
...         max_point = self.settings["max_point"]
...         for event in events:
...             if event.x > max_point:
...                 # We can inject objects.
...                 self.servo.reverse("x")
...             if event.y > max_point:
...                 self.servo.reverse("y")
...

```

When I apply blacken-docs to the markdown file it will remove empty lines between methods:

$ blacken-docs usage.md
diff --git a/usage.md b/usage.md
index 7c38b1b..ba2566f 100644
--- a/usage.md
+++ b/usage.md
@@ -1,23 +1,19 @@
 ```pycon

 >>> class Robot:
-...
 ...     def __init__(self, servo, controller, settings):
 ...         self.servo = servo
 ...         self.controller = controller
 ...         self.settings = settings
-...
 ...     def run(self):
 ...         while True:
 ...             events = self.accept_events()
 ...             if not events:
 ...                 break
 ...             self.process(events)
-...
 ...     def accept_events(self):
 ...         # We can inject methods.
 ...         return self.controller()
-...
 ...     def process(self, events):
 ...         # We can inject dictionaries.
 ...         max_point = self.settings["max_point"]

This change does not seams right.

You can use one of my projects if you need to test on real repository: https://github.com/proofit404/dependencies

Thoughts?

Best regards,
Artem.

Breaks on hard tab indents in rST

reStructuredText allows for using tabs for indenting code-blocks.

Traceback (most recent call last):
  File "~/.local/bin/blacken-docs", line 8, in <module>
    sys.exit(main())
  File "~/.local/lib/python3.10/site-packages/blacken_docs.py", line 246, in main
    retv |= format_file(filename, black_mode, skip_errors=args.skip_errors)
  File "~/.local/lib/python3.10/site-packages/blacken_docs.py", line 202, in format_file
    new_contents, errors = format_str(contents, black_mode)
  File "~/.local/lib/python3.10/site-packages/blacken_docs.py", line 189, in format_str
    src = RST_RE.sub(_rst_match, src)
  File "~/.local/lib/python3.10/site-packages/blacken_docs.py", line 109, in _rst_match
    min_indent = min(INDENT_RE.findall(match['code']))
ValueError: min() arg is an empty sequence

Eliminate trailing whitespace

Does (should?) blacken-docs eleimnated trailing whitespace in .md files?

I did run blacken-docs index.md on a file with some trailing space with no effect.

Eats rST required blank lines if trailing whitespace present

Input

.. code-block:: python
    
    foo()
    
and that's it

Note the blank lines have trailing whitespace.

Expected output

.. code-block:: python

    foo()

and that's it

(The trailing whitespace should be removed, and the blank lines should be preserved.)

Actual output

.. code-block:: python
    foo()
and that's it

(This is invalid reStructuredText.)

Does this project format doctests?

I have a number of doctests embedded in normal docstrings (i.e. these aren't formatted in a special way, like with Markdown)--I'm wondering if this project is intended to format these tests? In brief testing, it doesn't seem like blacken-docs picks up the doctests.

Format docstrings

It would be grand if this could format code after :: or .. code-block:: python within Python docstrings.

Support --check flag.

black supports --check flag, so it could be used on CI.

Do you think it worth to be supported?

possible issue in RST file

Looks like following example is not refactored in RST file:

..  code-block:: python

    import responses

    @responses.activate
    def test_simple():
        # Register via 'Response' object
        responses.add(
        responses.Response(
            method='PUT',
            url='http://example.com',
        )
    )
        # register via direct arguments
        responses.add(
        responses.GET,
        'http://twitter.com/api/1/foobar',
                      json={'error': 'not found'}, status=404)

if we take following python file:

import responses

@responses.activate
def test_simple():
    # Register via 'Response' object
    responses.add(
    responses.Response(
        method='PUT',
        url='http://example.com',
    )
)
    # register via direct arguments
    responses.add(
    responses.GET,
    'http://twitter.com/api/1/foobar',
                  json={'error': 'not found'}, status=404)

then black reformats it to:

import responses


@responses.activate
def test_simple():
    # Register via 'Response' object
    responses.add(
        responses.Response(
            method="PUT",
            url="http://example.com",
        )
    )
    # register via direct arguments
    responses.add(
        responses.GET,
        "http://twitter.com/api/1/foobar",
        json={"error": "not found"},
        status=404,
    )

image

Test-Docs folder

I didn't see any contributing guidelines, and so I submitted a PR. That was closed.
Searching through other closed PRs, I am submitting an issue first.

I want to suggest adding a doc-test folder, so that users can run the blacken-docs command in this repo to test black's formatter in specific file types.

pass --check option to black

black supports --check option, useful for CI:

--check                           Don't write the files back, just return the
                                  status.  Return code 0 means nothing would
                                  change.  Return code 1 means some files
                                  would be reformatted.  Return code 123 means
                                  there was an internal error.

I've read the discussion in #3. It is helpful, but I believe there should be a way to use blacken-docs in CI without depending on pre-commit tool.

Are you open to this feature?

Support tabbed indentation

Hi, thx for this nice package 🐰 recently we crossed difficulty, hard to decode where the code issue is
(I think that we have some problem in our code sample in docs, but no idea where...)

- hook id: blacken-docs
- exit code: 1

Traceback (most recent call last):
  File "/pc/clone/_hcfpzVEQoOuvvQz-hvKAg/py_env-python3/bin/blacken-docs", line 8, in <module>
    sys.exit(main())
  File "/pc/clone/_hcfpzVEQoOuvvQz-hvKAg/py_env-python3/lib/python3.8/site-packages/blacken_docs.py", line 231, in main
    retv |= format_file(filename, black_mode, skip_errors=args.skip_errors)
  File "/pc/clone/_hcfpzVEQoOuvvQz-hvKAg/py_env-python3/lib/python3.8/site-packages/blacken_docs.py", line 187, in format_file
    new_contents, errors = format_str(contents, black_mode)
  File "/pc/clone/_hcfpzVEQoOuvvQz-hvKAg/py_env-python3/lib/python3.8/site-packages/blacken_docs.py", line 175, in format_str
    src = RST_RE.sub(_rst_match, src)
  File "/pc/clone/_hcfpzVEQoOuvvQz-hvKAg/py_env-python3/lib/python3.8/site-packages/blacken_docs.py", line 100, in _rst_match
    min_indent = min(INDENT_RE.findall(match['code']))
ValueError: min() arg is an empty sequence

we are running it on this repo/PR Lightning-Universe/lightning-flash#635
and the pre-commit bot results https://results.pre-commit.ci/run/github/333857397/1628203901.qtZyKfFxSMOJBpcB4hdu1Q

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.