typeddjango / pytest-mypy-plugins Goto Github PK

View Code? Open in Web Editor NEW

92.0 6.0 26.0 187 KB

pytest plugin for testing mypy types, stubs, and plugins

Home Page: https://pypi.org/project/pytest-mypy-plugins/

License: MIT License

Python 99.31% Shell 0.69%

pytest mypy python pep484 pytest-plugin mypy-plugins mypy-stubs

pytest-mypy-plugins's People

Contributors

Stargazers

Watchers

pytest-mypy-plugins's Issues

Test that snippet passes

In mypy internal testing suite I can omit output to indicate that I expect snippet to pass

[case import]
from collections import namedtuple
[out]

This is very useful for basics checks.

However, I cannot get similar behaviour here. If I create a package

├── mypackage
│   └── __init__.py
├── mypy.ini
└── test
    └── test_mypackage.yaml

with

# ./mypackage/__init__.py
def bar(x: int) -> None:
    pass

and

./test/test_mypackage.yaml
- case: test_mypackage
  main: |
    from mypackage import foo

tests run without errors (pytest-mypy-plugins==1.9.1, mypy==0.910) despite incorrect import foo.

I have to put some output check

./test/test_mypackage.yaml
- case: test_mypackage
  main: |
    from mypackage import foo      
    reveal_type(True)  # N: Revealed type is "Literal[True]?"

to get an exception.

I also tried using empty out block

- case: test_mypackage
  main: |
    from mypackage import foo      
    reveal_type(True)
  out: ""

but it still silently ignores the broken import.

Cut down on noise in expected strings

Many times the full expectation string is long and includes low value information. I'd like to introduce some way to cut down on that noise. One thought is to make the expected string a fullmatch regex. This could also handle #45.

- case: test_logic_promotion
  main: |
    from hdltypes.logic import StdLogic, X01Z, Bit

    a: StdLogic
    b: X01Z
    c: Bit

    reveal_type(c & a) # N: .* "hdltypes.logic.StdLogic(?:\\*|`-?\\d+)"
    reveal_type(c & b) # N: .* "hdltypes.logic.X01Z(?:\\*|`-?\\d+)"
    reveal_type(c & c) # N: .* "hdltypes.logic.Bit(?:\\*|`-?\\d+)"

It might also be possible to make the expected strings templates to enable reuse, and perhaps package some of the common patterns with the tool.

- case: test_logic_promotion
  main: |
    # ...

    reveal_type(c & a) # N: {reveal} "hdltypes.logic.StdLogic{_}"
    reveal_type(c & b) # N: {reveal} "hdltypes.logic.X01Z{_}"
    reveal_type(c & c) # N: {reveal} "hdltypes.logic.Bit{_}"
  meta:
    reveal: "Revealed type is"
    _: "(?:\\*|`-?\\d+)"

The language here is strings, so using the common tools for that: regexes and templates, makes a lot of sense.

Fail fast on duplicated parameterized entries

Ref: kornicameister/loguru-mypy#31 (comment)

Idea here is to check if parametrized contains a duplicated dictionaries and if so fail fast the test.
Problem though is that pytest behavior on parameterized is different...basically pytest does not seem to care if you have duplicates.

Thing to think about for a minute, I guess :)

ERROR pytest.PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlTestFile is deprecated. Please use the (path: pathlib.Path) argument instead.

Hello !

It looks like this library is using a deprecated pytest function argument, that makes tests fail when warnings are not allowed in the testsuite.

See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path

==================================== ERRORS ====================================
______________________ ERROR collecting tests/__init__.py ______________________
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/pluggy/_hooks.py:265: in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/pluggy/_manager.py:80: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/pytest_mypy_plugins/collect.py:143: in pytest_collect_file
    return YamlTestFile.from_parent(parent, fspath=path)
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/_pytest/nodes.py:630: in from_parent
    return super().from_parent(parent=parent, fspath=fspath, path=path, **kw)
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/_pytest/nodes.py:261: in from_parent
    return cls._create(parent=parent, **kw)
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/_pytest/nodes.py:140: in _create
    return super().__call__(*k, **kw)
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/_pytest/nodes.py:585: in __init__
    path = _imply_path(type(self), path, fspath=fspath)
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/_pytest/nodes.py:114: in _imply_path
    stacklevel=3,
E   pytest.PytestRemovedIn8Warning: The (fspath: py.path.local) argument to YamlTestFile is deprecated. Please use the (path: pathlib.Path) argument instead.
E   See https://docs.pytest.org/en/latest/deprecations.html#fspath-argument-for-node-constructors-replaced-with-pathlib-path
=========================== short test summary info ============================
ERROR tests/__init__.py - pytest.PytestRemovedIn8Warning: The (fspath: py.pat...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.15s ===============================

PYTHONPATH and MYPYPATH cannot be specified in `env:` directive

Currently this code does not work:

-   case: mypy_path_from_env
    main: |
        from pair import Pair

        a: Pair
        reveal_type(a)  # N: Revealed type is 'pair.Pair'
    env:
        - MYPYPATH=./pytest_mypy_plugins/tests/fixtures

Because currently pytest-mypy-plugins does not respect this way of defining variables.

Problem with pytest-xdist and cache

Sometimes, when we have both cache and xdist, this might happen:

[gw1] linux -- Python 3.8.5 /home/runner/work/returns/returns/.venv/bin/python
Traceback (most recent call last):
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 294, in from_call
    result = func()  # type: Optional[_T]
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 247, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 161, in pytest_runtest_call
    raise e
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 153, in pytest_runtest_call
    item.runtest()
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/pytest_mypy_plugins/item.py", line 262, in runtest
    dependants = self.find_dependent_paths(path)
  File "/home/runner/work/returns/returns/.venv/lib/python3.8/site-packages/pytest_mypy_plugins/item.py", line 165, in find_dependent_paths
    if f'"{py_module}"' in path.read_text():
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/pathlib.py", line 1232, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/pathlib.py", line 1218, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/pathlib.py", line 1074, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/.mypy_cache/3.8/@plugins_snapshot.json.484c4696e62da2ed'

Build: https://github.com/dry-python/returns/pull/581/checks?check_run_id=1075226140#step:7:557

Add option to show traceback

Hi there! Thanks for this plugin, it has been really helpful, sometimes I get issues on CI that I don't get locally, for example:

E   pytest_mypy_plugins.utils.TypecheckAssertionError: Critical error occurred

would be possible to get a traceback from this?

TypeError: filelocrepr was renamed reprfileloc

In pytest-dev/pytest#6739, filelocrepr was renamed reprfileloc which now breaks pytest-mypy-plugins with this error:

[...]/site-packages/pytest_mypy/item.py", line 308, in repr_failure
TypeError: __init__() got an unexpected keyword argument 'filelocrepr'

This should only affect pytest >= 5.4.0.

The fix seems to simply be renaming the variable in item.py to match the new pytest name.

Does not work with pytest-cov

» pytest
================================ test session starts =================================
platform darwin -- Python 3.7.3, pytest-4.6.3, py-1.7.0, pluggy-0.12.0
Using --randomly-seed=1560429839
rootdir: /Users/sobolev/Documents/github/returns, inifile: setup.cfg
plugins: mypy-plugins-0.3.0, asyncio-0.10.0, cov-2.7.1, randomly-3.0.0
collected 49 items / 3 errors / 46 selected                                          
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/_pytest/main.py", line 206, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/_pytest/main.py", line 250, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/pytest_cov/plugin.py", line 229, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/pytest_cov/engine.py", line 171, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/coverage/control.py", line 818, in combine
INTERNALERROR>     self.data, aliases=aliases, data_paths=data_paths, strict=strict,
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/coverage/data.py", line 736, in combine_parallel_data
INTERNALERROR>     data.update(new_data, aliases=aliases)
INTERNALERROR>   File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/coverage/data.py", line 488, in update
INTERNALERROR>     raise CoverageException("Can't combine line data with arc data")
INTERNALERROR> coverage.misc.CoverageException: Can't combine line data with arc data

Configuration: https://github.com/dry-python/returns/blob/master/setup.cfg

Test: test-data/typecheck/return.test

[CASE no_incompatible_meta_nested_class_false_positive]
1 + '2'
[/CASE]

Support for mypy .test files.

This is a feature request.

It might be nice to get support for handling mypy-style .test files:

[case foo]

reveal_type(1 + 1)  # N: Revealed type is "builtins.int"

[out]]

-- Comment

[case bar]

reveal_type(1 + 1)

[out]
main:2: note: Revealed type is "builtins.int"

Add option to disable "--no-silent-site-packages`

I've got the same problem shown in the comment #34 (comment)

I've seen the code and I believe that this line

pytest-mypy-plugins/pytest_mypy_plugins/item.py

Line 299 in 1c49b23

"--no-silence-site-packages",

causes mypy to type check all files in the site-packages.

I believe that for local testing, it would be beneficial to turn it off. WDYT?
Adding this feature would be quite easy. e.g. --mypy-only-local-stub opt

You can try out this feature here: https://github.com/yuyupopo/pytest-mypy-plugins

pystache is not maintained

pystache support stopped in 2014 without reaching a stable release. The project is unsupported, PRs are unmerged, issues unanswered, the only author doesn't have any activity on github since 2018. Depending on pystache is dangerous.

As an alternative, I'd recommend Jinja2. Famous, well supported, in good hands of a nice community, has similar but more powerful syntax. However, if you really want the mustache syntax, chevron is a good option: it is maintained and much faster.

Drop Python 3.6 support

Python 3.6 is going to reach the end of its lifespan in 2021-12, so it is probably time to drop it from supported versions.

diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index af29a97..5787a03 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -13,7 +13,7 @@ jobs:
 
     strategy:
       matrix:
-        python-version: [3.6, 3.7, 3.8, 3.9]
+        python-version: [3.7, 3.8, 3.9]
 
     steps:
     - uses: actions/checkout@v2
diff --git a/setup.py b/setup.py
index 1c021aa..e058e7e 100644
--- a/setup.py
+++ b/setup.py
@@ -10,7 +10,6 @@ dependencies = [
     "pyyaml",
     "chevron",
     "regex",
-    "dataclasses ; python_version<'3.7'",
 ]
 
 setup(
@@ -27,11 +26,10 @@ setup(
     # the following makes a plugin available to pytest
     entry_points={"pytest11": ["pytest-mypy-plugins = pytest_mypy_plugins.collect"]},
     install_requires=dependencies,
-    python_requires=">=3.6",
+    python_requires=">=3.7",
     classifiers=[
         "Development Status :: 3 - Alpha",
         "License :: OSI Approved :: MIT License",
-        "Programming Language :: Python :: 3.6",
         "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.9",

python_version is ignored, unable to test python 2 types

I've got a test case that looks like this:

- case: testAttrsNewStyleClassPy2
  mypy_config:
    python_version = 2.7
  main: |
    import attr
    @attr.s
    class Good(object):
        pass
    @attr.s
    class Bad:  # E: attrs only works with new-style classes
        pass

But it doesn't pass because the python_version = 2.7 line seems to be ignored. I believe it has to do with how mypy is run.
mypy --python-version={current_interpreter}. But as there's no way to run mypy with python 2, then there's no way to test python2 types.

Output broken for multiline messages

Given following test case

- case: break_following
  main: |
    from typing import overload, TypeVar

    T = TypeVar("T", int, str)
    
    @overload
    def f(x: int) -> int: ...
    @overload
    def f(x: str) -> str: ...
    def f(x: T) -> T:
      return x

    reveal_type(f([1, 2, 3]))  # N: Revealed type is "Any"

will get message as follows:

E   pytest_mypy_plugins.utils.TypecheckAssertionError: Invalid output: 
E   Expected:
E     <45 (diff)
E   Actual:
E     main:12: error: No overload variant of "f" matches argument type "List[int]" (diff)
E     main:12: note:     def f(x: int) -> int       (diff)
E     main:12: note:     def f(x: str) -> str       (diff)
E     main:12: note: Possible overload variants:    (diff)
E     main:12: note: Revealed type is "Any"         (diff)
E   
E   Alignment of first line difference:
E     E: main:12: note: Revealed type is "Any"...
E     A: main:12: error: No overload variant of "f" matches argument type "List[i...
E

while mypy will return

main.py:13: error: No overload variant of "f" matches argument type "List[int]"
main.py:13: note: Possible overload variants:
main.py:13: note:     def f(x: int) -> int
main.py:13: note:     def f(x: str) -> str
main.py:13: note: Revealed type is "Any"

Please note that

E     main:14: note: Possible overload variants:    (diff)

is misplaced compared to mypy output.

It happens, because sorting key includes actual error message:

pytest-mypy-plugins/pytest_mypy_plugins/utils.py

Line 209 in f7b249d

def extract_parts_as_tuple(line: str) -> Tuple[str, int, str]:

so sort is not stable within (fname, line_number).

Environment:

Python: 3.9.7
pytest-mypy-plugins: 1.9.1
mypy: 0.910

Testing local stubs

Hi, first of all, thank you for this project! Does pytest-mypy-plugins allow to test the local stubs? Supposing the project has the following structure:

root/
├──  src/
|   ├── foo/
|   |   └── __init__.py
|   └── ...
├── typeshed/
|   ├── bar/
|   |   └── __init__.pyi
|   └── baz.pyi
└── mypy.ini

with mypy.ini setting:

[mypy]
files = src/
mypy_path = typeshed/
...

I can't figure out how to pass this setting to the tests. AFAICT the tests are ran in a temporary directory, so a relative mypy_path will resolve against this temp dir, so only absolute paths would do. The --mypy-testing-base doesn't help either, raising on missing temporary directory (however, this may also emerge because I have misconfigured something):

Traceback (most recent call last):
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/_pytest/runner.py", line 237, in from_call
    result = func()
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/_pytest/runner.py", line 210, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/_pytest/runner.py", line 134, in pytest_runtest_call
    item.runtest()
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/pytest_mypy/item.py", line 219, in runtest
    mypy_cmd_options = self.prepare_mypy_cmd_options(execution_path)
  File "/home/hoefling/projects/.venvs/typeshed-py36/lib/python3.6/site-packages/pytest_mypy/item.py", line 285, in prepare_mypy_cmd_options
    with mypy_config_file_path.open('w') as f:
  File "/usr/lib64/python3.6/pathlib.py", line 1183, in open
    opener=self._opener)
  File "/usr/lib64/python3.6/pathlib.py", line 1037, in _opener
    return self._accessor.open(self, flags, mode)
  File "/usr/lib64/python3.6/pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)
FileNotFoundError: [Errno 2] No such file or directory: 'pytest-mypy-u4fvf1rh/mypy.ini'

Errors reported for 3rd-party dependencies

When I run type-checking of my code with mypy - everything works correctly.
But, when I try to run type-tests on some module, then it breaks, example:

E     ../../../../../../Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/hypothesis/vendor/pretty:706: error: Call to untyped function "_safe_getattr" in typed context (diff)

Setup black + isort

It is the same tooling we use in other TypedDjango projects.

Improve error message when parsing tests

Currently, when parsing error happens, you will end up with something like this:

  File "/Users/sobolev/Documents/github/returns/.venv/lib/python3.7/site-packages/pytest_mypy_plugins/utils.py", line 256, in assert_string_arrays_equal
    lineno = int(first_failure.split(" ")[0].strip(":").split(":")[1])
IndexError: list index out of range

We need to improve it!

Fix test ids, now they are all the same

Related pytest-dev/pytest-randomly#378

Output-file rewriting removes '.py' in too many places

The last line of pytest_mypy_plugins.items.replace_fpath_with_module_name() does a blanket removal of all ".py" strings:

return line.strip().replace(".py", "")

This destroys paths in log information when mypy.verbosity > 0. Paths like "$HOME/.pyenv" and ".../python3.6/.../__init__.pyi" are converted into random strings.

I suspect you are only trying to replace "main.py: 1: note: ..." -type strings at the beginning of the line. If that is true (and I really haven't explored enough to be sure, so please check this) then a simple replacement my be replace(".py:", ":") to ensure the offending .py looks like a filespec.

Otherwise maybe only do the replacement in the first colon-delimited field?

PyPI 1.9.2 release on 0d790e7?

It would be great if we could get a PyPI release on 0d790e7 :)

Ellipsis in `out:` section leads to IndexError

Running Debian linux 10.5 with Python 3.8.0b4, this test case:

# typesafety/test_ellipsis.yml
- case: test_out_section_containing_ellipsis
  main: |
    reveal_type(int("1"))
  out: |
    ...
    main:1: note: Revealed type is 'builtins.int'

Produces this error:

$ pytest typesafety/test_ellipsis.yml 
==================================================== test session starts =====================================================
platform linux -- Python 3.8.0b4, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/aghast/Code/aghast/aghast-utils, configfile: pyproject.toml
plugins: cov-2.10.1, mypy-plugins-1.4.0
collected 1 item                                                                                                             

typesafety/test_ellipsis.yml F                                                                                         [100%]

========================================================== FAILURES ==========================================================
____________________________________________ test_out_section_containing_ellipsis ____________________________________________
Traceback (most recent call last):
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/_pytest/runner.py", line 294, in from_call
    result = func()  # type: Optional[_T]
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/_pytest/runner.py", line 247, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/_pytest/runner.py", line 161, in pytest_runtest_call
    raise e
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/_pytest/runner.py", line 153, in pytest_runtest_call
    item.runtest()
  File "/home/aghast/.pyenv/versions/3.8.0b4/envs/venv38/lib/python3.8/site-packages/pytest_mypy_plugins/item.py", line 253, in runtest
    assert_string_arrays_equal(expected=self.expected_output_lines, actual=output_lines)
  File "/home/aghast/.pyenv/versions/venv38/lib/python3.8/site-packages/pytest_mypy_plugins/utils.py", line 256, in assert_string_arrays_equal
    lineno = int(first_failure.split(" ")[0].strip(":").split(":")[1])
IndexError: list index out of range
================================================== short test summary info ===================================================
FAILED typesafety/test_ellipsis.yml::test_out_section_containing_ellipsis - IndexError: list index out of range

pyproject.toml support

The latest release of mypy supports configuration in pyproject.toml. However, pytest-mypy-plugins seems to not support it. From a quick glance at the code, it looks like the library generates its own ini config, overriding the default configuration file lookup.

UnboundLocalError when --mypy-testing-base is not valid

If tests are invoked with --mypy-testing-base that is not valid (doesn't exist, is a file, user has no permissions to write) we fail and escalate to UnboundLocalError:

Traceback (most recent call last):
  File "/path/to/pytest-mypy-plugins/pytest_mypy_plugins/item.py", line 231, in runtest
    temp_dir = tempfile.TemporaryDirectory(prefix="pytest-mypy-", dir=self.root_directory)
  File "/usr/lib/python3.9/tempfile.py", line 918, in __init__
    self.name = mkdtemp(suffix, prefix, dir)
  File "/usr/lib/python3.9/tempfile.py", line 498, in mkdtemp
    _os.mkdir(file, 0o700)
NotADirectoryError: [Errno 20] Not a directory: '/tmp/foo/pytest-mypy-k7frqt99'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/path/to/lib/python3.9/site-packages/_pytest/runner.py", line 311, in from_call
    result: Optional[TResult] = func()
  File "/path/to/lib/python3.9/site-packages/_pytest/runner.py", line 255, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/path/to/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/path/to/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/path/to/lib/python3.9/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/path/to/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/path/to/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/path/to/lib/python3.9/site-packages/_pytest/runner.py", line 170, in pytest_runtest_call
    raise e
  File "/path/to/lib/python3.9/site-packages/_pytest/runner.py", line 162, in pytest_runtest_call
    item.runtest()
  File "/path/to/pytest-mypy-plugins/pytest_mypy_plugins/item.py", line 276, in runtest
    temp_dir.cleanup()
UnboundLocalError: local variable 'temp_dir' referenced before assignm

This is because we create temp directory in a try block

pytest-mypy-plugins/pytest_mypy_plugins/item.py

Lines 230 to 231 in 1ad40a9

 try: 

 temp_dir = tempfile.TemporaryDirectory(prefix="pytest-mypy-", dir=self.root_directory)

that corresponds finally block responsible for cleanup

pytest-mypy-plugins/pytest_mypy_plugins/item.py

Lines 275 to 276 in 1ad40a9

 finally: 

 temp_dir.cleanup()

Broken mypy cache?

I am trying to investigate some issues related to caching behavior. When testing project with complex dependencies, I see serious performance degradation (roughly 20 fold on just 36 tests) compared to using mypy.test.testcheckTypeCheckSuite directly.

I thought the issue was a simple logic mistake (#82), but it seems it might be actually with find_dependent_paths

pytest-mypy-plugins/pytest_mypy_plugins/item.py

Line 187 in a2d4add

py_module = ".".join(path.with_suffix("").parts)

Since it uses at least main.py it includes all kinds of packages using main as a name (not necessarily as a module name, could be even an argument), for example

['/tmp/.mypy_cache/3.9/pdb',
 '/tmp/.mypy_cache/3.9/unittest/main',
 '/tmp/.mypy_cache/3.9/unittest/__init__',
 '/tmp/.mypy_cache/3.9/_pytest/pytester',
 '/tmp/.mypy_cache/3.9/_pytest/config/__init__',
 '/tmp/.mypy_cache/3.9/pytest/__init__',
 '/tmp/.mypy_cache/3.9/asyncio/runners']

This seems to escalate (in my case, to numpy annotations, for reason yet to be determined), and break caching in general.

Possibly related to #37

Originally posted by @zero323 in #82 (comment)

Improve line matching behavior

This is a feature request.

Summary:

Currently rows are matched by their position in the file / output.

User experience could be improved, if matching was performed by (file, line-number).

Details:

Let's assume that I have a test case like this

- case: break_following_2
  main: |
    reveal_type(1 + 1) 
    reveal_type(1.0 + 2.0) # N: Revealed type is "builtins.float"
    reveal_type("foo" + "bar") # N: Revealed type is "builtins.str"

When I run tests I see:

E   pytest_mypy_plugins.utils.TypecheckAssertionError: Invalid output: 
E   Expected:
E     main:2: note: Revealed type is "builtins.float" (diff)
E     main:3: note: Revealed type is "builtins.str" (diff)
E   Actual:
E     main:1: note: Revealed type is "builtins.int" (diff)
E     main:2: note: Revealed type is "builtins.float" (diff)
E     main:3: note: Revealed type is "builtins.str" (diff)
E   
E   Alignment of first line difference:
E     E: main:2: note: Revealed type is "builtins.float"
E     A: main:1: note: Revealed type is "builtins.int"
E

If you analyze the test case, you'll see that actual state is like this:

line	actual	expected	match
1	Revealed type is "builtins.int"		✘
2	Revealed type is "builtins.float"	Revealed type is "builtins.float"	✓
3	Revealed type is "builtins.str"	Revealed type is "builtins.str"	✓

however alignment message

E   Alignment of first line difference:
E     E: main:2: note: Revealed type is "builtins.float"
E     A: main:1: note: Revealed type is "builtins.int"

clearly shows that we start with comparing line 2 of expected and line 1 of actual.

This escalates to all the following lines and probably gets worse with multi-line messages (I wanted to investigate that, hence #66).

I am aware that this is consistent with behavior of the internal mypy test suite, which returns

Expected:
  main:2: note: Revealed type is "builtins.float" (diff)
  main:3: note: Revealed type is "builtins.str" (diff)
Actual:
  main:1: note: Revealed type is "builtins.int" (diff)
  main:2: note: Revealed type is "builtins.float" (diff)
  main:3: note: Revealed type is "builtins.str" (diff)

Alignment of first line difference:
  E: main:2: note: Revealed type is "builtins.float"
  A: main:1: note: Revealed type is "builtins.int"
          ^

for equivalent input, but it seems a bit counter-intuitive. While it detects presence of output mismatch, it cannot do much beyond that.

Ideally, I'd like to see something around these lines:

E   pytest_mypy_plugins.utils.TypecheckAssertionError: Invalid output: 
E   Expected:
E     (Empty)
E      ...
E   Actual:
E     main:1: note: Revealed type is "builtins.int" (diff)
E     ...

(no alignment needed for an empty line).

This should generalize to multiple interleaved blocks of matching and not matching lines, where matching blocks are indicated, but omitted.

Furthermore, errors shouldn't propagate beyond current line, in case of multline output.

Originally posted by @zero323 in #65 (comment)

Add option to ignore `*` and `1 in types

There are cases when some types are revealed like:

# N: Revealed type is 'builtins.bool*'

Or like so:

reveal_type(dekind(container))  # N: Revealed type is 'main.IO[def (_ValueType`1) -> _NewValueType`-1]'

Here are the problems with these two things:

They can change! Sometimes if you use a different technique inside your type, you will loose this * while everything else will work the exact same way
Different versions of python in different environments might produce different numeric results after

char in type vars.

So, I propose to add strict: false parameter to our test cases to ignore these chars (and possibly other things in the future).

Does not work with [email protected]

Source code: https://github.com/dry-python/returns/tree/master/typesafety

Error:

/home/travis/build/dry-python/returns/typesafety/test_result_types/safe.test:8: 
E   pytest_mypy.utils.TypecheckAssertionError: Invalid output: 
E   Expected:
E     main:7: error: Revealed type is 'def () -> returns.result.Result[builtins.int*, builtins.Exception]' (diff)
E   Actual:
E     main:7: note: Revealed type is 'def () -> returns.result.Result[builtins.int*, builtins.Exception]' (diff)
E   
E   Alignment of first line difference:
E     E: main:7: error: Revealed type is 'def () -> returns.result.Result[builtin...
E     A: main:7: note: Revealed type is 'def () -> returns.result.Result[builtins...
E                ^

Link to PR: dry-python/returns#98
Link to CI: https://travis-ci.org/dry-python/returns/jobs/548053832

Remove usage of distutils

distutils are deprecated in Python 3.10 and going to be removed in 3.12 (PEP 632).

Currently distutils are used in pytest_mypy_plugins.item:

pytest-mypy-plugins/pytest_mypy_plugins/item.py

Line 199 in 6c769da

import distutils.spawn

to find mypy executable.

Cannot be used on Windows

capturer, a dependency of this project, cannot be used on Windows, as it clearly states in the first paragraph of the documentation.

Is capturer a hard requirement? As far as I can tell, it's only used in typecheck_in_same_process to capture mypy's output. However:

Both mypy.main.process_options and mypy.build.build accept an optional TextIO for stdout and stderr.
contextlib.redirect_stdout and contextlib.redirect_stderr could be used here, since nothing runs in a subprocess (as far as I can tell? otherwise I don't see how mypy could even reliably support the parameters above)

I'd be happy to submit a PR, unless I've missed something.

Incorrect output formatting for Expected

Expected block is not correctly formatted with all lines printed as <45 (diff). For example if I try to test:

- case: failing_case_1
  main: |
    reveal_type(42)  # N: Revealed type is "builtins.str"
    reveal_type("foo")  # N: Revealed type is "builtins.int"

I get

/path/to/test-file.yaml:3:
E   pytest_mypy_plugins.utils.TypecheckAssertionError: Invalid output: 
E   Expected:
E     <45 (diff)
E     <45 (diff)
E   Actual:
E     main:1: note: Revealed type is "Literal[42]?" (diff)
E     main:2: note: Revealed type is "Literal['foo']?" (diff)
E   
E   Alignment of first line difference:
E     E: ...te: Revealed type is "builtins.str"
E     A: ...te: Revealed type is "Literal[42]?"
E                                 ^

while I expect

/path/to/test-file.yaml:3:
E   pytest_mypy_plugins.utils.TypecheckAssertionError: Output is not expected: 
E   Actual:
E     main:1: note: Revealed type is "Literal[42]?" (diff)
E     main:2: note: Revealed type is "Literal['foo']?" (diff)
E   Expected:
E     main:1: note: Revealed type is "builtins.str" (diff)
E     main:2: note: Revealed type is "builtins.int" (diff)
E   Alignment of first line difference:
E     E: ...te: Revealed type is "builtins.str"
E     A: ...te: Revealed type is "Literal[42]?"

This happens because OutputMatcher is fed directly into str.format, instead of converting it first to str

pytest-mypy-plugins/pytest_mypy_plugins/utils.py

Line 254 in f7b249d

error_message += " {:<45} (diff)".format(expected[i])

See for example:

>>> from pytest_mypy_plugins.utils import OutputMatcher
>>>
>>> expected = OutputMatcher("foo", 1, "E", "bar", False)
>>> "  {:<45} (diff)".format(expected)
## '  <45 (diff)'
>>> "  {:<45} (diff)".format(str(expected)) 
'  foo:1: E: bar                                 (diff)'

Environment:

Python: 3.9.7
pytest-mypy-plugins: 1.9.1

Skipping tests conditionally

Let's think about how a test can be skipped. I suggest the following api:

- case: test_with_skip
   skip: python_value_or_expression_to_eval
   main: reveal_type('')  # N: builtins.str

This way we can skip some tests on different python versions or missing dependencies.

Eval context

The tricky part here is the eval context. What should be there?

http://doc.pytest.org/en/latest/skipping.html

Does not work with `[email protected]`

All types generate this error:

E     main:1: error: Cannot find implementation or library stub for module named 'returns.example' (diff)
E     main:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports (diff)
E     main:3: note: Revealed type is 'Any'          (diff)

This is a critical bug.

Setting `--python-version` overrides option in `--mypy-ini-file`

I'm running into an issue where running mypy gives no errors, but pytest-mypy-plugins gets mypy errors due to it overriding --python-version. I'm setting python_version = 3.7 for compatibility reasons, which causes an attribute error for types.UnionType which I ignore.

But because pytest-mypy-plugins overrides the python_version set in config, in my case to 3.10 this causes mypy to think that types.UnionType is always there and yield and warning for the now unused ignore comment.

Naively I can't figure out why pytest-mypy-plugins needs to set --python-version at all. Perhaps it's worth considering not doing that?

As a less intrusive way forward, perhaps we could make it so that --python-version is only set when no ini config file is given?

Allow to test docstrings

I have a popular case when I use docstrings to illustrate my examples right in the docs.
But, the problem is that cannot be checked.

I use regular --doctest-modules feature from pytest, but it is not enough for reveal_type(...)
We can probably use the same idea and to check cases in the docs.

All imported packages seem to be tested as well

When I run mypy on the following code, I get no errors:

import numpy

However, when I run the pytest-mypy-plugin on:

- case: test
  main: |
    import numpy
    reveal_type(1)  # N: Revealed type is "Literal[1]?"

I get a lot of errors about things that apparently go wrong within pytest and numpy:

E   pytest_mypy_plugins.utils.TypecheckAssertionError: Invalid output: 
E   Expected:
E     <45 (diff)
E   Actual:
E     ../../../../../../Users/nanneaben/opt/anaconda3/envs/tmp/lib/python3.8/site-packages/_pytest/_argcomplete:103: error: Cannot find implementation or library stub for module named "argcomplete" (diff)
E     ../../../../../../Users/nanneaben/opt/anaconda3/envs/tmp/lib/python3.8/site-packages/_pytest/_argcomplete:103: error: Cannot find implementation or library stub for module named "argcomplete.completers" (diff)
E     ../../../../../../Users/nanneaben/opt/anaconda3/envs/tmp/lib/python3.8/site-packages/_pytest/_code/code:33: error: Skipping analyzing "pluggy": found module but no type hints or library stubs (diff)
E     ../../../../../../Users/nanneaben/opt/anaconda3/envs/tmp/lib/python3.8/site-packages/_pytest/_code/code:369: error: Signature of "__getitem__" incompatible with supertype "list" (diff)
....
E     ../../../../../../Users/nanneaben/opt/anaconda3/envs/tmp/lib/python3.8/site-packages/numpy/__init__.pyi:995: error: Overloaded function signatures 22 and 51 overlap with incompatible return types (diff)
E     ../../../../../../Users/nanneaben/opt/anaconda3/envs/tmp/lib/python3.8/site-packages/numpy/__init__.pyi:997: error: Overloaded function signatures 23 and 51 overlap with incompatible return types (diff)
E     ../../../../../../Users/nanneaben/opt/anaconda3/envs/tmp/lib/python3.8/site-packages/numpy/__init__.pyi:999: error: Overloaded function signatures 24 and 51 overlap with incompatible return types (diff)
E     ../../../../../../Users/nanneaben/opt/anaconda3/envs/tmp/lib/python3.8/site-packages/numpy/__init__.pyi:1001: error: Overloaded function signatures 25 and 51 overlap with incompatible return types (diff)
E     ../../../../../../Users/nanneaben/opt/anaconda3/envs/tmp/lib/python3.8/site-packages/numpy/__init__.pyi:1003: error: Overloaded function signatures 26 and 51 overlap with incompatible return types (diff)
...

Is this the expected behaviour? Am I doing something wrong? Is it possible to make pytest-mypy-plugin match the behaviour of mypy here?

The above is a simplified example, of course. The real use-case here is that I'd like to use pytest-mypy-plugin to typecheck my package, but my package imports numpy (as well as some other packages that seem to show this behaviour), so currently all tests that import my package result in an error.

Thanks in advance!

How can I test that an assignment is legal? i.e that no output is generated

Thanks for this great tool, its been indispensable to my project!

I have a plugin that defines a protocol intersection type, and I want to test that it is invariant to argument order. My test case looks like this:

from typing import Protocol

from my_package import Intersection

class P1(Protocol):
    pass

class P1(Protocol):
    pass

i: Intersection[P1, P2]
i2: Intersection[P2, P1] = i  # I want to assert that no output is generated on this line

Document cache directory usage

When tests are executed with typecheck_in_new_subprocess and disable_cache is False, --cache-dir is passed directly to mypy call as a path relative to root directory.

This means that the following are ignored

MYPY_CACHE_DIRin execution environment orenv` block of the test case.
cache_dir in mypy configuration.

are ignored.

This behavior is confusing, so it might be a good idea to document it.

In a long run, YamlTestItem should check if any of these are provided, and omit --cache-dir in such cases (this would be useful for example with remote cache).

python3.8 support

Test flakes when running tests in parallel

If I install pytest-xdist and run tests with pytest -n8, sometimes (half the time?) the test models_imported_inside_init_file_one_to_one_field fails with a failure like:

Traceback (most recent call last):
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/_pytest/runner.py", line 226, in from_call
    result = func()
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/_pytest/runner.py", line 198, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/pluggy/hooks.py", line 289, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/pluggy/manager.py", line 68, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/pluggy/manager.py", line 62, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/_pytest/runner.py", line 123, in pytest_runtest_call
    item.runtest()
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/pytest_mypy/item.py", line 139, in runtest
    return_code = typecheck_with_mypy(mypy_cmd_options)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/pytest_mypy/item.py", line 76, in typecheck_with_mypy
    flush_errors=flush_errors, fscache=fscache)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/mypy/build.py", line 162, in build
    result = _build(sources, options, alt_lib_path, flush_errors, fscache)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/mypy/build.py", line 217, in _build
    graph = dispatch(sources, manager)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/mypy/build.py", line 2360, in dispatch
    process_graph(graph, manager)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/mypy/build.py", line 2653, in process_graph
    process_fresh_modules(graph, prev_scc, manager)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/mypy/build.py", line 2731, in process_fresh_modules
    graph[id].fix_cross_refs()
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/mypy/build.py", line 1714, in fix_cross_refs
    self.options.use_fine_grained_cache)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/mypy/fixup.py", line 25, in fixup_module
    node_fixer.visit_symbol_table(tree.names)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/mypy/fixup.py", line 77, in visit_symbol_table
    self.quick_and_dirty)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/mypy/fixup.py", line 263, in lookup_qualified_stnode
    return lookup_fully_qualified(name, modules, raise_on_missing=not quick_and_dirty)
  File "/home/msullivan/src/django-stubs/env/lib/python3.6/site-packages/mypy/lookup.py", line 47, in lookup_fully_qualified
    assert key in names, "Cannot find %s for %s" % (key, name)
AssertionError: Cannot find app for myapp.models.app.App

Fix deprecation warnings from pytest

.venv/lib/python3.8/site-packages/pytest_mypy_plugins/collect.py:108
  /Users/sobolev/Documents/github/returns/.venv/lib/python3.8/site-packages/pytest_mypy_plugins/collect.py:108: PytestDeprecationWarning: direct construction of YamlTestFile has been deprecated, please use YamlTestFile.from_parent
    return YamlTestFile(path, parent=parent, config=parent.config)

.venv/lib/python3.8/site-packages/pytest_mypy_plugins/collect.py:89
.venv/lib/python3.8/site-packages/pytest_mypy_plugins/collect.py:89
.venv/lib/python3.8/site-packages/pytest_mypy_plugins/collect.py:89
.venv/lib/python3.8/site-packages/pytest_mypy_plugins/collect.py:89
.venv/lib/python3.8/site-packages/pytest_mypy_plugins/collect.py:89
  /Users/sobolev/Documents/github/returns/.venv/lib/python3.8/site-packages/pytest_mypy_plugins/collect.py:89: PytestDeprecationWarning: direct construction of YamlTestItem has been deprecated, please use YamlTestItem.from_parent
    yield YamlTestItem(

-- Docs: https://docs.pytest.org/en/latest/warnings.html

Broken disable_check logic

I might be wrong, but it seems like the conditions for invoking cache removal has been unintentionally reversed:

pytest-mypy-plugins/pytest_mypy_plugins/item.py

Lines 297 to 304 in a2d4add

 if not self.disable_cache: 

 for file in self.files: 

 path = Path(file.path) 

 self.remove_cache_files(path.with_suffix("")) 

 dependants = self.find_dependent_paths(path) 

 for dependant in dependants: 

 self.remove_cache_files(dependant)

As-is, cache is removed when disable_cache is False and preserved when it is True.

pdb not catching although --mypy-same-process

Hey @sobolevn, hope you're doing great in those crazy days !

Anyway, I want to keep this short. I am trying to debug a certain issue in my code but I have the problem.
Neither pycharm nor pdb is able to break in a place I want. It makes no difference whether I make a breakpoint or do pdb.set_trace(). It just ignores that.

Here's relevant portion of my setup https://github.com/kornicameister/axion/blob/master/pytest.ini#L9
Any hints are welcome.

Introduce a test case schema

Here's what I have in mind.
What about creating a @dataclass or NamedTuple or ish that will describe using a model what is a test case?
This should make it easier to maintain a code IMHO.

A suggestion, if you are interested :)

AttributeError: type object 'YamlTestFile' has no attribute 'from_parent' in pytest_collect_file

The latest update to pytest_mypy_plugins broke my CI tests with the following error

==================================== ERRORS ====================================

______________________ ERROR collecting tests/__init__.py ______________________

../../../virtualenv/python3.8.0/lib/python3.8/site-packages/pluggy/hooks.py:286: in __call__

    return self._hookexec(self, self.get_hookimpls(), kwargs)

../../../virtualenv/python3.8.0/lib/python3.8/site-packages/pluggy/manager.py:92: in _hookexec

    return self._inner_hookexec(hook, methods, kwargs)

../../../virtualenv/python3.8.0/lib/python3.8/site-packages/pluggy/manager.py:83: in <lambda>

    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(

../../../virtualenv/python3.8.0/lib/python3.8/site-packages/pytest_mypy_plugins/collect.py:139: in pytest_collect_file

    return YamlTestFile.from_parent(parent, fspath=path)

E   AttributeError: type object 'YamlTestFile' has no attribute 'from_parent'

!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!

=============================== 1 error in 0.11s ===============================

The command "pytest" exited with 2.

view the log outputs here: https://travis-ci.org/github/lovasoa/marshmallow_dataclass/jobs/705848682

regex not listed as a dependency in setup.py

Currently regex is imported at https://github.com/typeddjango/pytest-mypy-plugins/blob/master/pytest_mypy_plugins/utils.py#L23 but it is not listed in setup.py

I ran into the issue with 1.8 in a project https://github.com/nucleic/atom/pull/122/checks?check_run_id=3538057249

Would it be possible to fix it and publish a new release ?

Add `mark` support

Sometimes I want to run all FutureResult (or tests for any other type) related tests from returns.
It is a very complex task right now.

I need to:

manually find all FutureResult inside tests
manually copy all the paths from the query
provide long list of arguments to pytest

All I want instead is:

- case: test_future_result
   marks:
     - future_result
  main: ...

And then pytest -m future_result to run this (and all other similarly marked tests).

Fails with mypy>=0.780

I think I am raporting bug against correct repository.
Seems like something changed in mypy and with mypy>=0.780 things are rather breaking up.
I am attaching a commit from my own repository here.
You can checkout it or just go here to see how errors look like.

In short, it seems that something did change around how mypy deals with paths? Otherwise there wouldn't be any issue.
Something that was originally relative to the root of the project is not relative to something else making "expected" errors message impossible to deduce.

	try:
	temp_dir = tempfile.TemporaryDirectory(prefix="pytest-mypy-", dir=self.root_directory)

	if not self.disable_cache:
	for file in self.files:
	path = Path(file.path)
	self.remove_cache_files(path.with_suffix(""))

	dependants = self.find_dependent_paths(path)
	for dependant in dependants:
	self.remove_cache_files(dependant)

typeddjango / pytest-mypy-plugins Goto Github PK

pytest-mypy-plugins's People

Contributors

Stargazers

Watchers

Forkers

pytest-mypy-plugins's Issues

Eval context

Related

Recommend Projects

Recommend Topics

Recommend Org