smarie / python-pytest-cases Goto Github PK

View Code? Open in Web Editor NEW

340.0 5.0 40.0 13.71 MB

Separate test code from test cases in pytest.

Home Page: https://smarie.github.io/python-pytest-cases/

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

pytest test case testcase test-case decorator parametrize parameter data dataset

python-pytest-cases's People

Stargazers

Watchers

python-pytest-cases's Issues

Allow `cases_generator` to have a function to return the ids rather than only the template.

Like in pytest parametrize. For example providing a callable instead of a template in name_template. Also supporting an explicit name list could be great.

Explain custom type hints in usage doc

Allow case functions to have parameters, to be used in test suites

Explain how cases can be injected in test suites with several steps

Add support for test case data caching

See https://stackoverflow.com/questions/46913043/pytest-parametrised-fixture-with-different-cases

As of v0.3.0, using @lru_cache on case functions raises an exception.

Change all examples so that they are more intuitive

For example

Cases: Orange, Strawberry

Test: test_fruit

Test steps (only for the pytest-steps example):

peel (may fail)
mix (does not depend on peel)
eat (depends on mix)

pytest_fixture_plus does not honor parameter id

Hi, another one! :)

Example:

from pytest_cases import pytest_fixture_plus as fixture
import pytest

@pytest.fixture(params=[
    pytest.param(1, id="p_a"),
    pytest.param(2, id="p_b"),
])
def pyfix(request):
    return request.param

@fixture
@pytest.mark.parametrize("arg1, arg2", [
    pytest.param(1,2, id="f_a"),
    pytest.param(3,4, id="f_b")
])
def myfix(arg1, arg2):
    return arg1, arg2

@pytest.mark.parametrize("arg3, arg4", [
    pytest.param(10,20, id="t_a"),
    pytest.param(30,40, id="t_b")
])
def test_one(pyfix, myfix, arg3, arg4):
    print(myfix)

result:

test_one.py::test_one[p_a-1-2-t_a] PASSED                                                                                                                                                                                                                [ 12%]
test_one.py::test_one[p_a-1-2-t_b] PASSED                                                                                                                                                                                                                [ 25%]
test_one.py::test_one[p_a-3-4-t_a] PASSED                                                                                                                                                                                                                [ 37%]
test_one.py::test_one[p_a-3-4-t_b] PASSED                                                                                                                                                                                                                [ 50%]
test_one.py::test_one[p_b-1-2-t_a] PASSED                                                                                                                                                                                                                [ 62%]
test_one.py::test_one[p_b-1-2-t_b] PASSED                                                                                                                                                                                                                [ 75%]
test_one.py::test_one[p_b-3-4-t_a] PASSED                                                                                                                                                                                                                [ 87%]
test_one.py::test_one[p_b-3-4-t_b] PASSED                                                                                                                                                                                                                [100%]

expected result:

test_one.py::test_one[p_a-f_a-t_a] PASSED                                                                                                                                                                                                                [ 12%]
test_one.py::test_one[p_a-f_a-t_b] PASSED                                                                                                                                                                                                                [ 25%]
test_one.py::test_one[p_a-f_b-t_a] PASSED                                                                                                                                                                                                                [ 37%]
test_one.py::test_one[p_a-f_b-t_b] PASSED                                                                                                                                                                                                                [ 50%]
test_one.py::test_one[p_b-f_a-t_a] PASSED                                                                                                                                                                                                                [ 62%]
test_one.py::test_one[p_b-f_a-t_b] PASSED                                                                                                                                                                                                                [ 75%]
test_one.py::test_one[p_b-f_b-t_a] PASSED                                                                                                                                                                                                                [ 87%]
test_one.py::test_one[p_b-f_b-t_b] PASSED                                                                                                                                                                                                                [100%]

`param_fixtures` (with an s) seems to create auto-use fixtures

This test fails because instead of being [1] the node id is [a-1]. In other words the first arg fixture created seems to be in 'auto use' mode.

import pytest
from pytest_cases import param_fixtures

arg = param_fixtures('arg', ['a', 'b'])

@pytest.mark.parametrize("arg", [1])
def test_reference2(arg, request):
    assert '[1]' in request.node.nodeid

Note that this does not happen with param_fixture.

Fixtures get called with `NOT_USED` as a parameter when using a fixture union

from pytest_cases import param_fixture, fixture_union, pytest_fixture_plus

a = param_fixture('a', ['x', 'y'])

@pytest_fixture_plus(params=[1, 2])
def b(request):
    assert request.param in [1, 2]  # this fails !
    return request.param

c = fixture_union('c', [a, b])

def test_fixture_union(c, a):
    print(c, a)

It should be easy to include cases and tests in the same file

Maybe a special constant THIS_MODULE will do the trick ?

ValueError: 1 tests specified with X ids

Improve filtering mechanism

It should be easier to

select cases one by one in a test function
select them based on basic and / or logic on the tags
use advanced filtering logic

See this discusssion

`fixture_union()` able to take a list of fixtures instead of a list of values

This seems like the missing piece. When we have this feature, combining it with param_fixture / param_fixtures will enable us to reproduce any of the scenarios mentioned in
pytest-dev/pytest#349

Note that we could also provide a helper goodie @pytest_parametrize_plus(..., fixtures=[...]) to do the same but directly on a function.

`@pytest_parametrize_plus` decorator able to handle `fixture_ref`s in parameter values

According to many users (see pytest-dev/pytest#349) it would be useful to use fixture references in pytest parameter values.

Now that we support union fixtures this should be feasible.

Explain "case fixtures" in documentation

Allow cases for several test functions to coexist in the same module file

For example we could provide a way to 'tag' the case functions with any object(s), and to 'filter' the case functions based on tag match in the @cases_data decorator

id for fixtures parametrized with pytest.mark.parametrize

I can explain it in the following example:

from pytest_cases import pytest_fixture_plus
import pytest


###### Using Pytest Fixture Plus ######


@pytest_fixture_plus
@pytest.mark.parametrize("arg1,arg2", [
    (1, 2),
    (3, 4),
])
def myfix(arg1, arg2):
    return arg1, arg2

@pytest.mark.parametrize("arg1", [
    1, 2,
])
def test_one(myfix, arg1):
    print(myfix)


##### Pytest Normal parametrization #####


@pytest.fixture(params=[10,20])
def myfix2(request):
    return request.param

@pytest.mark.parametrize("arg1, arg2", [
    (1, 2),
    (3, 4),
])
def test_two(arg1, arg2, myfix2):
    print(arg1, arg2)

Which renders:

test_one.py::test_one[myfix__arg1Xarg20-1] PASSED                                                                                                                   [ 12%]
test_one.py::test_one[myfix__arg1Xarg20-2] PASSED                                                                                                                   [ 25%]
test_one.py::test_one[myfix__arg1Xarg21-1] PASSED                                                                                                                   [ 37%]
test_one.py::test_one[myfix__arg1Xarg21-2] PASSED                                                                                                                   [ 50%]
test_one.py::test_two[10-1-2] PASSED                                                                                                                                [ 62%]
test_one.py::test_two[10-3-4] PASSED                                                                                                                                [ 75%]
test_one.py::test_two[20-1-2] PASSED                                                                                                                                [ 87%]
test_one.py::test_two[20-3-4] PASSED                                                                                                                                [100%]

So basically the test id for test_one, parametrized using pytest_fixture_plus is not very pretty. It would be better to follow the same rules pytest uses for test ids in parametrization, resulting perhaps in something like this:

test_one.py::test_one[1-2-1] PASSED                                                                                                                   [ 12%]
test_one.py::test_one[1-2-2] PASSED                                                                                                                   [ 25%]
test_one.py::test_one[3-4-1] PASSED                                                                                                                   [ 37%]
test_one.py::test_one[3-4-2] PASSED                                                                                                                   [ 50%]
test_one.py::test_two[10-1-2] PASSED                                                                                                                                [ 62%]
test_one.py::test_two[10-3-4] PASSED                                                                                                                                [ 75%]
test_one.py::test_two[20-1-2] PASSED                                                                                                                                [ 87%]
test_one.py::test_two[20-3-4] PASSED                                                                                                                                [100%]

`pytest_fixture_plus(unpack_into=)` not taken into account when no parameters are provided

Error when using session- or module- scoped fixtures and fixture unions.

While trying to solve this : https://stackoverflow.com/questions/46909275/parametrizing-tests-depending-of-also-parametrized-values-in-pytest

I found out that my proposal with union fixtures was not working correctly.

Make `@cases_fixture` support `@pytest.mark.parametrize`

This idea will probably eventually be part of pytest, if users find it interesting.

But for older versions of pytest and also in the meantime, we could provide a workaround and support the following syntax:

@cases_fixture(module=xxxx, scope='session')
@pytest.mark.parametrize('param', [1, 2], ids=str)
def my_fixture(case_data, param):
    return case_data.get(param)

Note that fixing it would also immediately make additional @cases_data usable, assuming we change the argument name. Indeed they are nothing but an automatized @pytest.mark.parametrize

@cases_fixture(module=xxxx, scope='session')
@cases_data(module=yyyy, case_data_argname='case_data2')
def my_fixture(case_data, case_data2):
    ... whatever you wish to do with the two cases ...

ValueError: too many values to unpack (expected 2)

With pytest 3.7

../../../virtualenv/python3.5.6/lib/python3.5/site-packages/pytest_cases/plugin.py:455: in getfixtureclosure
    ref_fixturenames, ref_arg2fixturedefs = fm.__class__.getfixtureclosure(fm, fixturenames, parentnode)

Issue in pytest-cases 1.8.1 with markers

First of all, thanks for this neat library, it has simplified our tests quite a bit for cookiecutter-django.

I'm not sure if I missed a change in 1.8 or if it's an actual bug, but it seems that 1.8.1 is not playing nicely with pytest markers. We use markers to spread markers over multiple workers on our CI:

One worker runs tests marked with black
One worker runs tests marked with flake8
A last worker runs all unmarked tests flake8

We basically have 3 test function which all use a pretty big parametrized fixture.

Until <1.8.1, the black output was running in ~8 mins. With 1.8.1, it takes ~20 mins to run and judging by the output, it looks like a lot more test cases are executed.

I tried to bump to 1.8.0 and the output is consistent with the existing behaviour and execution around ~9mins.

Am I still using the library as it's intended or is there a regression in 1.8.1?

Add `unpack_into` to `union_fixture` too

Issues when parametrize argnames contains a list

IndexError: list index out of range or AttributeError: 'list' object has no attribute 'replace' can happen even with normal pytest parametrize when argnames contains a list/tuple.

Provide a parameter to guarantee fixture uniqueness in `@fixture`

session-scoped fixtures are not guaranteed to be unique by session, and module-scoped fixtures are not guaranteed to be unique for an unique parameter, in pytest. See pytest-dev/pytest#2846

We could at least try to provide a workaround in the @cases_fixture decorator, such as ensure_unique=True.

Note that pytest-dev/pytest#3393 will not solve the problem because if users have two parametrized session-scoped fixtures, one of them will end-up being setup/teardown twice for each of its parameters.

Propose additional mechanisms to collect cases

Two things would be cool:

load all subclasses of a given class
load all python scripts located in a folder (less sure of the need)

The parametrization order appears to be random when `@pytest_fixture_plus` is used with several `@pytest.mark.parametrize`

Consider the following example:

A = [1, 2]
B = [10, 20]

@pytest_fixture_plus
@pytest.mark.parametrize("a", A)
@pytest.mark.parametrize("b", B)
def my_fix(a, b):
    return a + b

def test_my_fix(my_fix):
    pass

The order of execution will randomly be 1-10, 1-20, 2-10, 2-20 or 10-1, 10-2, 20-1, 20-2.

If 2 fixtures without parameters are used in a fixture union, their setup/teardown should be done in sequence too

Show an example where test steps results are stored in the case object

Independent parameter

Hi, this is a feature suggestion, which I believe is the missing piece of parametrization

We can define an individual parameter used for parametrization that is not either intrinsically tied to a fixture or test function like this:

@fixture(params=[1,2,3,4])
def my_parameter(request):
    return request.param

@fixture
def fixture_uses_param(my_parameter):
    pass

def test_uses_param(my_parameter, fixture_uses_param):
    pass

The example above is bad because we are using a fixture to mask what is really a parameter. But we can solve this by doing:

def parameter(values):
    @pytest.fixture(scope="session", params=params)
    def parameter_fixture(request):
        return request.param
    return parameter_fixture

and then the test becomes:

my_parameter = parameter([1,2,3,4])

@fixture
def fixture_uses_param(my_parameter):
    pass

def test_uses_param(my_parameter, fixture_uses_param):
    pass

And we can use many parameters to have a cartesian product, etc etc. This still has two problems though:

It does not use the string definition of arguments. In other words, maybe it would be preferred to have parameter("my_parameter", [1,2,3,4]) instead of my_parameter = parameter([1,2,3,4])
I cannot parametrize a pair of arguments like this. This is related to the previous bullet point as we would need the alternate syntax for this: parameter("my_parameter1, my_parameter2", [(1,2),(3,4)]).

So, the feature proposal would be to have this parametrization ability through the module body with the mentioned syntax:

parameter("my_parameter, my_other_parameter", [(1,1), (2,2), (3,3), (4,4)])

@fixture
def fixture_uses_param(my_parameter):
    pass

def test_uses_param(my_parameter, my_other_parameter, fixture_uses_param):
    pass

Is this even possible? I know very little about how parametrization works under the hood in pytest, so if you might be not interested in this feature, I would appreciate some pointers.

thanks!

feature to unpack a fixture into several fixtures

See

Exception in fixture_plus if parametrization has whitespace between argument string

This works:

from pytest_cases import pytest_fixture_plus
import pytest

@pytest_fixture_plus
@pytest.mark.parametrize("arg1,arg2", [
    (1, 2),
    (3, 4),
])
def myfix(arg1, arg2):
    return arg1, arg2

def test_one(myfix):
    print(myfix)

Now adding a whitespace to the parametrize arguments, after the comma:

from pytest_cases import pytest_fixture_plus
import pytest

@pytest_fixture_plus
@pytest.mark.parametrize("arg1, arg2", [  # ADDED WHITESPACE AFTER COMMA
    (1, 2),
    (3, 4),
])
def myfix(arg1, arg2):
    return arg1, arg2

def test_one(myfix):
    print(myfix)

fails with

test_one.py:8: in <module>
    (3, 4),
..\venv368\lib\site-packages\decopatch\main.py:349: in new_decorator
    return call_in_appropriate_mode(impl_function, dk, disambiguation_result)
..\venv368\lib\site-packages\decopatch\utils_calls.py:34: in call_in_appropriate_mode
    return no_parenthesis_usage(impl_function, dk.first_arg_value)
..\venv368\lib\site-packages\decopatch\utils_calls.py:71: in no_parenthesis_usage
    return decorator_function()(decorated)
..\venv368\lib\site-packages\decopatch\utils_modes.py:129: in _apply_decorator
    return user_provided_applier(*args, **kwargs)
..\venv368\lib\site-packages\pytest_cases\main.py:365: in pytest_fixture_plus
    new_sig = remove_signature_parameters(old_sig, *old_parameter_names)
..\venv368\lib\site-packages\makefun\main.py:739: in remove_signature_parameters
    del params[param_name]
E   KeyError: ' arg2'

This works fine when parametrizing a test in pytest.

Support case data generators (using yield)

Test on python 2

pytest_fixture_plus error with simple example

Hi,

I am testing this code:

from pytest_cases import pytest_fixture_plus
import pytest

@pytest_fixture_plus(scope="module")
@pytest.mark.parametrize("arg1", [
    "one", "two"
])
@pytest.mark.parametrize("arg2", [
    "one", "two"
])
def myfix(arg1, arg2):
    return (arg1, arg2)


def test_one(myfix):
    print(myfix)

And getting this exception:

========================================================================================================================= test session starts ==========================================================================================================================
platform win32 -- Python 3.6.4, pytest-4.3.1, py-1.8.0, pluggy-0.9.0
rootdir: C:\Users\victo\PycharmProjects\testing_parameterize_in_fixtures, inifile:
collected 0 items / 1 errors                                                                                                                                                                                                                                            

================================================================================================================================ ERRORS ================================================================================================================================
___________________________________________________________________________________________________________________ ERROR collecting test_simple.py ____________________________________________________________________________________________________________________
venv\lib\site-packages\makefun\main.py:761: in add_signature_parameters
    for param in reversed(first):
E   TypeError: 'generator' object is not reversible

During handling of the above exception, another exception occurred:
test_simple.py:9: in <module>
    "one", "two"
venv\lib\site-packages\decopatch\utils_modes.py:129: in _apply_decorator
    return user_provided_applier(*args, **kwargs)
venv\lib\site-packages\pytest_cases\main.py:368: in pytest_fixture_plus
    for n in reversed(new_parameter_names)))
venv\lib\site-packages\makefun\main.py:768: in add_signature_parameters
    if first.name in params:
E   AttributeError: 'generator' object has no attribute 'name'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
======================================================================================================================= 1 error in 0.34 seconds ========================================================================================================================

This is my environment:

atomicwrites (1.3.0)
attrs (19.1.0)
colorama (0.4.1)
decopatch (1.4.1)
makefun (1.6.0)
more-itertools (6.0.0)
pip (9.0.1)
pluggy (0.9.0)
py (1.8.0)
pytest (4.3.1)
pytest-cases (1.3.1)
setuptools (28.8.0)
six (1.12.0)

Maybe I am doing something wrong?

Thanks!

Support `fixture_ref()` in tuple of parameters

Compliance with pytest 2.x, pytest 3.x, and python 2.x, python 3.x

as mentioned in #13 we should at least make travis run all configurations to be sure the lib works everywhere.

pytest_fixture_plus does not seem to work with pytest.param parameters

First of all let me thank you for the nice pytest_fixture_plus, I am really excited about it.

The problem I am having can be seen in the following example.

This works:

from pytest_cases import pytest_fixture_plus as fixture
import pytest

@fixture
@pytest.mark.parametrize("arg1, arg2", [
    (1,2),
    (3,4)
])
def myfix(arg1, arg2):
    return arg1, arg2

@pytest.mark.parametrize("arg3, arg4", [
    pytest.param(10,20),
    pytest.param(30,40)
])
def test_one(myfix, arg3, arg4):
    print(myfix)

test_one.py::test_one[1-2-10-20] PASSED                                                                                                                                                                                                                  [ 25%]
test_one.py::test_one[1-2-30-40] PASSED                                                                                                                                                                                                                  [ 50%]
test_one.py::test_one[3-4-10-20] PASSED                                                                                                                                                                                                                  [ 75%]
test_one.py::test_one[3-4-30-40] PASSED                                                                                                                                                                                                                  [100%]

But if we enclose the fixture parameters using pytest.param, it does not anymore:

from pytest_cases import pytest_fixture_plus as fixture
import pytest

@fixture
@pytest.mark.parametrize("arg1, arg2", [
    pytest.param(1,2),
    pytest.param(3,4)
])
def myfix(arg1, arg2):
    return arg1, arg2

@pytest.mark.parametrize("arg3, arg4", [
    pytest.param(10,20),
    pytest.param(30,40)
])
def test_one(myfix, arg3, arg4):
    print(myfix)

test_one.py:8: in <module>
    pytest.param(3,4)
..\venv368\lib\site-packages\decopatch\main.py:349: in new_decorator
    return call_in_appropriate_mode(impl_function, dk, disambiguation_result)
..\venv368\lib\site-packages\decopatch\utils_calls.py:34: in call_in_appropriate_mode
    return no_parenthesis_usage(impl_function, dk.first_arg_value)
..\venv368\lib\site-packages\decopatch\utils_calls.py:71: in no_parenthesis_usage
    return decorator_function()(decorated)
..\venv368\lib\site-packages\decopatch\utils_modes.py:129: in _apply_decorator
    return user_provided_applier(*args, **kwargs)
..\venv368\lib\site-packages\pytest_cases\main.py:299: in pytest_fixture_plus
    raise ValueError("Internal error - unsupported pytest parametrization+mark combination. Please "
E   ValueError: Internal error - unsupported pytest parametrization+mark combination. Please report this issue

Add the possibility to use pytest marks on cases

For example I would like to mark a particular case as skipped

@pytest.mark.skip("skipped because it's too easy")
def case_skipped():
    return "hello"

marks on cases do not work with pytest 3.3

Update the tests so that they run pytest (move to "meta-testing")

Basically we should copy what is done in pytest-harvest. Note that the continuous integration configuration should also be updated so as to run the correct matrix of pytest version x python version

Could generator be useful ? Provide a `case_data.yield()` function

If the case function is written as

def case_generate():
    yield foo_for_step1
    yield foo_for_step2

then I guess a a case_data.yield() could be used to retrieve the case data for the various steps.

Try it and add it in the documentation if it works as expected

Improve the error message when the name template is wrong in `@cases_generator`

As of today the exception can be tricky to understand :)

(...)
..\..\..\..\..\_Libs_OpenSource\python-pytest-cases\pytest_cases\main_params.py:279: in get_all_cases
    for m in module:
E   TypeError: 'object' object is not iterable

During handling of the above exception, another exception occurred:
(...)
C:\Miniconda3\envs\baseenv\lib\site-packages\decopatch\utils_modes.py:132: in _apply_decorator
    return user_provided_applier(*new_args, **kwargs)
..\..\..\..\..\_Libs_OpenSource\python-pytest-cases\pytest_cases\main_params.py:210: in cases_data
    _cases = get_all_cases(cases, module, test_func, has_tag, filter)
..\..\..\..\..\_Libs_OpenSource\python-pytest-cases\pytest_cases\main_params.py:285: in get_all_cases
    _cases = extract_cases_from_module(m, has_tag=has_tag, filter=filter)
..\..\..\..\..\_Libs_OpenSource\python-pytest-cases\pytest_cases\main_params.py:344: in extract_cases_from_module
    _get_case_getter_s(f, code, cases_dct)
..\..\..\..\..\_Libs_OpenSource\python-pytest-cases\pytest_cases\main_params.py:398: in _get_case_getter_s
    gen_case_name = names(**gen_case_params_dct)
..\..\..\..\..\_Libs_OpenSource\python-pytest-cases\pytest_cases\main_params.py:384: in names
    return _formatter.format(**params)
E   IndexError: tuple index out of range

`pytest` Acceptance tests concerning fixture order should pass

With our new replacement for the fixture closure and parametrization engine (to enable fixture unions in addition to cartesian products), we might have broken the basics. Let's make sure this is not the case.

See https://github.com/pytest-dev/pytest/blob/master/testing/acceptance_test.py#L1096

From pytest-dev/pytest#2405 that was fixed by pytest-dev/pytest#3306

But see also

pytest-dev/pytest#5303
... (?)

Improve generated fixture names for `@pytest_fixture_plus`

gen_paramfixture__<fixturename>__<paramname> is not a very convenient name,

and besides it does not appear next to <fixturename> which is the decorated fixture name.

Suggested naming: <fixturename>__<paramname>. Simple and straightforward

Instead of always disabling the pytest "smart ordering" for session and module scoped fixtures, propose an option

In current version, the pytest order optimization (reorder_items here) is completely disabled (the original order is restored after it has been executed), so that the annoying bug concerning module-scoped fixtures order disappears.

That allows the right order to be set for example in this example.

However

that bug was not there in old version of pytest
things are evolving and new PRs arrive to try to solve it (see pytest#3551)
our current implementation has a huge side-effect: it completely ignores the effect of ordering plugins such as pytest-ordering.

For these reasons it would seem much more reasonable to only propose this "skip pytest optimized order" as an option.

Ids look ok but not values in param_fixtures

In this somewhat complex example:

from pytest_cases import param_fixtures
from pytest_cases import pytest_fixture_plus as fixture
import pytest

parg1, parg2 = param_fixtures("parg1, parg2", [("a", "b"), ("c", "d")])

@fixture
@pytest.mark.parametrize("arg1, arg2", [
    pytest.param(1, 2, id="f_a"),
    pytest.param(3, 4, id="f_b")
])
def myfix(arg1, arg2, parg1):
    return arg1, arg2, parg1

@pytest.mark.parametrize("arg3, arg4", [
    pytest.param(10, 20, id="t_a"),
    pytest.param(30, 40, id="t_b")
])
def test_one(myfix, arg3, arg4, parg1, parg2):
    print(f"{parg1} {parg2} {myfix} {arg3} {arg4}")

The ids are ok but returned values are not when printing them:

Alternatives created by fixture unions should be easily visible in test ids

from pytest_cases import param_fixture, fixture_union

a = param_fixture("a", [1, 2])
b = param_fixture("b", [3, 4])

c = fixture_union('c', ['a', 'b'])

def test_the_ids(c):
    pass

Currently leads to the following ids:

test_fixture_union_ids.py::test_the_ids[a-1] 
test_fixture_union_ids.py::test_the_ids[a-2] 
test_fixture_union_ids.py::test_the_ids[b-3] 
test_fixture_union_ids.py::test_the_ids[b-4]

It will maybe not be very understandable in more complex cases. Besides as of today if one tries to set the ids manually he will get an error ValueError: ids cannot be set on a union fixture.

I propose to (1) allow ids to be set on union fixtures, and (2) propose an additional "idstyle" argument with three options:

idstyle=None: a, b (current implementation): there is no way to distinguish this from other parameters
idstyle='compact': Ua, Ub
idstyle='explicit': c_is_a, c_is_b. This would become the default.

Note that this new parameter should be available in pytest_parametrize_plus too.

Replace `my_decorator` with `makefun`

decorator stays focused on signature-preserving scenarii (micheles/decorator#58 ), and that makes sense.

I therefore created makefun to separate the concerns and focus solely on dynamic function generation in particular for signature modification scenarii.

We can now safely replace the my_decorator.py hack that was pending in here, by usage of makefun.

Ability to create "case fixtures"

As of today a test function decorated with @case_data is a great way to perform the data retrieval inside the test function body: that way, if a single dataset fails to be retrieved, we do not fail the whole test suite.

However the fact that data retrieval is done inside the test function body might be a concern if test duration is monitored: the time spend to retrieve and/or parse each dataset is not "test duration" per se. Besides if users put some caching in place on their case functions, the first test's duration will be different from subsequent tests'.

For all those reasons it might be a better choice to allow users to create "case fixtures". They would work exactly the same way as current decorator (same arguments to specify where the cases are located and how to filter them).

Documentation would need to be improved then.

smarie / python-pytest-cases Goto Github PK

python-pytest-cases's People

Stargazers

Watchers

Forkers

python-pytest-cases's Issues

Recommend Projects

Recommend Topics

Recommend Org