Support `@seed` decorator about pytest-randomly HOT 3 CLOSED

What if there was a pytest fixture or context manager? Then we could have

@pytest.mark.parametrize('seed', [9001, 1337])
def test_old_bad_seeds(seed, randomly):
    randomly.reseed(seed)
    # test that has failed in the past with those exact seeds

Tbh I'm not that excited about this though. I think if you find a failing test with a particular seed, it's not the seed that should be turned into a new test case but the actual underlying data that caused the failure. For example if seed 9001 caused your data generator to create a user with a really long name, add a test with a user with a fixed really long name.

Fixing the seed in a test run is pretty fragile, as the data generator could change in the future, meaning the seed values don't mean the same thing, so the tests could stop testing the specific bugs they exposed. For example, new versions of faker come with new data, remapping the data returned for a given seed.

Do you have a use case that prevents making tests with fixed test data, or at makes it impractical?

from pytest-randomly.

Fixing the seed in a test run is pretty fragile, as the data generator could change in the future, meaning the seed values don't mean the same thing, so the tests could stop testing the specific bugs they exposed

Yeap, that seems reasonable. hypothesis spoiled me here.
I will stick to the data examples.

from pytest-randomly.

Btw hypothesis has @example for this - https://hypothesis.readthedocs.io/en/latest/reproducing.html#providing-explicit-examples . But that can't have an analog in pytest-randomly because we don't touch data generation :)

from pytest-randomly.