Using dicts as @data generates variable test names about ddt HOT 11 CLOSED

Please see santtu@11fad53 -- with that I get with Python 3.4.0:

$ python3 `which nosetests` -v
test_dict_1 (test.Test) ... ok
test_scalar_1_1 (test.Test) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.010s

OK
$ PYTHONHASHSEED=1 python3 `which nosetests` -v
test_dict_1___b___2___a___1_ (test.Test) ... ok
test_scalar_1_1 (test.Test) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.008s

OK

and Python 2.7.5:

$ python2.7 `which nosetests` -v
test_dict_1___a___1___b___2_ (test.Test) ... ok
test_scalar_1_1 (test.Test) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.007s

OK
$ python2.7 -R `which nosetests` -v
test_dict_1 (test.Test) ... ok
test_scalar_1_1 (test.Test) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.009s

OK
$ PYTHONHASHSEED=1 python2.7 -R `which nosetests` -v
test_dict_1___a___1___b___2_ (test.Test) ... ok
test_scalar_1_1 (test.Test) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.009s

OK

from ddt.

Comment, see https://docs.python.org/3.2/library/sys.html -- apparently sys.flags.hash_randomization was added to 3.x versions on 3.2.3, so if < 3.2.3 is supported then the use of sys.flags.hash_randomization needs to be guarded better.

from ddt.

Thanks for this.

I have only read through your description and solution without analysing it too much, but here are the first thoughts that came to my mind. These are raw ideas I haven't thought of in detail re: feasibility or ease of use:

1. We could extend DDT with some flag or environment variable that will not try to serialise the value into the test name and only use the ordinal: testx_1, testx_2... on a global basis - this you would use when you want to always have simple short and predictable function names
2. We could do a similar thing by passing those flags to the ddt decorator
3. Same but with the data decorator
4. We could improve on the options we have to give data items "names". There are already some but maybe not documented enough, or not easy to use enough. It may be a better option for the test writer to identify what a test case is rather than us serialising what can be quite complex data.

Examples of how these could look in practice:

DDT_SHORT_NAMES=1 nosetests 

@ddt(short_names=True)

@data(x, y, z, __short_names=True)

@data(labelled("simple", x), labelled("complex", y))

@data(L("simple", x), L("complex", y))  # alt name

@data(_("simple", x), _("complex", y))  # alt name

@labelled_data(("simple", x), ("complex", y))

What do you think? Which one looks better to you? I think we should choose only one for now.

I'm open to suggestions re: names of the variable/flag that defines this behaviour, and the new decorator or wrapper.

from ddt.

In santtu@11fad53 I basically did the first one, but it checks Python version, sys.flags.hash_randomization and PYTHONHASHSEED values and based on those either generates names or falls back to using sequence numbers alone. So it doesn't need anything from the user and will work produce consistent names in all cases.

from ddt.

I think getting having consistent test names with hash randomization enabled is really the most important thing. Everything after that is nice, but not absolutely required.

from ddt.

Hello Santtu,
I was having a problem with dictionaries being converted to strings so large that I had OS errors. So, how do I force ddt to give me the short test names?
It would be awesome if it was possible to force short names to be on. Thanks!

from ddt.

Hi @mcookson. I take you have a separate problem? Mine was that the names of the tests were changing from run to run, but I understand that yours is that the names of the tests are large (255 or more chars?) and you get some failures due to that, right? Why is that -- does nose use the test names as file names or why would OS itself give errors based on test names?

Anyway if you check https://github.com/santtu/ddt/blob/11fad53fbb7295de5a47b66d0bd4533e47f911fb/ddt.py#L66 you'll see that it's the mk_test_name routine that needs to be changed.

BTW, if you are having problems with large test names, you might want to consider the method described in docs of how to set __name__ attribute on the dict (or any @data value) to change the "name" used for the test data. Would that work?

from ddt.

@santtu I haven't had time to work on this, but I'll take your PR if you send one, then when I get some more time I plan to work on making option 1 available via an env var on any version of python.

from ddt.

There you go. I have tested this with python 2.7 and 3.4, but tests on other (pre 2.7.3 and pre 3.2.3 especially) would be appreciated.

from ddt.

Part because of this and part because test data in my case is usually very long I switched to use simple TestCase class(inherited from dict and used as plain dict) with single overridden str method that just returns a custom name that I give on init along with dictionary itself.

from ddt.

Closing as solved in #20

from ddt.