behave vs py.test about pyethereum HOT 19 CLOSED

BDD style is much younger than traditional test case style.
But it's attractive.

Definetely it's not only suitable for business level behavior,
low level can also be benefited, especially when the logic is sophisticated:

1. It helps yourself more to sort out your mind. In fact, I have found and fixed two bugs in this way today, besides the one you fixed.
2. It can serve as documentation for later contributor.
3. The auto pipe ways to form a whole test, though equal to explicit function call, make each piece of codes(the step) small and consice.

You'll love this style :-)

from pyethereum.

Thanks for the explanation. Though I'm pretty sure you would love pytest too!

from pyethereum.

Yes, I use nose before, but I think pytest is even better especially now it has fabulous documentation :-)

from pyethereum.

I'm probably ignorant, but the following looks a bit noisy and seems to introduce a lot of incidental complexity:

https://github.com/ethereum/pyethereum/blob/master/features/trie_hook.py#L40

we have the "load_data" decorator in trie.features, which is implemented in trie_hook.py, and it's being injected manually in environment.py?

is there anyone else who believes behave is a better fit for pyethereum than pytest?

@freizeit what's your take on this?

from pyethereum.

I think the most important thing is to have tests at all. For a technology like Ethereum tests are crucial. Writing tests is not the most rewarding task for a developer. Therefore a test framework at least should not distract developers from writing tests. py.test is pretty standard and well accepted, while at least four devs here are unhappy with behave.

I propose to either switch to py.test completely or at least to allow py.test tests in addition to behave tests.

So the question is: would anyone object if we continue to submit py.test tests?

from pyethereum.

@chenhouwu: do you know another contributor who likes behave?

at the moment I feel handicapped by the lack of a decent testing tool and lack of coordination. I did look into trie.py's code yesterday, but probably should not have done that, since it looks much better today.

from pyethereum.

I am also a js/node developer.
In fact, BDD tools like Mocha/Jasmine are already dominant in the js world,
and I have a fabulous experience with BDD.

So back to the python world, I want to BDD whenever possible.
Here possible I mean that it don't hurt team collabration.

Anyway, if all of you oppose behave, we can shift back to py.test.
But....If you have no experience with BDD,
since it only take you 20 minutes to learn behave,
I suggest you can be open mind and have a try this time.

from pyethereum.

Quoto @schmir

I'm probably ignorant, but the following looks a bit noisy and seems to introduce a lot of incidental complexity:

https://github.com/ethereum/pyethereum/blob/master/features/trie_hook.py#L40

What you think verbose is the following codes:

context.execute_steps(u'''
    Given a pair with key "AB"
    And a pair with key "AC"
    And a pair with key "ABCD"
    And a pair with key "ACD"
    And a pair with key "A"
    And a pair with key "B"
    And a pair with key "CD"
    And a pair with key "BCD"
    When clear trie tree
    And insert pairs
''')

Here I want to explain, that they actually can be written in a more programming style

for key in ["AB", "AC", "ABCD", "ACD", "A", "B", "CD", "BCD"]:
   context.execute_steps(u"Given a pair with {0}".format(key))
context.execute_steps(u'''
    When clear trie tree
    And insert pairs
''')

OR even did not execute steps like this, but use pure function call:

for key in ["AB", "AC", "ABCD", "ACD", "A", "B", "CD", "BCD"]:
     insert_func(context, key)

behave itself does not impose how you achieve this.
I use the most verbose form just because it has the most readability

from pyethereum.

quote @schmir

we have the "load_data" decorator in trie.features, which is implemented in trie_hook.py, and it's being injected manually in environment.py?

the load_data here is a tag, you can have multiple tags for each feature and scenario, and provide specific behavior for those tags.

The trie_hook is just act like setup/teardown for the trie tag. Of cause you can also set setup/teardown by name instead of tag.

I'll write some codes to make the hook auto discovered

from pyethereum.

As far as I can see the .feature files are a way to parametrize certain test functions in a subset of plain english. That might be nice if you have to work with "non-technical" people. I don't see the use case in this project.

from pyethereum.

quoto @schmir

 .feature files are a way to parametrize certain test functions 

Aha, I know why you are so opposit to BDD. The statement is abosultely wrong. You must have no experience with BDD before.

It's functionaility is not to parameterize test functions, in fact, it's act a description for the behavior of the unit to be testing, and the description is runable, files in the steps directory are only means to support the running of the description. Parameterize is just mean to reduce the verbosity of the feature.

quote @schmir

That might be nice if you have to work with "non-technical" people. 

As such, if we look at a unit test that passes, with no clear description of what the unit does other than the code in it, what have we got to contrast it against? That is why specifying the conditions under which an action takes place, describing the action taking place and defining the expected results serve as a means to verify that the code is doing what the specification is describing.

In Business applications it is true that not all developers are familiar with the domain they work on. However, in software development, where the domain is software, this also holds true.

from pyethereum.

Please be open minded and give it a try for some time.
If the majority of us are still opposite to behave after that(for example one month), we can shift back to py.test.
But....please have real experience first, so you can make meaningful comparision.

from pyethereum.

please take a look at brainbot-com@370d1c9

I've implemented the rlp behave tests in pytest. This is 87 lines of code vs 188 for behave step file for rlp plus 50 lines for the feature file.

I still think the feature files are a way to parametrize certain test functions and a way to break down those test functions into "steps" and smear it all over the place. Have a look at the above patch!

If the lack of description of what the unit test does bothers you, docstrings or comments may be a solution for that.

from pyethereum.

@schmir , I've read your version for rlp test.

Here is the new version of test codes with behave: 9405ff0

Note: the feature file act as a runable documentation, if you just compare the lines of python codes in function body, you will found the behave version is even less, because there is no need to write dst = rlp.encode(src) and res = rlp.decode(src) and other similar invokes for each of the scenario.

behave itself is simple to learn, so every one can get familiar within 20 minutes, it's also powerful and flexible enough to fit your own need.

I am building some infrastruction see #47, so it will be even more compfortable to write test codes.

from pyethereum.

The feature file now looks quite nice, but the step file is highly coupled to the feature file and vice versa. In addition each step may depend on the steps run before.

In the end it's twice the number of lines of code.

I think we should stop wasting each others time here. You've made your decision.

from pyethereum.

Quote @schmir

but the step file is highly coupled to the feature file

Yes, step file is expected to support the feature file, that's its role.

But the vice versa does not stand.
It's kinda like the relationship between interface and implementation.

Quote @schmir

In addition each step may depend on the steps run before. 

Yes, previous steps produce data in the context, and later steps consume those data.
It's exactly the core mechanism of behave

Quote @schmir

In the end it's twice the number of lines of code.

Yes, but if you put the same doc in py.test codes, the code lines will also double.

Though still don't like it, seems you do not oppose behave like before :-)
I'll finish the "hook auto discover" and perhaps some other features soon, then you will find writing test codes more smooth.

from pyethereum.

Sorry, if I haven't been clear: I'm still completely opposed to behave or BDD in general if it results in code like we currently have in the steps files.

from pyethereum.

As I said before, the most important thing is to have tests at all. While I can see the strengths of Behave in certain situations, py.test seems to be the current champion when it comes to coder acceptance (in this project). Let me therefore propose that we stick with both test frameworks for now.

While this might not the optimal solution for this issue, it might be one which helps to keep everyone motivated to contribute.

from pyethereum.

@heikoheiko agree. So every one can have fun now.

from pyethereum.