Cucumber domain language: scenarios vs. test cases about cucumber-ruby-spike-donotuse HOT 8 CLOSED

I like the unit of execution metaphor. Or maybe "execution unit". In grammar:

execution_unit ::= execution_step*
execution_step ::= hook | background_step | scenario_step | compiled_scenario_outline_step
compiled_scenario_outline_step = "scenario outline step with values from the current example row substituted"

I don't think it's valuable to report "3 test cases from 1 scenario outline"

from cucumber-ruby-spike-donotuse.

I like the idea too.

But the word "test" scares me a little. Just for the same reasons as to why TDD has a bad name (i.e. focus on testing instead of describing behaviour, etc.).

So yes I think it's nice to remove the confusion around the word "scenario" and no, I don't think it should be exposed to end-users.

from cucumber-ruby-spike-donotuse.

On 13 Oct 2011, at 17:05, Mike Sassak wrote:

I'm about to start driving out the reporter / formatter API, and I'd like to discuss our language a bit, particularly how it relates to reporting and execution. In 1.x the unit of execution is the scenario, but that overloaded the idea of scenario (e.g. a scenario in Cucumber contains the steps from the Gherkin scenario and the Gherkin background), so in cucumber-ruby the test case is the basic unit of execution, and the Gherkin input is compiled into test cases. My question is if this aspect of Cucumber is worth exposing to our technical users. If you executed a scenario outline with 3 examples, and the results said, "3 test cases (3 passed)", would you be confused? Maybe something like "3 test cases from 1 scenario outline"? FWIW I find it very helpful to think of Cucumber as a Gherkin compiler and executor, but from my perspective I would not be surprised if I have missed the forest for the trees. WDYT?

I spotted that and I've been thinking about it for a while. I think it's useful to have a separate term for the executable thing that can pass or fail, abstract from the way it was expressed in the Gherkin specification file (as a Scenario or ExampleRow). I would prefer 'example' myself, but I think test case is OK too.

from cucumber-ruby-spike-donotuse.

Thanks guys. @mattwynne I thought of using example, but decided on test case because the name made it very distinct from what was contained in the Gherkin.

from cucumber-ruby-spike-donotuse.

Strictly speaking the thing in the Gherkin is an ExampleRow. In my way of thinking, Scenarios and ExampleRows generate Examples, which are the things that you test. I can see that the similarity might confuse some people.

For me, Example is better than TestCase is better than Scenario. So TestCase is OK, but wouldn't it be nice if the summary at the bottom were the same as RSpec's, where it talks about 'examples'?

Alternatively, we could also call them checks!
http://www.developsense.com/blog/2009/08/testing-vs-checking/

from cucumber-ruby-spike-donotuse.

On Oct 15, 2011, at 23:17, Matt Wynne
[email protected]
wrote:

Strictly speaking the thing in the Gherkin is an ExampleRow. In my way of thinking, Scenarios and ExampleRows generate Examples, which are the things that you test. I can see that the similarity might confuse some people.

For me, Example is better than TestCase is better than Scenario. So TestCase is OK, but wouldn't it be nice if the summary at the bottom were the same as RSpec's, where it talks about 'examples'?

No, that would confuse the living hell out of people.

90% of cucumber tests are regular Scenarios, and that's what people
think of them as.

ExampleRow is the exception, and that's just a special kind of Scenario.

"Examples" is a syntax element, and overloading the meaning of the
word would be disastrous.

Users don't care about how we name classes inside Cucumber, but they
care what the report says.

Let's us "Scenario".

Aslak

Alternatively, we could also call them checks!
http://www.developsense.com/blog/2009/08/testing-vs-checking/

Reply to this email directly or view it on GitHub:
cucumber/cucumber-ruby#16 (comment)

from cucumber-ruby-spike-donotuse.

@mattwynne I think the wires got crossed when I said "what was contained in the Gherkin". I meant by that the names for all the things that have a representation in Gherkin-the-language, e.g. Scenario, Scenario Outline, Feature, Background, Examples, ExampleRow, DataTable, DocString, and so on. As the name of the unit of execution, I prefer TestCase to Example because "Example" sounds like it fits in that group of things, whereas TestCase sticks out like a sore thumb.

from cucumber-ruby-spike-donotuse.

Interesting article on testing vs. checking, btw. Makes me think of epistemological falsifiability: http://en.wikipedia.org/wiki/Falsifiability. If we could find a way to express system properties that we believe always hold true, it would be super neat to then automate attempts at their falsification.

from cucumber-ruby-spike-donotuse.