The idea of steps is the following. Often you want to hide certain details of a step, because they are not directly important for a certain scenario. For example:

when().I_fill_out_the_registration_form_with_valid_values();

will fill out the registration form with valid values, but the concrete values are not shown. If a developer is interested in the concrete values she can look into the implementation of the step. However, in the report these details are completely gone. Thus, if someone reads the report and he wants to know the values, he has to ask a developer.

Nested Steps

Nested steps are steps that happen within another step. For example, the above step could be defined as follows:

public SELF I_fill_out_the_registration_form_with_valid_values() {
    return I_enter_as_name("Franky")
        .and().I_enter_as_email_address("[email protected]")
        .and().I_enter_as_password("password1234")
        .and().I_enter_as_repeated_password("password1234");
}

The idea is now that in the generated report these sub-steps can be made visible somehow. For example by expanding the parent step. That way, the original scenario stays small, but the details can still be looked up.

Annotation

Nested steps should get an additional annotation @NestedSteps to indicate that the step contains sub-steps.

In order to get a better overview over my Scenarios,
As a developer,
I want to have a generated Statistics page in the HTML report of JGiven

Consider the following scenarios:

Given a failing test for an unimplemented feature
and the test has a @NotYetImplemented annotation
When JGiven is run
Then the test is executed
and the test passes
and the output makes it clear that this test passes because it actually fails, but has the annotation

Given a passing test for an implemented feature
and the test has a @NotYetImplemented annotation
When JGiven is run
Then the test is executed
and the test fails
and the output makes it clear that this test fails because it claims not to be implemented yet

Would this maybe be a good idea? Currently, annotating tests with @notimplementedyet is a bit of an annoyance because a developer needs to remove the annotation to run the test and see if it passes.
The new behavior would also enforce the TDD idea, that the test must fail before the implementation is done (as a "passing" test with annotation would actually fail).

When an exception is thrown in an @AfterStage annotated method, that exception is not logged.

Currently test classes and tags in the navigation bar at the left are not further structured. When the number of classes and tags get large, there is no overview anymore.
Test-Classes could be structured by their package name, this could even be hierarchical. Tags should at least be structured by their type.

It is already possible to provide an extended description to steps by using the @ExtendedDescription annotation. However, sometimes you want to comment a certain step-invocation and not all invocations of that step. This comment should also appear in the report.

Example:

given().something();
when().some_step()
  .and().some_surprising_step()
   .comment("a comment is especially important here because the step surprises the reader")
  then().something();

Since the switch to Gradle the code coverage is at 95% where it was at 89% before, this seems to be wrong.

In com.tngtech.jgiven.impl.util.ReflectionUtil there is a reference to log4j. This seems to be inconsistent with the usage of slf4j in the rest of the project. This means I can't use JGiven in a project that uses a different slf4j back-end.

Switching to the slf4j Logger seems to be trivial.

When using the JUnit Parameterized Runner, the different cases appear in multiple scenarios with the same name instead in a single scenario.

Given a Scenario with HTML tags in step or method parameters
when the HTML report is generated
then the parameters are escaped in the resulting report

The following code does not work as expected as the @Description annotation is ignored:

@Test
@Description("My 'special' description")
public void my_special_description() {
   ...
}

Currently the JSON report model of JGiven only supports strings as arguments of step methods. However, it might be useful to think about additional data types. One example would be to support lists or tables. Depending on the report generator, e.g. text or html, different representations could be chosen.

A table data type would enable to write something like this:

given().the_following_users_are_logged_in( new Object[][] { 
 { "First Name", "Last Name", "Email" },
 { "Joe", "Doe", "[email protected]" },
 { "Lisa", "May", "[email protected]"}});

The corresponding step method would define the argument as a table:

public SELF the_following_users_are_logged_in( @Table Object[][] data ) {
...
}

In the JSON model this data must be stored in a special way, so that the report generators can present the content accordingly.

Given the following JGiven test:

   TestStage stage = addStage(TestStage.class);
   stage.some_step_with_a_$_parameter(test);

then the text report prints:

   some step with a'test' parameter

instead of

   some step with a 'test' parameter

Given a scenario
and the scenario has a tag with value #42
and the HTML5 report has been generated
when trying to click on the corresponding tag
then the tag page is not opened

It would sometimes be useful to have stages that live longer than a single scenario. For example, to do expensive setups that can be reused between tests. In JUnit this is typically done by using @BeforeClass annotations. However, that requires to have static fields which has several drawbacks.

Idea: Introduce a new kind of stage, namely singleton stages. Singleton stages can live longer than a single scenario and are reused between scenarios. In contrast to normal stages, a singleton stage can only be injected, it cannot be a real scenario stage.

• Singleton stages can provide values to other stages, but no values are injected back into a singleton stage
• Singleton stages only support @BeforeStage, @AfterStage, and @ScenarioRule life-cycle anotations.

Example:

public class MySingletonStage {
    @ProvidedScenarioState
    protected ExpensiveRessource expensiveRessource;

    @BeforeStage
    protected void setupExpensiveRessource() {
         expensiveRessource = ... // create ressource
    }

    @AfterStage
    protected void shutdownExpensiveRessource() {
         expensiveRessource.shutdown();
    }
}

Usage in other stage:

public class MyNormalStage {
    @SingletonStage
    protected MySingletonStage mySingletonStage;

    @ExpecedScenarioState
    protected ExpensiveRessource expensiveRessource;


}

Given a failing scenario defined in a JUnit test method
and a @After-annotated method that throws an exception
when executing the test
then only the exception of the @After-annotated method is thrown
and the exception of the failing scenario is not visible

As scenarios are now initially collapsed it is difficult to see whether a scenario consists of multiple cases or not. To solve this one could indicate the number of cases in the headline of the scenario.

Current Behavior

Given a scenario
  and the scenario has one step with an empty string as argument
when creating the HTML report
  then the empty string is not visible in the report

Expected Behavior

Given a scenario
  and the scenario has one step with an empty string as argument
when creating the HTML report
  then the empty string is displayed as '<empty>' in a gray color

Given a scenario
and a stage has an AfterScenario method
When a step throws an exception
Then the AfterScenario method is not called, but should be called

The content could then be included in the generated report.

When two scenario cases have a structural difference, so that a data table cannot be created, highlight the difference so that the reader can faster grasp the difference.

Example:

Case 1:
Given something
When foo
Then bar

Case 2:
Given something
When foo
   and buzz  <-- highlight this line
Then bar

Given a WhenExecuteApplication stage with a hidden method

@Hidden
public WhenExecuteApplication an_error_is_expected() {
    return this;
}

and a test with the first call to the When stage being the hidden method

@Test
public void test() {
    given().nothing();

    when().an_error_is_expected()
        .and().something_happens();

    then().nothing();
}

when the test is executed

then the output discards the "when" in the output

Given nothing
And something happens
Then nothing

The correct output instead should be

Given nothing
When something happens
Then nothing

The @Description tag on test methods and step methods is somehow misnamed. In fact it is not a description but the title. So instead a @Title annotation should be introduced and the @Description tag should be deprecated. In a next step the @Description annotation should get the meaning of the @ExtendedDescription annotation.

Excerpt from jgiven-core/target/surefire-reports/TEST-com.tngtech.jgiven.report.model.ReportModelBuilderTest.xml

  <testcase name="printf_annotation_uses_the_PrintfFormatter" classname="com.tngtech.jgiven.report.model.ReportModelBuilderTest" time="0.04">
    <failure message="expected:&lt;&apos;5[.]20&apos;&gt; but was:&lt;&apos;5[,]20&apos;&gt;" type="org.junit.ComparisonFailure"><![CDATA[org.junit.ComparisonFailure: expected:<'5[.]20'> but was:<'5[,]20'>
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at com.tngtech.jgiven.report.model.ReportModelBuilderTest.printf_annotation_uses_the_PrintfFormatter(ReportModelBuilderTest.java:191)
]]></failure>
  </testcase>

Take the following example:

@Test
public void test(String param) {
   given().something(param);
   when().something(param);
}

then the resulting data table contains the param twice, where it should only contain it once.

As a developer,
I want to be able to organize my tags hierarchically,
so that I get a better overview over my tags.

Idea:
Allow tags to be tagged by meta-tags. These meta-tags should also appear in the report and should allow for a hierarchical structuring of the tags in the report.

Example:
In order to hierarchically structure features under the meta-tag @Feature. We define an annotation @Feature:

@IsTag
@Retention( RetentionPolicy.RUNTIME )
public @interface Feature { }

Then we annotated feature tags with that tag:

@Feature 
@IsTag( type = "Feature", value = "Data Tables",
    description = "In order to get a better overview over the different cases of a scenario<br>"
            + "As a human,</br>"
            + "I want to have different cases represented as a data table" )
@Retention( RetentionPolicy.RUNTIME )
public @interface FeatureDataTables { }

The resulting report should then show something like this:

Feature
 - Data Tables
 - ...

It should also be collapsible

With an @ExtendedDescription tag on test methods it would be possible to explain in more detail the rational behind a scenario.

Given more than 800 JGiven scenarios
when trying to open the resulting HTML5 report
then Firefox throws an "script too large" error
and does not render the report

Replace the given Maven build system with Gradle

Given a scenario where the last stage has a method annotated with AfterStage
when the scenario is executed
then the AfterStage method of the last stage is not executed as expected.

Given a step method that takes a primitive array as parameter
When the step method is executed in a Scenario
Then an exception is thrown with error message "[I cannot be cast to [Ljava.lang.Object"

It would be nice if one could add an extended description to a step method. This could be shown in the report as a tooltip, for example. The extended description could provide more detailed information about a step.

Example:

class GivenCustomer {

   @ExtendedDescription("Sets up a default customer in the database."
      + "<br>The customer has name 'John' and lastname 'Doe' " 
      + "as well as email address '[email protected]'");
   public void a_customer() {
       ...
   }
}

For sure it would be nice if instead the JavaDoc of the method could be used, but that would require a separate parsing of the Java source, so this would be rather difficult to achieve without major effort.

Now that the new HTML5 Report exists, it is not necessary anymore to have a sophisticated static html report. Instead the static html report should be very basic again. This means:

• Remove search functionality
• Remove collapsing of scenarios
• Remove dependency to font awesome
• Remove the navigation panel on the left, instead just provide a breadcrumbs line

When trying to use JGiven together with version 1.9.0 of the JUnit DataProvider, the following exception is thrown:

java.util.EmptyStackException
    at java.util.Stack.peek(Stack.java:85)
    at com.tngtech.jgiven.junit.ScenarioReportRule.getTestCaseModel(ScenarioReportRule.java:25)
    at com.tngtech.jgiven.junit.ScenarioTest.<init>(ScenarioTest.java:13)
    at com.tngtech.jgiven.junit.DataProviderTest.<init>(DataProviderTest.java:22)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:195)
    at org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:244)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:241)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at com.tngtech.java.junit.dataprovider.DataProviderRunner$2.evaluate(DataProviderRunner.java:162)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

The command to run the HTML generator in the readme is:

java com.tngtech.jgiven.report.html.HtmlReportGenerator [--dir=<jsonreports>] [--toDir=<targetDir>]

But should be (todir instead of toDir)

java com.tngtech.jgiven.report.html.HtmlReportGenerator [--dir=<jsonreports>] [--todir=<targetDir>]

Currently the elapsed time is shown in milliseconds, with large numbers, this is hard to read for humans. Thus the unit should be adapted to the value.

Given the following scenario:

@Test
public void some_scenario() {
   given().something();

   assertThat(true).isFalse();
}

In practice a scenario could fail for other reasons, for example JUnit rules.

In the HTML report the scenario is marked as success, but still a failure message is shown. Expected behavior: the overall scenario is marked as failed.

The scenario also does not appear in the 'failed.html' page

Currently JGiven creates data tables if parameters of cases are different. However, if arguments differ that are no parameters, data tables are not generated.
Example:

Case 1: a = 5
Given some 5
When foo 6
Then bar

Case 2: a = 7
Given some 7
When foo 8
Then bar

As the argument of foo is not a parameter, no data table is generated.

Desired output:

Given some <a>
When foo <b>
Then bar

|  a  |  b  |
|  5  |  6  |
|  7  |  8  |

The name of the variable should be derived from the variable name of the called method

In a printed report the scenarios should not be collapsed (new feature in v0.4.0)

Given a scenario with parameters 'param1'
and the scenario has 2 cases
and case 1 has argument 'baz1'
and case 2 has argument 'baz2'
and all cases have a step 'foo' with argument 'bar'
when creating the text report
then the report contains 'foo '
but it should contain 'foo 'bar''