Promote preconditions to test methods about ets-wfs20 HOT 6 CLOSED

One disadvantage of the suggested approach is that the precondition tests must be associated with a <test> tag (conformance class). That is, each precondition may be run (unnecessarily) multiple times rather than just once at the beginning of the test run. It might be better to add the preconditions to a mandatory conformance class and set a flag which is then read to determine whether other conformance classes are run or skipped.

from ets-wfs20.

Maybe call it: preconditions conformance class.

from ets-wfs20.

That would be one approach, but the problem of signalling an unsatisfied precondition between conformance classes remains. Since there is always at least one mandatory conformance class, including the preconditions there makes sense. For example:

<test name="Conformance Level 1">
    <packages>
      <package name="org.opengis.cite.abc10.level1" />
    </packages>
    <classes>
      <class name="org.opengis.cite.abc10.SuitePreconditions" />
    </classes>
  </test>

If a test method in SuitePreconditions detects that a precondition is not satisfied, then a suite attribute is set like so:

testContext.getSuite().setAttribute("preconditions-ok", Boolean.FALSE);

A @BeforeTest method defined for some other conformance class can then check the suite attribute:

Boolean preconditionsOk = (Boolean) testContext.getSuite().getAttribute("preconditions-ok");
Assert.assertTrue(preconditionsOk, "One or more test suite preconditions not satisfied.");

All tests in this conformance class are skipped if any precondition was not satisfied, and the failing test in the mandatory conformance class will indicate what the problem was.

Note: In the test suite definition, the suite/@configfailurepolicy attribute must have the value "skip".

from ets-wfs20.

Maybe, instead of packaging the preconditions as a conformance class, make them part of the most basic mandatory conformance class.

from ets-wfs20.

The advantage of breaking out the preconditions into a separate <test> group is to make them more visible in the report. And it emphasizes that these checks do not actually correspond to a conformance class defined in some base specification.

<test name="Preconditions">
    <classes>
        <class name="org.example.testng.demo.SuitePreconditions"/>
    </classes>
</test>

If a precondition is not satisfied, it is added to the value of the "failed-preconditions" attribute, and this is checked in subsequent BeforeTest methods in other conformance classes.

@BeforeTest
public void checkPreconditions(ITestContext context) {
    Object failedPreconditions = context.getSuite().getAttribute("failed-preconditions");
    if (null != failedPreconditions) {
        throw new SkipException("One or more preconditions not satisfied: " + failedPreconditions);
    }
}

The preconditions--and their descriptions--become much more visible in the report, thus eliminating a potential source of confusion regarding skipped tests.

from ets-wfs20.

How can differentiate these "Preconditions" from a conformance class. Both from a user interface point of view and a program parsing the results?

from ets-wfs20.