Should act as the 'test goal' see #10

maven plugin 4.5.1 is now released. If necessary, create a branch for 4.5.0 support.

This is an old request that appears several times in the soapui forum

• http://www.soapui.org/forum/viewtopic.php?f=5&t=17374
• http://www.soapui.org/forum/viewtopic.php?f=5&t=17784
• http://www.soapui.org/forum/viewtopic.php?f=2&t=4149
• http://www.soapui.org/forum/viewtopic.php?f=2&t=17209
• http://www.soapui.org/forum/viewtopic.php?f=5&t=19591

We should add a JunitReportCollector and make it be used by the TestCaseRunner.
Be carreful to not break the security-test goal which also depends on the same collector.

For implementation discussion, see: http://www.soapui.org/forum/viewtopic.php?f=5&t=19820

Currently, it is possible to generate a war based on soapui mock services

• from the UI (see http://www.soapui.org/Service-Mocking/deploying-mock-services-as-war-files.html)
• from the command line wargenerator.sh (or wargenerator.bat) script packaged with the UI (see http://www.soapui.org/Test-Automation/mock-services.html and http://www.soapui.org/forum/viewtopic.php?f=2&t=10858). The script is located in $SOAPUI_HOME/bin

So, this is not easy to integrate the war generation in a maven build and it will be nice to have a new goal to do the job.

This is supposed to have been fixed with #7, but junit dependency has not been added at that time.

See http://www.soapui.org/forum/viewtopic.php?f=2&t=8052&p=23287&hilit=maven+missing+dependencies#p24975

Because of this, we get an exception "groovy.lang.MissingMethodException: No signature of method" when registrating a jdbc driver via the new API (see Soapui forum).

This API - com.eviware.soapui.support.GroovyUtils.registerJdbcDriver() - has been added in soapui 3.6.1.
To fix the issue, we need to force the use of soapui 3.6.1 dependencies

This is already available from command line (http://www.soapui.org/Test-Automation/mock-services.html) but this parameter does not exist in SmartBear plugin implementation.

Should act as the 'test goal' see #11

With #38, given a property name, we are able to override this property in all test suites of the project.

We should now add an option to restrict this overriding to a given testsuite.

This is already displayed with maven 3

[INFO] --- maven-soapui-extension-plugin:4.0.0.1-SNAPSHOT:test (project-with-groovy) @ groovy-with-soapui-extension ---
SoapUI Pro 4.0.0 Maven2 TestCase Runner

but not when running the plugin with maven 2

[INFO] [soapui-pro:test {execution: project-with-groovy}]
SoapUI Pro 4.0.0 Maven2 TestCase Runner

So, add this information to the runner name

If the output parameter is not set, files are generated in project basedir.

proposition for default value:

${project.build.directory}/soapui/test-output

see

• soapui forum: http://www.soapui.org/forum/viewtopic.php?t=10857
• soapui pro forum: http://www.soapui.org/forum/viewtopic.php?f=2&t=12077
• http://blog.hlyh.dk/2011/10/using-soapui-maven-integration.html

The problem

Currently, it is possible to configure the destination directory for almost all log files by doing the following (the trailing / is a mandatory in the log output directory)

<plugin>
  <groupId>eviware</groupId>
  <artifactId>maven-soapui-plugin</artifactId>
  <version>4.0.1</version>
  <configuration>
    <soapuiProperties>
      <property>
        <name>soapui.logroot</name>
        <value>${project.build.directory}/soapui/my_logs/</value>
      </property>
    </soapuiProperties>
  </configuration>
</plugin>

But this should be done in all projects that used the soapui maven plugin.

The proposed solution

All log files should be generated in ${project.build.directory}/soapui/logs/ by default to avoid configuring this in all maven projects

We are going to need this because when want to add feature to standard runner as for the oss runner.
For instance, we need it to develop #39.

As for #38, we need to modify the log4j configuration file and update the integration tests that rely on the name of the runner for test sucess validation.

Currently, we can only run

• all suites
• a single suite (with the testSuite parameter)

Now, we would like to run more than one suite. This has been requested in the soapui forum (at least in):

• pro: http://www.soapui.org/forum/viewtopic.php?f=3&t=8701
• oss:
  • http://www.soapui.org/forum/viewtopic.php?f=1&t=13725&p=39948
  • http://www.soapui.org/forum/viewtopic.php?f=1&t=14992&p=39951

Possible configuration

<testSuites>
  <testSuite></testSuite>
</testSuites>

If the testSuite parameter is also used, then the test suite will runned as if it has been defined with the testSuites parameter.

The pro runner generates by default an html report when the "junitReport" parameter is set to true. It also generates an xml file. See the post of the SmartBear Support which validates this behaviour.

This xml file mess the surefire-plugin-report or jenkins as it duplicates junit report already generated. So the tests seems to be duplicated.
See this soapui forum post for more information.

With this new parameter, we provide a way to avoid this duplication.

Use same configuration as for the 'test goal'. See #14

1.8 version availability: see maven central

When upgrading, remove the workaround of http://jira.codehaus.org/browse/MINVOKER-137

Details of the workaround
Previously, with maven 2.2.1, mergeUserSettings option generates a NPE (java 5 and 6)

java.lang.NullPointerException
        at org.apache.maven.settings.SettingsUtils.merge(SettingsUtils.java:110)
        at org.apache.maven.plugin.invoker.AbstractInvokerMojo.runBuilds(AbstractInvokerMojo.java:1035)
        at org.apache.maven.plugin.invoker.AbstractInvokerMojo.execute(AbstractInvokerMojo.java:670)
        at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
        at hudson.maven.agent.PluginManagerInterceptor.executeMojo(PluginManagerInterceptor.java:182)

A workaround has been done in our project (we do not use the mergeUserSettings parameter, a profile is defined in a super parent pom used by all pom in integration tests)

Add basic support for PRO runner as done in #38.

because we support maven 2.2.1, we got in trouble with

• unable to use enum parameter as it has been introduced with maven 3 see MNG-4292. Force us to use String parameter for runnerType and projectConversionType
• a bug with the maven-invoker-plugin (see #30)
• unable to use List and Set param, see #108

Should act as the 'test goal' see #11

Currently, we only display version number of the plugin and the soapui version when starting a runner (this is duplicated in all runners)

[INFO] You are using maven-soapui-extension-plugin 4.5.1.1
SoapUI Pro 4.5.1 Maven2 TestCase Runner

We should

• not display soapui version when starting the runner but when displaying plugin version
• also display build date and commit id

[INFO] You are using maven-soapui-extension-plugin 4.5.1.1 (c92693761816e569f4feefa06c38000fa46e7fb4; 2012-11-28 10:11:06.016 +0100)
[INFO] Soapui version: 4.5.1
SoapUI Pro Maven2 TestCase Runner

Implementation note: see #29
Commit id: short?

Currently (at least from 3.5.1 to 4.0.1 version), this file is always generated in ${project.basedir} even if the "soapui.logroot" property is set (see #5 for a configuration example)

An issue is still opened for this (see http://sourceforge.net/tracker/index.php?func=detail&aid=2882169&group_id=136013&atid=737766) even if it seemed to have been fixed in the unreleased 3.6.2 version.
There is a workaround but again, you must do it in all projects (use a specific log4j configuration file)

• comments in http://sourceforge.net/tracker/index.php?func=detail&aid=2882169&group_id=136013&atid=737766
• http://www.soapui.org/forum/viewtopic.php?f=5&t=6949

maven plugin 4.5.0 is now released. If necessary, create a branch for 4.0.x support.

Should act as the 'test goal' see #5
When implemented, update the log documentation

Even if SmartBear says that soapui 4.5.1 can be run with java 6:

• the windows standalone distribution embedded a jre7 version
• some bug only occurs when running with jre6 (see this bug when using ssl for instance)

So we could remove support of java6 build setting target to 1.7 when compiling the code.

See this post on Soapui forum.

This issue will add a basic support for this feature

• properties will be set/overriden in all test suites of the project
• this will only work with the OSS runner

4.0.1 (open source and pro version)
junit 4.4 http://www.soapui.org/forum/viewtopic.php?f=13&t=6552&p=30551&hilit=maven+missing+dependencies + see #18

Currently, we rely on the SmartBear implementation for this goal.
We need to provide our own implementation to be able to add features

To be started when #15 is completed.
Add a parameter to specify the conversion type, by default it will be set to a composite to standard conversion

Currently, when running the "test" goal, we have 2 options:

• by default, the build fails just after errors occur
• if the "testFailIgnore" is set to true, the build does not fail after plugin execution but we do not have any way to easily know if there are errors

Usually, there is only a soapui project that stores the tests, so the first option is ok.
But when there are several projects, this is different.

Soapui currently does not support multi projects. The workaround with maven is to configure several executions.
See http://www.eviware.com/forum/viewtopic.php?f=14&t=12052 or http://www.soapui.org/forum/viewtopic.php?f=14&t=5381#p17979
In this case, we must set the "testFailIgnore" parameter to true to be sure that all projects are run but we cannot get the status of the runs.

To fix this, I propose to add a new "test-verify" goal that will check if there have been errors in previous test runs.
This means that at each runs, the "test" goal must set something in the maven project context to be used later by the "test-verify" goal (setting a property to true on errors seems to be ok)

This duplicates the error log, so remove it as we did for the test goal (see #2)

        try {
            runner.run();
        } catch (Exception e) {
            getLog().error(e.toString());
            throw new MojoFailureException(this, "SoapUI MockService(s) failed", e.getMessage());
        }

affected version: 4.5.1.1 (maybe also 4.5.1.0)

see test ouputs

[INFO] You are using maven-soapui-extension-plugin 4.5.1.1-SNAPSHOT
SoapUI Pro 4.5.1-SNAPSHOT Maven2 TestCase Runner

The 'skip' parameter is not documented in smartbear documentation so update our wiki page

Note: this parameter is currently not documented in soapui website

OSS code: the skip parameter exist in java mojo implementation but is not exposed as goal parameter - missing annotation (see https://github.com/SmartBear/soapui-maven-plugin/blob/edff7a7209308847811b303a3a5612f2ff28e9a4/src/main/java/com/eviware/soapui/maven2/MockServiceMojo.java#L146)

On error/failure, standard maven-soapui-plugin displays the details of the error, for instance, the request, the response, some logs.
This is very annoying when several tests fails: the console is flooded with those useless messages.
We already have the printReport parameter to get a summary of test execution and report files are also generated to know error causes .

So, remove these error details.

Let us use our configuration file

My first private implementation has been done over soapui 3.6.1 version. The customer I worked for at that time would like to use the open source implementation to get rid of the private implementation (and also get new features now in maven-soapui-extension-plugin).

Soapui 3.6.1 requires java 5 (not Java 6) and does not work really well with Java 7 (see Soapui forum and soapui 4.5.0 release notes)

In Smartbear implementation, default value is ${project.artifactId}-soapui-project.xml
This bug affects all versions of maven-soapui-extension-plugin

Cause: the build filters the plugin.xml file so ${project.artifactId} is filtered in the generated file

For example, see first failure at BuildHive: https://buildhive.cloudbees.com/view/My%20Repositories/job/redfish4ktc/job/maven-soapui-extension-plugin/1/console
As soapui dependencies are not in Central, BuildHive can not access to them and the build fails.

in 4.0.0 (pro version)
www.soapui.org/forum/viewtopic.php?f=2&t=8052&p=23287&hilit=maven+missing+dependencies

When setting the skip parameter with the standard maven-soapui-plugin, plugin only exits.
It should log a message as maven-surefire-plugin does

Currently, the plugin use the pro runner.
Add a parameter that will let the user choose the open source runner.

Proposition
parameter name: runnerType
values: an Enum of PRO, OSS

The current artifactId does not follow maven convention

[ERROR]`Artifact Ids of the format maven-___-plugin are reserved for
plugins in the Group Id org.apache.maven.plugins
Please change your artifactId to the format ___-maven-plugin
In the future this error will break the build.

The update could be done when

• the following soapui pull request has been merged or implemented in another way
• the new release of the plugin integrates the change

Be carreful, we need to document this change in the README/wiki. Expecially, document starting which version the modification applies.

Should act as the 'test goal' see #5
When implemented, update the log documentation

Currently, use system out to log key/value.
Example

Setting soapui.scripting.library value /dev/maven-soapui-extension-plugin/target/it/groovy_log_directory/soapui-pro-4.0.0/../soapui-conf/groovy_scripts
Setting soapui.logroot value  /dev/maven-soapui-extension-plugin/target/it/groovy_log_directory/soapui-pro-4.0.0/target/soapui/logs

Should use the maven logger instead

Currently, we cannot open project that requires a password.
This will be ok with this new parameter

At least usefull for our integration tests.
Currently, we did not test our plugin with composite projects.

With this new goal, we could convert the "standard" soapui projects and execute integration tests with them.

As for SoapUITestCaseRunner, we will need to add features.
We may have to update the integration tests is some rely on the name of the runner for test sucess validation.

As for SoapUITestCaseRunner, we will need to add features.