According to JUnit v4 XSD, having multiple system-out for a test-case is invalid.

Example to invalid XML:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite errors="0" failures="0" tests="6" skipped="0" name="test.ipsums" timestamp="2017-02-21T15:02:29">
    <properties>
      <property name="platform" value="dartium" />
    </properties>
    <testcase classname="ipsum" name="habla hable" time="0.015" />
    <system-out>Muhaha</system-out>
    <system-out>Another line</system-out>
  </testsuite>
</testsuites>

The converter should merge the prints of that belong to the same testsuite.

On my flutter project by using junitreport I am making junit report but it is XML,

 flutter test --machine | tojunit --output test.xml

Now by Ant, I want to make Html from it. test.xml is at the root of my project, here I made build.xml:

<project name="genTestReport" default="gen" basedir=".">
  <description>
    Generate the HTML report from JUnit XML files
  </description>

  <target name="gen">
    <property name="genReportDir" location="${basedir}/unitTestReports"/>
    <delete dir="${genReportDir}"/>
    <mkdir dir="${genReportDir}"/>
    <junitreport todir="${basedir}/unitTestReports">
      <fileset dir="${basedir}">
        <include name="test.xml"/>
      </fileset>
      <report format="frames" todir="${genReportDir}/html"/>
    </junitreport>
  </target>
</project>

Now I run this command in root folder:

alt@Alis-MBP rr-front % ant -buildfile build.xml
Buildfile: /Users/alt/Projects/rr-front/build.xml

gen:
   [delete] Deleting directory /Users/alt/Projects/rr-front/unitTestReports
    [mkdir] Created dir: /Users/alt/Projects/rr-front/unitTestReports
[junitreport] [Fatal Error] test.xml:264:23: Character reference "&#x1B" is an invalid XML character.
[junitreport] The file /Users/alt/Projects/rr-front/test.xml is not a valid XML document. It is possibly corrupted.

Does it seem I need some configuration ?

Inside test.xml I have some testcase like below:

<testcase 
classname=".Users.alt.Projects.rr_front.test.widgets.top3_pro.pro_container" name="overlay tap tap work" time="0.312">
      <system-out>&#x1B;[34m[🌎 Easy Localization] [WARNING] Localization key [widgets.pro_container.selected_a_pro] not found&#x1B;[0m</system-out>

As the title states. A failed test is being reported as an "Error" instead of a "Failure". See image below:

This seems odd to me, but maybe this is intentional? If it is intentional, what constitutes an "Error" vs a "Failure"?

This package doesn't yet have a null safe version available and it would be nice to have one.

My application has intl version 0.18 and we cannot downgrade the version, how can we update junitreporter

Our application depends on the intl 0.18.1 plugin, as Flutter SDK > 3.7 forces this.

Error message:

Because junitreport 2.0.2 depends on intl ^0.17.0 and no versions of junitreport match >2.0.2 <3.0.0, junitreport ^2.0.2 requires intl ^0.17.0.

Solution:
update extension to s intl ^0.18.1

Currently, the json input to this tool includes a url with the filepath of a ran test, something like this:

{"test":{"id":3,"name":"package test","suiteID":0,"groupIDs":[2],"metadata":{"skip":false,"skipReason":null},"line":4,"column":3,"url":"file:///root/project/flutter/package/test/package_test.dart"},"type":"testStart","time":8218}

However the output XML will only include the classname currently. Something like this:

<testcase classname="flutter.package.test.package" name="package test" time="0.042">
  <failure message="1 failure, see stacktrace for details">Failure:\n\nExpected: false Actual: <true> </failure>
</testcase>

I'd expect the output XML to look something like this instead:

<testcase file="/root/project/flutter/package/test/package_test.dart" classname="flutter.package.test.package" name="package test" time="0.042">
  <failure message="1 failure, see stacktrace for details">Failure:\n\nExpected: false Actual: <true> </failure>
</testcase>

I'm happy to throw up a PR to work on this as well.

Running dartanalyzer --strong ./ displays

6 errors, 15 warnings and 4 hints found.

Hi, Thanks for the great tool!

I like to put all my reports into a separate directory, ie reports.

Currently

tojunit -o reports/test.xml

will fail because reports does not exist.
Creating the directory ahead of time is trivial, it would be a nice experience if the tool would go ahead on make all the directories.

Cheers!

I'd like to avoid having the classes full paths as names. Is there a way to do that?

<testsuite errors="0" failures="0" tests="4" skipped="0" name=".Users.robsonribeiro.flutter.my_app.test.bottom_navigation.tab_navigator" timestamp="2021-04-08T15:11:48">

would be:

<testsuite errors="0" failures="0" tests="4" skipped="0" name=".bottom_navigation.tab_navigator" timestamp="2021-04-08T15:11:48">

I understand that the name comes from the json file generated by flutter test --machine but still I couldn't find a way to change it.

I'd be happy to have a PR open if that's the case and if you'd point the direction to achieve such thing. Thank you!

Hey thank you for your help, this is a great tool!
I was wondering if you could add stack trace data in the report?
thank you

<testcase classname="vault.range" name="CreateAccount()" time="0.062">
   <failure message="'5' != 1" type="AssertionError">
   <![CDATA[Traceback (most recent call last):
       package:test_api                
         expect
          test/range/range_test.dart 6:5 main.<fn>\n]]>		
   </failure>
</testcase>

When I have tried
flutter drive --target=test_driver/tests/splash_screen.dart --no-build --no-pub --machine > example.jsonl
I am getting this error
The --machine flag is only valid with the --version flag.

As integration tests can give a quite cumbersome system-out, if we execute a large number of tests the generated XML can exceed the memory limit. There should be the possibility to not populate the system-out but only the error node

We noticed today that all our fresh builds are failling after xml lib update to version 3.2.4

Hi running dart tests --reporter json or flutter flutter --machine results in \n characters in the failure messaging:

 {"testID":3,"error":"Expected: <false>\n  Actual: <true>\nmy failure reason\n","stackTrace":"package:test_api         expect\ntest\\main.test.dart 5:5  main.<fn>\n","isFailure":true,"type":"error","time":1171}

We're viewing the reports on CircleCI which is understandably rendering the characters as is.

I don't know if this is an issue for the dartlang project or if it is possible for this library to handle given that the output is a readable Junit report?

Expected
\n results in linebreaks in rendered xml

Actual
\n characters output in xml report

Version

pub -v
FINE: Pub 2.3.0

Do you have an example of using this for flutter tests?

I've tried flutter pub pub run test --reporter json | tojunit but running tests this way doesn't seem to work as flutter test does. I get errors like this:

 <error message="Failed to load foo_test.dart Unable to spawn isolate: dart:ui:1: Error: 
Not found: dart:ui.">dart:isolate/runtime/libisolate_patch.dart 457:14  Isolate.spawnUri

Before I dig further, wondering if you have a working example for flutter project.

I'm hoping to use this library to do builds in Bamboo. Thanks!

Dependencies are outdated, leading to conflicts when used in a project with other packages.

Package Name                Current   Upgradable  Resolvable  Latest   

direct dependencies:       
args                        *2.3.0    2.3.1       2.3.1       2.3.1    
xml                         *5.3.1    *5.4.1      6.1.0       6.1.0    

dev_dependencies:          
lints                       *1.0.1    *1.0.1      2.0.0       2.0.0    
test                        *1.20.1   1.21.4      1.21.4      1.21.4  

After running this command flutter test --machine | tojunit --output TEST-report.xml
I get an error Bad state: not started and empty TEST-report.xml file
At the same time this command flutter test --machine work well and writes correct results to console.

This behavior appeared after flutter update
flater version 2.0.3.
junitreport version 1.3.1.

When running tojunit the following error is thrown:

junitreport as globally activated doesn't support Dart 2.17.5, try: dart pub global activate junitreport

Flutter doctor output:

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.3.9, on macOS 13.0 22A380 darwin-x64, locale en-IL)
[!] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.
[✓] Xcode - develop for iOS and macOS (Xcode 14.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2021.3)
[✓] VS Code (version 1.73.1)
[✓] Connected device (3 available)
[✓] HTTP Host Availability

Hello,

for my use case I would like to use this package's functionality as library from my source code.

Right now most of this package's logic resides in the bin/tojunit.dart file. It would be more helpful to move the logic to the lib directory and have e.g., a CliRunner class that takes the arguments from the binary file. This would allow me to use the same logic as the binary includes right now.

For now I have to rebuild some functionalities (e.g. reading and then splitting per line my machine test output file) and also add additional dependencies such as "testreport", which is unnecessary since this package already contains this logic, but unfortunately inaccessible from source code.

Best regards!

flutter test --machine --coverage | tojunit > test-results.xml
Error Msg:

This library depends on xml: ^3.7.0 which is now quite old (latest version is 4.3.0). Depending on the older version is causing version conflicts in our apps which now need newer versions.

Would it be possible to relax the version requirement to allow the latest version of xml?

Thanks
Michael

Hi, we use a lot dart-junitreport. Thank you for creating it.

We are bumping flutter in our project and require some of the dependencies to be updated.

It would be perfect if the following dependencies are bumped to the last version:

  args: ^2.1.0
  intl: ^0.17.0
  xml: ^5.1.0

It looks like the input characters are not properly XML-encoded.
E.g. when we get this message from flutter test:

{
  "testID": 4,
  "messageType": "print",
  "message": "\u001b[38;5;196m⛔  2021-03-26 15:48:38.982 MockArticleData - Cannot load promotions\u001b[0m",
  "type": "print",
  "time": 2357
}

tojunit produces this illegal XML:

<system-out>�[38;5;196mâ›”  2021-03-26 15:14:50.656 MockArticleData - Cannot load promotions�[0m
....
</system-out>

The \u001b should be encoded.

I imagine it's been a while since pubspec.lock was last regenerated. It might be a good idea to update it.
With later versions of SDK, some packages just got newer versions (such as collection and test).

I've tested things locally, functionality does not seem to break.

Hello rspilker, we use this dart-junitreport tool in our Jenkins to generate junit-version xml test report. Recently we upgraded flutter to 2.13 to support null safety, but tool dart-junitreport doesn't provide a null safety version. So could you please help to update this tool to support null safety? Thx!

If skipped integration tests are present in the machine.log from flutter test --machine, they are marked as follows:

{"test":{"id":37,"name":"Test title","suiteID":0,"groupIDs":[2,32],"metadata":{"skip":true,"skipReason":null},"line":159,"column":5,"url":"package:flutter_test/src/widget_tester.dart"},"type":"testStart","time":554187}
{"testID":37,"result":"success","skipped":true,"hidden":false,"type":"testDone","time":554187}

They should be omitted from the resulting JUnit test report

Resolving dependencies...

• args 1.5.1
• charcode 1.1.2
• collection 1.14.11
• convert 2.1.1
• intl 0.15.7
• junitreport 0.3.3
• path 1.6.2
• petitparser 2.1.1
• testreport 0.3.1
• typed_data 1.1.6
• xml 3.3.0
  Precompiling executables...
  Failed to precompile junitreport:tojunit:
  file:///data/jenkins/.pub-cache/hosted/pub.dartlang.org/junitreport-0.3.3/lib/src/impl/xml.dart:6:8: Error: Error when reading 'file:///data/jenkins/.pub-cache/hosted/pub.dartlang.org/xml-3.3.0/lib/xml/visitors/pretty_writer.dart': No such file or directory
  import 'package:xml/xml/visitors/pretty_writer.dart';
  ^
  file:///data/jenkins/.pub-cache/hosted/pub.dartlang.org/junitreport-0.3.3/lib/src/impl/xml.dart:28:23: Error: Method not found: 'XmlPrettyWriter'.
  document.accept(new XmlPrettyWriter(buffer, 0, ' '));
  ^^^^^^^^^^^^^^^
  file:///data/jenkins/.pub-cache/hosted/pub.dartlang.org/junitreport-0.3.3/lib/src/impl/xml.dart:28:38: Error: Too many positional arguments: 0 allowed, but 3 found.
  Try removing the extra positional arguments.
  document.accept(new XmlPrettyWriter(buffer, 0, ' '));

Our application uses Logger package which color codes log output. This introduces Unicode character code in the XML output which are not encoded into and XML entity.

See attached log file

I am curious what your thoughts are on this. My question is two-fold:

1. Looking at what's checked into the repo, I am wondering what the purpose of example-tests is.
2. If it would worth the effort to write some kind of automated tests (would put my mind at ease)