keithmorris / gulp-nunit-runner Goto Github PK

gulp.task("run-tests", function () {
	var nunitConfig = {
		executable: paths.tools.nunit,
		stoponerror: true,
		work: paths.dist.tests
	};
	
	console.log(nunitConfig);

    return gulp.src(paths.tests.assembly, {read: false})
        .pipe(nunit(nunitConfig));
});

Now I can see the output shows _dist/tests as the directory passed into work but it never produces the TestResults.xml file there it just outputs it in the root of the gulp execution point (i.e where the gulpfile.js lives).

I have tried setting output and result properties to a path and filename, like _dist/tests/test-result.xml etc and that is ignored too.

As the output data in the console is:

Run Settings
    DisposeRunners: True
    WorkDirectory: c:\blahblah\Visual Studio 2015\Projects\Projects\blah // this should be blah/_dist/tests
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.6.1
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    NumberOfTestWorkers: 8

You have to use the where switch. Unfortunately, the include and exclude switches are not included in NUnit3.

However, I have managed to find a workaround. I tried to use where: 'cat == LogicOnly', but this includes the quotes in the switch:

Test Filters
    Where: 'cat == LogicOnly'

Unexpected token 'cat == LogicOnly' at position 0 in selection expression.

Using an array doesn't add quotes. So you can do something like this and it works:

gulp.task('nunit3',
    function() {
        return gulp
            .src('./**/*Tests.dll', {read: false})
            .pipe(nunit({
                executable: './Application/packages/NUnit.Console.3.0.1/tools/nunit3-console.exe',
                options: {
                    where: ['cat == LogicOnly'],
                    framework: 'net-4.5',
                    labels: labelTests
                }
            }));
    });

The solution is to either to add a where switch that does not include the quotes in the switch or to add a more sophisticated set of parameters that corresponds to the NUnit Test Selection Language.

So I'm moving to gulp and was going to migrate my grunt-nunit-runner task but saw you already had one out there. Wanted to see if you were open to a PR with some enhancements instead of creating yet another gulp nunit plugin. Here's what I had in mind:

• Add build script with test, lint and watch.
• Add Travis CI integration.
• Add a TeamCity reporter.
• Add support for multi options e.g. include, exclude and privatebinpath.
• Add the ability to 1) omit the explicit nunit path entirely and rely on it being in the PATH and 2) just supply the path to the bin folder but not explicitly specify the filename. I have a platform flag in the Grunt task that goes along with both these to chose the x86 or anycpu version of nunit; defaults to anycpu.
• Document all the config options in the README (Like this). The NUnit docs on the options are more prose than a list which makes it difficult to quickly scan them.

Anyways, this is a lot of stuff so if you want pass on a PR for these I totally understand.

I notice you support the switches with mono. I gave this a quick run and i think you may need to execute mono directly.

exe which is run needs to be mono,
(https://github.com/keithmorris/gulp-nunit-runner/blob/develop/lib/index.js#L140)

and the location of the nunit runner will become a parameter.

With the 'executable' option now assuming a path and not an executable the task can not find the nunit runner on OS X which installs to /usr/local/bin/nunit-console by default. Ideally one could really specify an executable and not a path or .exe would be dropped on OS X.

gulp-nunit-runner is now the only dep I have pulling in vulnerable dependencies. Please update and re-publish.

This repo looks like it's been inactive for a while. If you don't have time to maintain the package, please contact me: I'm willing to take over, as I already did for gulp-msbuild.

Thanks.

flatmap-stream has been removed from npm so the following command
npm install gulp-nunit-runner
results in the following error
404 Not Found: flatmap-stream

If the supplied executable path has a space in it, ie C:\Projects\Foo Bar\Nunit\nunit.exe, the runner fails:

'C:\Projects\Foo' is not recognized as an internal or external command,
operable program or batch file.

Running the gulp task from the visual studio task runner fails with the exception:

Error: Implement me. Unknown stdin file type!
    at process.stdin (node.js:673:17)
    at run (node_modules\gulp-nunit-runner\index.js:125:18)
    at Stream.end (node_modules\gulp-nunit-runner\index.js:35:3)

Seems the trouble is that the visual studio task runner doesn't support stdin, so the following line fails.

    opts = {
        stdio: [process.stdin, process.stdout, process.stderr, 'pipe']
    };

I was able to work around the issue by changing it to

    opts = {
        stdio: [null, process.stdout, process.stderr, 'pipe']
    };

Not sure if this is a viable solution, but maybe since I don't think the runner would need to support input.

I am currently using the nunit runner for various tests, but some of the specflow ones need to be run through the specflow reporter.

Problem seems to be that the specflow reporter needs you to do:

nunit3-console.exe --labels=All --out=TestResult.txt "--result=TestResult.xml;format=nunit2" bin\Debug\BookShop.AcceptanceTests.dll

I cannot find a way to specify the format=nunit2 though, as if I try appending it to the end of the result file:

result: "api-test-results.xml;format=nunit2",

it just blows up saying Unknown result format: nunit2,

So is there any way to get this working with the runner? as I would rather use this than directly use command line.

It looks like the parseSwitches() function is ignoring options passed in as number types. For example, in the options specified below, the number of workers is not passed through as an argument (it works if passed through as a string).

nunit({
    executable: 'nunit3-console.exe',
    options: {
        workers: 3,
        framework: 'net-4.5',
        teamcity: true,
        test: namespaces,

    }
})

My build fails intermittently on TeamCity 9.0.3 (latest at time of writing) with this message (but never in my command line window):

[16:10:08][Step 1/1] child_process.js:1155
[16:10:08][Step 1/1]     throw errnoException(err, 'spawn');
[16:10:08][Step 1/1]           ^
[16:10:08][Step 1/1] Error: spawn EBADF
[16:10:08][Step 1/1]     at exports._errnoException (util.js:746:11)
[16:10:08][Step 1/1]     at ChildProcess.spawn (child_process.js:1155:11)
[16:10:08][Step 1/1]     at Object.exports.spawn (child_process.js:988:9)
[16:10:08][Step 1/1]     at run (C:\TeamCity\buildAgent\work\243a5b945bf3db45\node_modules\gulp-nunit-runner\index.js:116:28)
[16:10:08][Step 1/1]     at Stream.end (C:\TeamCity\buildAgent\work\243a5b945bf3db45\node_modules\gulp-nunit-runner\index.js:31:3)
[16:10:08][Step 1/1]     at _end (C:\TeamCity\buildAgent\work\243a5b945bf3db45\node_modules\gulp-nunit-runner\node_modules\event-stream\node_modules\through\index.js:65:9)
[16:10:08][Step 1/1]     at Stream.stream.end (C:\TeamCity\buildAgent\work\243a5b945bf3db45\node_modules\gulp-nunit-runner\node_modules\event-stream\node_modules\through\index.js:74:5)
[16:10:08][Step 1/1]     at DestroyableTransform.onend (C:\TeamCity\buildAgent\work\243a5b945bf3db45\node_modules\gulp\node_modules\vinyl-fs\node_modules\through2\node_modules\readable-stream\lib\_stream_readable.js:523:10)
[16:10:08][Step 1/1]     at DestroyableTransform.g (events.js:199:16)
[16:10:08][Step 1/1]     at DestroyableTransform.emit (events.js:129:20)
[16:10:08][Step 1/1] Process exited with code 1

This is the gulp task:

gulp.task('test', ['restoreSlnPackages', 'build'], function () {
  return gulp
    .src(['src/**/bin/' + configuration + '/*.Tests.dll'], { read: false })
    .pipe(nunit({
      executable: 'src/packages/NUnit.Runners.2.6.4/tools/nunit-console.exe'
    }));
});

It might have something to do with the fact that my nunit executable is in the solution packaged path.

I'm not even sure this is a gulp-nunit-runner issue, but I thought I might report it here in case anyone could shed any light on the situation.