Giter Club home page Giter Club logo

nyc's Introduction

nyc

ci NPM version Conventional Commits

Having problems? want to contribute? join our community slack.

Istanbul's state of the art command line interface, with support for:

  • applications that spawn subprocesses.
  • source mapped coverage of Babel and TypeScript projects

How Istanbul works

Istanbul instruments your ES5 and ES2015+ JavaScript code with line counters, so that you can track how well your unit-tests exercise your codebase.

The nyc command-line-client for Istanbul works well with most JavaScript testing frameworks: tap, mocha, AVA, etc.

Installation & Usage

Use your package manager to add it as a dev dependency: npm i -D nyc or yarn add -D nyc. You can use nyc to call npm scripts (assuming they don't already have nyc executed in them), like so (replace mocha with your test runner everywhere you see it):

{
  "scripts": {
    "test": "mocha",
    "coverage": "nyc npm run test"
  }
}

You can use also npx instead of installing nyc as a dependency, but you might get updates you are not ready for; to get around this, pin to a specific major version by specifying, e.g. nyc@14.

{
  "scripts": {
    "test": "npx nyc@latest mocha"
  }
}

This is a good way of testing upcoming releases of nyc, usually on the next tag.

Note: If you use jest or tap, you do not need to install nyc. Those runners already have the IstanbulJS libraries to provide coverage for you. Follow their documentation to enable and configure coverage reporting.

Configuring nyc

nyc accepts a wide variety of configuration arguments, run npx nyc --help for thorough documentation.

Configuration arguments on the command-line should be provided prior to the program that nyc is executing. As an example, the following command executes ava, and indicates to nyc that it should output both an lcov (lcov.info + html report) and a text-summary coverage report.

nyc --reporter=lcov --reporter=text-summary ava

Babel projects

Please start with the pre-configured @istanbuljs/nyc-config-babel preset. You can add your custom configuration options as shown below.

TypeScript projects

Please start with the pre-configured @istanbuljs/nyc-config-typescript preset.

Adding your overrides

nyc allows you to inherit other configurations using the key extends in the package.json stanza, .nycrc, or YAML files. You can then add the specific configuration options you want that aren't in that particular shared config, e.g.

{
  "extends": "@istanbuljs/nyc-config-typescript",
  "all": true,
  "check-coverage": true
}

Configuration files

Any configuration options that can be set via the command line can also be specified in the nyc stanza of your package.json, or within a separate configuration file - a variety of flavors are available:

File name File Association
.nycrc JSON
.nycrc.json JSON
.nycrc.yaml YAML
.nycrc.yml YAML
nyc.config.js CommonJS export

Common Configuration Options

See nyc --help for all options available. You can set these in any of the files listed above, or from the command line. This table is a quick TLDR for the rest of this readme and there are more advanced docs available.

Option name Description Type Default
all Whether or not to instrument all files (not just the ones touched by your test suite) Boolean false
check-coverage Check whether coverage is within thresholds, fail if not Boolean false
extension List of extensions that nyc should attempt to handle in addition to .js Array<String> ['.js', '.cjs', '.mjs', '.ts', '.tsx', '.jsx']
include See selecting files for coverage for more info Array<String> ['**']
exclude See selecting files for coverage for more info Array<String> list
reporter May be set to a built-in coverage reporter or an npm package (dev)dependency Array<String> ['text']
report-dir Where to put the coverage report files String ./coverage
skip-full Don't show files with 100% statement, branch, and function coverage Boolean false
temp-dir Directory to output raw coverage information to String ./.nyc_output

Configuration can also be provided by nyc.config.js if programmed logic is required:

'use strict';

const defaultExclude = require('@istanbuljs/schema/default-exclude');
const isWindows = require('is-windows');

let platformExclude = [
  isWindows() ? 'lib/posix.js' : 'lib/win32.js'
];

module.exports = {
  exclude: platformExclude.concat(defaultExclude)
};

Publish and reuse your nyc configuration(s)

To publish and reuse your own nyc configuration, simply create an npm module that exports your JSON config (via index.json or a CJS index.js).

A more advanced use case would be to combine multiple shared configs in a nyc.config.js file:

'use strict';

const babelConfig = require('@istanbuljs/nyc-config-babel');
const hookRunInThisContextConfig = require('@istanbuljs/nyc-config-hook-run-in-this-context');

module.exports = {
  ...babelConfig,
  ...hookRunInThisContextConfig,
  all: true,
  'check-coverage': true
};

Selecting files for coverage

By default, nyc only collects coverage for source files that are visited during a test. It does this by watching for files that are require()'d during the test. When a file is require()'d, nyc creates and returns an instrumented version of the source, rather than the original. Only source files that are visited during a test will appear in the coverage report and contribute to coverage statistics.

nyc will instrument all files if the --all flag is set or if running nyc instrument. In this case all files will appear in the coverage report and contribute to coverage statistics.

nyc will only collect coverage for files that are located under cwd, and then only files with extensions listed in the extension array.

You can reduce the set of instrumented files by adding include and exclude filter arrays to your config. These allow you to shape the set of instrumented files by specifying glob patterns that can filter files from the default instrumented set. The exclude array may also use exclude negated glob patterns, these are specified with a ! prefix, and can restore sub-paths of excluded paths.

Globs are matched using minimatch.

We use the following process to remove files from consideration:

  1. Limit the set of instrumented files to those files in paths listed in the include array.
  2. Remove any files that are found in the exclude array.
  3. Restore any exclude negated files that have been excluded in step 2.

Using include and exclude arrays

If there are paths specified in the include array, then the set of instrumented files will be limited to eligible files found in those paths. If the include array is left undefined all eligible files will be included, equivalent to setting include: ['**']. Multiple include globs can be specified on the command line, each must follow a --include, -n switch.

If there are paths specified in the exclude array, then the set of instrumented files will not feature eligible files found in those paths. You can also specify negated paths in the exclude array, by prefixing them with a !. Negated paths can restore paths that have been already been excluded in the exclude array. Multiple exclude globs can be specified on the command line, each must follow a --exclude, -x switch.

The default exclude list is defined in the @istanbuljs/schema module. Specifying your own exclude property completely replaces these defaults.

For example, the following nyc config will collect coverage for every file in the src directory regardless of whether it is require()'d in a test. It will also exclude any files with the extension .spec.js.

{
  "all": true,
  "include": [
    "src/**/*.js"
  ],
  "exclude": [
    "**/*.spec.js"
  ]
}

Note: Be wary of automatic OS glob expansion when specifying include/exclude globs with the CLI. To prevent this, wrap each glob in single quotes.

Including files within node_modules

We always add **/node_modules/** to the exclude list, even if not specified in the config. You can override this by setting --exclude-node-modules=false.

For example, "excludeNodeModules: false" in the following nyc config will prevent node_modules from being added to the exclude rules. The set of include rules then restrict nyc to only consider instrumenting files found under the lib/ and node_modules/@my-org/ directories. The exclude rules then prevent nyc instrumenting anything in a test folder and the file node_modules/@my-org/something/unwanted.js.

{
  "all": true,
  "include": [
    "lib/**",
    "node_modules/@my-org/**"
  ],
  "exclude": [
    "node_modules/@my-org/something/unwanted.js",
    "**/test/**"
  ],
  "excludeNodeModules": false
}

Setting the project root directory

nyc runs a lot of file system operations relative to the project root directory. During startup nyc will look for the default project root directory. The default project root directory is the first directory found that contains a package.json file when searching from the current working directory up. If nyc fails to find a directory containing a package.json file, it will use the current working directory as the default project root directory. You can change the project root directory with the --cwd option.

nyc uses the project root directory when:

  • looking for source files to instrument
  • creating globs for include and exclude rules during file selection
  • loading custom require hooks from the require array

nyc may create artifact directories within the project root, with these defaults:

  • the report directory, <project-root>/coverage
  • the cache directory, <project-root>/node_modules/.cache/nyc
  • the temp directory, <project-root>/.nyc_output

Require additional modules

The --require flag can be provided to nyc to indicate that additional modules should be required in the subprocess collecting coverage:

nyc --require esm mocha

Interaction with --all flag

The --require flag also operates on the main nyc process for use by --all. For example, in situations with nyc --all --instrument false and babel-plugin-istanbul setup the --all option only works if --require @babel/register is passed to nyc. Passing it to mocha would cause the tests to be instrumented but unloaded sources would not be seen. The @istanbuljs/nyc-config-babel package handles this for you!

Caching

nyc's default behavior is to cache instrumented files to disk to prevent instrumenting source files multiple times, and speed nyc execution times. You can disable this behavior by running nyc with the --cache false flag. You can also change the default cache directory from ./node_modules/.cache/nyc by setting the --cache-dir flag.

Coverage thresholds

You can set custom coverage thresholds that will fail if check-coverage is set to true and your coverage drops below those thresholds. For example, in the following nyc configuration, dropping below 80% branch, line, functions, or statements coverage would fail the build (you can have any combination of these):

{
  "branches": 80,
  "lines": 80,
  "functions": 80,
  "statements": 80
}

To do this check on a per-file basis (as opposed to in aggregate), set the per-file option to true.

High and low watermarks

Several of the coverage reporters supported by nyc display special information for high and low watermarks:

  • high-watermarks represent healthy test coverage (in many reports this is represented with green highlighting).
  • low-watermarks represent sub-optimal coverage levels (in many reports this is represented with red highlighting).

You can specify custom high and low watermarks in nyc's configuration:

{
  "watermarks": {
    "lines": [80, 95],
    "functions": [80, 95],
    "branches": [80, 95],
    "statements": [80, 95]
  }
}

Parsing Hints (Ignoring Lines)

There may be some sections of your codebase that you wish to purposefully exclude from coverage tracking, to do so you can use the following parsing hints:

  • /* istanbul ignore if */: ignore the next if statement.
  • /* istanbul ignore else */: ignore the else portion of an if statement.
  • /* istanbul ignore next */: ignore the next thing in the source-code ( functions, if statements, classes, you name it).
  • /* istanbul ignore file */: ignore an entire source-file (this should be placed at the top of the file).

Ignoring Methods

You can ignore every instance of a method simply by adding its name to the ignore-class-method array in your nyc config.

{
  "ignore-class-method": ["render"]
}

Combining reports from multiple runs

If for whatever reason you have different test runners in your project or a different series of test runs for different kinds of tests, nyc will automatically combine the coverage report for you if configured correctly with the --no-clean flag and the report command. Originally inspired by @janiukjf in #1001, here's an example, where the test:* scripts (not shown) invoke only your test runner(s) and not nyc:

{
  "scripts": {
    "cover": "npm run cover:unit && npm run cover:integration && npm run cover:report",
    "cover:unit": "nyc --silent npm run test:unit",
    "cover:integration": "nyc --silent --no-clean npm run test:integration",
    "cover:report": "nyc report --reporter=lcov --reporter=text"
  }
}

What about nyc merge?

The nyc merge command is for producing one raw coverage output file that combines the results from many test runs. So if you had the above setup and needed to produce a single coverage.json for some external tool, you could do:

{
  "scripts": {
    "cover:merge": "npm run cover:unit && npm run cover:integration && nyc merge .nyc_output coverage.json"
  }
}

Source-Map support for pre-instrumented codebases

If you opt to pre-instrument your source-code (rather than using a just-in-time transpiler like @babel/register) nyc supports both inline source-maps and .map files.

Important: If you are using nyc with a project that pre-instruments its code, run nyc with the configuration option --exclude-after-remap set to false. Otherwise nyc's reports will exclude any files that source-maps remap to folders covered under exclude rules.

Integrating with TAP formatters

Many testing frameworks (Mocha, Tape, Tap, etc.) can produce TAP output. tap-nyc is a TAP formatter designed to look nice with nyc.

Tutorials and Advanced Documentation

See more nyc tutorials and advanced nyc documentation.

Please feel free to contribute documentation to help us improve.

nyc for enterprise

Available as part of the Tidelift Subscription.

The maintainers of nyc and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source dependencies you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact dependencies you use. Learn more.

nyc's People

Contributors

addaleax avatar andrewfinlay avatar bcoe avatar brettz9 avatar calvinf avatar coreyfarrell avatar danielruf avatar dependabot[bot] avatar gr2m avatar greenkeeperio-bot avatar isaacs avatar jakxz avatar jamestalmage avatar jasisk avatar lalem001 avatar linusu avatar lloydcotten avatar novemberborn avatar rapzo avatar revelt avatar rmg avatar rundef avatar ryanv avatar schutm avatar scottrudiger avatar shackpank avatar simenb avatar tehsenaus avatar wbyoung avatar xhmikosr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nyc's Issues

Support for async functions

Right now nyc fail while handling reverse source-mapping when the original source contains async functions like this one:

export function func() {
    return new Promise((resolve, reject) => {
        console.log('test');
        resolve(true);
    });
}
/tmp/test-cover/node_modules/nyc/node_modules/source-map/lib/array-set.js:91
    throw new Error('No element indexed by ' + aIdx);
    ^

Error: No element indexed by 1
    at ArraySet_at [as at] (/tmp/test-cover/node_modules/nyc/node_modules/source-map/lib/array-set.js:91:11)
    at SourceMapConsumer_originalPositionFor [as originalPositionFor] (/tmp/test-cover/node_modules/nyc/node_modules/source-map/lib/source-map-consumer.js:618:36)
    at mapLocation (/tmp/test-cover/node_modules/nyc/lib/source-map-cache.js:33:25)
    at /tmp/test-cover/node_modules/nyc/lib/source-map-cache.js:93:18
    at Array.forEach (native)
    at SourceMapCache._rewriteStatements (/tmp/test-cover/node_modules/nyc/lib/source-map-cache.js:92:40)
    at /tmp/test-cover/node_modules/nyc/lib/source-map-cache.js:23:11
    at Array.forEach (native)
    at SourceMapCache.applySourceMaps (/tmp/test-cover/node_modules/nyc/lib/source-map-cache.js:15:23)
    at NYC.writeCoverageFile (/tmp/test-cover/node_modules/nyc/index.js:263:25)
    at EventEmitter.onExit.alwaysLast (/tmp/test-cover/node_modules/nyc/index.js:240:11)

Immediate failure without further info

I ran into a strange bug that will kill nyc right away without any error message.

Just comment this line out and running that test will kill nyc (that line should actually be deleted but the tests won't run afterwards anymore).

Script for automatically "smoke testing" against a set of libraries, looking for regressions.

#108 (comment)

I think this could be an awesome general purpose module. My thoughts:

  • It needs a simple config file that lists the projects that depend on the one you want to "smoke test".
  • Mimic .travis.yml for configuring scripts (just the script names from the travis Build Lifecyle, not all their config options).
  • The config file would need to point to the repository and branch to clone.
  • The module would clone each individual downstream dependent in a temp directory and run the scripts described above, npm linking the module under test.
  • If there are any failures, it will open the temp folder in your Finder/File Explorer, and list the failing dependents (maybe deleting the folders of passing dependents).

Folder structure:

some/temp/dir/
   downstream-dependent-1/
     ...
   downstream-dependent-2/
     ...

// @bcoe @novemberborn

I think we have a need for something like this for AVA as well.

// @sindresorhus @vdemedes

Maximum call stack size exceeded

Hello.

I'm trying to use nyc with Babel.
But I got a stackoverflow error.
What am I doing wrong?

> nyc --require babel-register node_modules/.bin/mocha test.js

path.js:174
  resolvedTail = normalizeArray(resolvedTail.split(/[\\\/]+/),
                                             ^

RangeError: Maximum call stack size exceeded
    at String.split (native)
    at Object.win32.resolve (path.js:174:46)
    at Object.win32.relative (path.js:259:16)
    at getRelativePath (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\babel-register\lib\node.js:72:28)
    at shouldIgnore (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\babel-register\lib\node.js:123:12)
    at require.extensions.(anonymous function) (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\babel-register\lib\node.js:137:9)
    at requireHook (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\nyc\index.js:139:7)
    at require.extensions.(anonymous function) (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\babel-register\lib\node.js:138:7)
    at requireHook (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\nyc\index.js:139:7)
    at require.extensions.(anonymous function) (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\babel-register\lib\node.js:138:7)

Windows 7 64bit
Node v4.2.2
npm v3.5.0

package.json

{
  "name": "nyc-example",
  "version": "0.0.0",
  "private": true,
  "description": "Please run `npm install` and `npm test`.",
  "scripts": {
    "test": "nyc --require babel-register node_modules/.bin/mocha test.js"
  },
  "devDependencies": {
    "babel-core": "^6.3.2",
    "babel-preset-es2015": "^6.1.18",
    "babel-register": "^6.3.2",
    "mocha": "^2.3.4",
    "nyc": "^4.0.1"
  }
}

.babelrc

{
  "presets": ["es2015"]
}

test.js

class Foo {
}

console.log(Foo);

nyc-test wraps itself again and again

Multiple tests in test/nyc-test instantiated NYC and call wrap(), wrapping the current process. This leads to poor test isolation. Each test should probably be run in its own process.

Feature request: supress test output

Running nyc tape test/*.js, I see the output of both the tests and the coverage report. When I run the coverage reporter, I'm not actually interested in the test output. It would be nice if there was an option like --silent for the test command itself, to suppress its output and only have the coverage report printed to my terminal.

feature request: fail on user-defined minimum coverage level

It'd be cool to specify (via env probably, so as to not mess with args?) a minimum level of statement/branch coverage, below which it's considered a failure.

Coveralls provides this, so the travis/coveralls wire-up I've got for node-tap meets my needs for PRs and such. But it'd be good to surface this to a potential contributor ahead of time. Like, "Yes, your tests pass, but you dropped the coverage below acceptable amounts, so fix that first."

Specify output folder

Is there any way to control where it spits out the generated report without involving .istanbul.yml?

new versions of nyc break tap test suite

the tap suite runs several tests on require-hooks itself, these seem to be failing on newer versions of nyc:

test/require-hooks.js ................................. 4/6 6s
  compile-to-js require hook > failing test with unmatched stack-trace
  not ok error happened in the *.faux file
    found: ''
    pattern: 'file: test/fixtures/using-require-hook.faux'
    at:
      file: test/require-hooks.js
      line: 9
      column: 1000
      function: verifyOutput
    source: |
      test('compile-to-js require hook', function (t) {
    stack: |
      verifyOutput (test/require-hooks.js:9:1000)
      ChildProcess.exithandler (child_process.js:204:5)
      maybeClose (internal/child_process.js:764:16)
      Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)

  compile-to-js require hook > failing test with unmatched stack-trace
  not ok error happened in the *.faux file
    found: >
      /Users/benjamincoe/bcoe/node-tap/test/fixtures/using-require-hook.js



          1..0



      not ok 1 -
      /Users/benjamincoe/bcoe/node-tap/test/fixtures/using-require-hook.js



        ---



        results:



          ok: false



          count: 0



          pass: 0



          plan:



            start: 1



            end: 0



            skipAll: true



        arguments:



          - /Users/benjamincoe/bcoe/node-tap/test/fixtures/using-require-hook.js



        exitCode: 1



        timeout: 240000



        command: /Users/benjamincoe/.nvm/versions/io.js/v3.2.0/bin/iojs



        file: /Users/benjamincoe/bcoe/node-tap/test/fixtures/using-require-hook.js



        ...







      1..1
    pattern: 'file: test/fixtures/using-require-hook.faux'
    at:
      file: test/require-hooks.js
      line: 9
      column: 1000
      function: verifyOutput
    source: |
      test('compile-to-js require hook', function (t) {
    stack: |
      verifyOutput (test/require-hooks.js:9:1000)
      ChildProcess.exithandler (child_process.js:204:5)
      maybeClose (internal/child_process.js:764:16)
      Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)

I bet this is being caused by the overall of require-hook that we've been iterating on.

It would be great to get this fixed so that @isaacs can pull the new improved version into tap.

Any thoughts: @jamestalmage, @novemberborn?

Ignoring code

Hi! Are there any alternatives to /* istanbul ignore next */ and similar? Doesn't seem like the Istanbul variant is taken into consideration at least.

cache instrumentation for speed.

It would be interesting to profile how much time is spent applying the istanbul transform and consider caching results across processes. This would help with test suites that spawn/fork a lot. AVA's tests currently spawn about 70 times, each one loading up at least 3 files that need to be instrumented.

yarsay tests fail when running with --cache flag

@jamestalmage I know this is something that was fixed at one point, but I can't seem to get the lcov report working if --cache is set for yarsay.

Benjamins-MacBook-Pro:yarsay benjamincoe$ nyc report --reporter=lcov
/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:241
            text = structuredText[startLine].text;
                                            ^

TypeError: Cannot read property 'text' of undefined
    at /Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:241:45
    at Array.forEach (native)
    at annotateFunctions (/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:224:26)
    at HtmlReport.Report.mix.writeDetailPage (/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:427:9)
    at /Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:489:26
    at SyncFileWriter.extend.writeFile (/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/util/file-writer.js:57:9)
    at FileWriter.extend.writeFile (/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/util/file-writer.js:147:23)
    at /Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:488:24
    at Array.forEach (native)
    at HtmlReport.Report.mix.writeFiles (/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:482:23)

I rolled back several commits, and it doesn't seem as though this was ever working on master.

Fix remaining tests.

There are a few tests I think I did a poor adapting to fit our new testing strategy. @novemberborn had an alternate test strategy PR that we didn't go with, but I think it did adapt a few of the tests better.

We need to make sure we identify those and merge them in.

See:

add tests for ./bin/nyc.js

It would be nice to have some unit tests for the nyc bin, I'm fine with these simply using spawn and asserting against stdout/stderr.

`--all` appears to be broken in 5.1.0

In [email protected], I am getting the following error whenever --all is used:

user$nyc --all ava 'test/**/*.js'
fs.js:549
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: ENOENT: no such file or directory, open '/Users/user/project/.nyc_output/7910.json'
    at Error (native)
    at Object.fs.openSync (fs.js:549:18)
    at Object.fs.writeFileSync (fs.js:1156:15)
    at NYC.writeCoverageFile (/Users/user/project/node_modules/nyc/index.js:259:6)
    at NYC.addAllFiles (/Users/user/project/node_modules/nyc/index.js:171:8)
    at Object.<anonymous> (/Users/user/project/node_modules/nyc/bin/nyc.js:122:23)
    at Module._compile (module.js:435:26)
    at Object.Module._extensions..js (module.js:442:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:313:12)

Not sure where to start looking on this issue.

Add better examples to the documentation

Update from moderator:

The biggest issue is that our documentation shows global use of nyc without ever suggesting that nyc be installed globally.

Original Message:

I am not quite sure how the instructions in the README ever worked. npm i nyc installs the module just locally and nothing gets added to the path. Calling

nyc tap ./test/*.js

gives a "command not found" (to no surprise). One would think that

./node_modules/nyc/bin/nyc.js tap ./test/*.js

should work. But instead that gives

    $ ./node_modules/nyc/bin/nyc.js tap ./test/*.js
    events.js:141
          throw er; // Unhandled 'error' event
          ^

    Error: spawn tap ENOENT
        at exports._errnoException (util.js:874:11)
        at Process.ChildProcess._handle.onexit (internal/child_process.js:178:32)
        at onErrorNT (internal/child_process.js:344:16)
        at doNTCallback2 (node.js:439:9)
        at process._tickCallback (node.js:353:17)
        at Function.Module.runMain (module.js:469:11)
        at startup (node.js:134:18)
        at node.js:961:3

remove del dependency

We already have rimraf as a dependency, and we're not using del's glob functionality -- we can probably use rimraf where we currently use delete.

Problem with absolute path appended after relative one

I have a test file that require a relative file (the file to test) https://github.com/MoOx/statinamic/blob/19fc8085ceb14a55a149feb330526f4275889cfa/src/PageContainer/__tests__/component.js#L8
But when I run test using nyc babel-tape-runner src/**/__tests__/*.js (or nyc --require babel-core/register tape src/**/__tests__/*.js) I got weird result that include the entire filename after the relative parent folder like this

...

1..23
# tests 23
# pass  23

# ok

-------------------------------------------------------------------------------------------------------------------|----------|----------|----------|----------|----------------|
File                                                                                                               |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
-------------------------------------------------------------------------------------------------------------------|----------|----------|----------|----------|----------------|
 PageContainer/Users/MoOx/Sync/Development/statinamic/src/PageContainer/                                           |    73.33 |    71.88 |      100 |    73.33 |                |
  component.js                                                                                                     |    73.33 |    71.88 |      100 |    73.33 |... 79,80,84,87 |
 PageContainer/__tests__/Users/MoOx/Sync/Development/statinamic/src/PageContainer/__tests__/                       |      100 |      100 |       25 |      100 |                |
  component.js                                                                                                     |      100 |      100 |       25 |      100 |                |

When I check the report, I can clearly see the issue

$ nyc report --reporter=text-lcov
TN:
SF:./src/PageContainer/__tests__/Users/MoOx/Sync/Development/statinamic/src/PageContainer/__tests__/component.js
FN:15,noop
FN:16,Page
FN:17,PageError
FN:19,(anonymous_5)
FNF:4
FNH:1
...

Note that es6 import syntax is transpiled by babel to the same relative path

$ babel src/PageContainer/__tests__/component.js 

...

var _react2 = _interopRequireDefault(_react);

var _reactAddonsTestUtils = require("react-addons-test-utils");

var _component = require("../component"); /// <- same relative path

What am I doing wrong to get this weird path?

Hide the `nyc_output` folder

Make it nyc_output => .nyc_output.

It's only a temp folder for reports so there's nothing for end-users to see there.

use `--` to prevent `nyc` from processing further args.

Trying to use latest (master) nyc with a custom tap reporter:

$ nyc tap --reporter=spec

Things work fine until the end, when nyc tries to find a spec reporter.

The typical solution is to stop processing after a -- delimiter is found

$ nyc tap -- --reporter=spec

Feature request: ability to stop nyc process propagation

It would be nice if there was some environment variable that could be set that would disable the nyc injection in a child process (and any of its descendants).

The use case is for writing multi-process tests for things like process supervisors where instrumenting both the supervisor and the example app it is running is needless overhead when what is being tested is how the supervisor responds in integration tests.

I'm thinking something like NYC_DISABLE or something, or maybe a max depth on the number of process boundaries to cross.

fails when running `npm test`

Hi

I'm using nyc with mocha and coveralls
I've setup my .travis.yml and package.json as specified on the readme

The test is passing on travis and test coverage are send to coveralls.
But when I do npm test on my computer, it fails

> [email protected] test C:\perso\node-annotation-parser
> nyc mocha

events.js:85
      throw er; // Unhandled 'error' event
            ^
Error: spawn mocha ENOENT
    at exports._errnoException (util.js:746:11)
    at Process.ChildProcess._handle.onexit (child_process.js:1053:32)
    at child_process.js:1144:20
    at process._tickCallback (node.js:355:11)
    at Function.Module.runMain (module.js:503:11)
    at startup (node.js:129:16)
    at node.js:814:3
npm ERR! Test failed.  See above for more details.

The repo: https://github.com/mastilver/node-annotation-parser

nyc-test needs tap --coverage

$(npm bin)/tap test/nyc-test.js fails, where $(npm bin)/tap --coverage test/nyc-test.js succeeds. This seems wrong, tap comes bundled with nyc for its own coverage reports but that shouldn't impact the tests.

Report has bogus uncovered lines report

Files in .nyc_output are at https://gist.github.com/aredridel/67a635a727047e541c32

Output:

:; nyc report
------------|----------|----------|----------|----------|----------------|
File        |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
------------|----------|----------|----------|----------|----------------|
 __root__/  |    94.92 |    81.13 |      100 |    94.92 |                |
  common.js |    94.37 |    69.23 |      100 |    94.37 | 94,146,150,154 |
  index.js  |    95.74 |    92.59 |      100 |    95.74 |          48,68 |
------------|----------|----------|----------|----------|----------------|
All files   |    94.92 |    81.13 |      100 |    94.92 |                |
------------|----------|----------|----------|----------|----------------|

What about my untested files?

Hi,

So I tried out nyc on my project that I test with tape. I noticed that it doesn't report about files that are never touched by unit tests. Is this expected behavior? Everything looks very green, when in reality some files aren't yet tested at all.

nyc fails with "Unexpected token ILLEGAL" when `exec`ing shell script

As first reported via https://twitter.com/boennemann/status/619103847043203072

I'm trying to instrument the integration tests for https://github.com/semantic-release/semantic-release.
They are running a shell script to start the npm-registry-couchapp before tap/nixt tests are running against it. Everything works fine, but once I run them with nyc I get this error (multiple times):

[eval]:1
/private/tmp/node-spawn-wrap-17602-9d862ee799ec/node
                                   ^
SyntaxError: Unexpected token ILLEGAL
    at Object.exports.runInThisContext (vm.js:53:16)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (module.js:426:26)
    at node.js:567:27
    at doNTCallback0 (node.js:408:9)
    at process._tickCallback (node.js:337:13)

Steps to reproduce:

git clone [email protected]:semantic-release/semantic-release.git
cd semantic-release
git fetch origin  next:next
git checkout next
npm install
npm run build && npm run test:build
npm run test:unit # nyc built in, passes and reports coverage
npm run test:integration # passes
nyc npm run test:integration # fails

You need couchdb installed. Sorry for the bloated test case, will try to reduce it if this doesn't get you anywhere.

The relevant script is here: https://github.com/semantic-release/semantic-release/blob/next/test/registry/start.sh
Which gets execd here: https://github.com/semantic-release/semantic-release/blob/next/test/registry/index.js#L9

I see coverage reports for my node_modules

The README suggests that this shouldn't happen.

I notice I see it for the only one of my deps that are npm linked into my project.

Its not a big deal, I can visually skip over it the parts I don't care of, but is this intended behaviour?

Split out the caching and source-map-fixing into separate modules.

NYC now does 3 things:

  1. Installs an (optionally cacheable) transform via append-transform that captures coverage and source-map data.
  2. Remaps istanbul reports using source-map data.
  3. Uses spawn-wrap to make all this happen across forked processes.

I think 1 and 2 need to be split out.

Clean state before every test.

There is at least one test that still requires existing state from a previous test.

I propose we wipe out .nyc_output and test/fixtures/.nyc_output between tests, as well as the cache directories.

Any tests that fail due to these changes should be updated to explicitly generate their prerequisite state.

babel support broken on master

there was a regression with the most recent patches that landed on master:

screen shot 2015-12-14 at 9 20 22 pm

EDIT: looks like the remapping branch hasn't landed yet -- nothing is immediately jumping out at me as to why this is now cropping up.

.//node_modules/babel-cli/node_modules/convert-source-map/index.js:  // //# sourceMappingURL=foo.js.map                       /*# sourceMappingURL=foo.js.map */

Leading to:

[Error: An error occurred while trying to read the map file at /Users/benjamincoe/bcoe/yarsay/node_modules/babel-core/node_modules/convert-source-map/foo.js.map

Reproducing

This issue can be reproduced by running tests on this library:

https://github.com/bcoe/yarsay

CC: @novemberborn, @jamestalmage

Incorrect? coverage report with child_process.exec

I'm trying out nyc on my project https://github.com/Janpot/fantastiq with nyc npm test which yields:

-----------------|----------|----------|----------|----------|----------------|
File             |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
-----------------|----------|----------|----------|----------|----------------|
 bin/            |      100 |      100 |      100 |      100 |                |
  fantastiq.js   |      100 |      100 |      100 |      100 |                |
 lib/            |    29.05 |    10.98 |     8.21 |    29.11 |                |
  Queue.js       |    39.73 |       20 |     12.5 |       40 |... 412,416,420 |
  QueueClient.js |    51.11 |        0 |       10 |    51.11 |... 50,52,58,59 |
  Worker.js      |    15.19 |        0 |        0 |    15.19 |... 131,134,139 |
  index.js       |    56.52 |      100 |    16.67 |    56.52 |... 29,30,31,37 |
  jobUtils.js    |    30.43 |     7.14 |       25 |    30.43 |... 36,38,39,40 |
  message.js     |       12 |        0 |        0 |       12 |... 32,33,34,39 |
  metrics.js     |    41.67 |       25 |    33.33 |    41.67 |... 32,33,34,37 |
  router.js      |    10.91 |        0 |        0 |    10.91 |... 247,253,255 |
-----------------|----------|----------|----------|----------|----------------|
All files        |    35.44 |    15.85 |    16.33 |    35.51 |                |
-----------------|----------|----------|----------|----------|----------------|

This looked very odd to me as I expected my coverage to be closer to 100%. Checking out some of the lines I was sure they were all called in my tests. After some trial and error I found that after skipping some tests I get the (hopefully) correct coverage:

----------|----------|----------|----------|----------|----------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
----------|----------|----------|----------|----------|----------------|
----------|----------|----------|----------|----------|----------------|
All files |      100 |      100 |      100 |      100 |                |
----------|----------|----------|----------|----------|----------------|

The tests I skipped were https://github.com/Janpot/fantastiq/blob/master/test/cli.spec.js#L13 (describe.skip(...)

This part of the code runs some commands with child_process.exec.

If you want to try this out you'll need to have a redis running somewhere, if it's not at 127.0.0.1 you can specify one with the REDIS_HOST variable.

documentation updates

Based on work we've been doing on wrap-ansi, there are a couple documentation updates we should make:

  • point out that nyc report only works if you have already run tests inside the same project directory.
  • point out that COVERALLS_TOKEN is optional if the repo is open-source (verify this to be the case).

CC: @sindresorhus, @Qix-

Expose `check-coverage` functionality

I was able to get istanbul check-coverage working using it's glob functionality

istanbul check-coverage --branches=100 --lines=100 --functions=100 '.nyc_output/*.json'

It would be nice if there was a nyc check-coverage command that did this for me and was documented with the nyc tool.

coverage using exclude option

I am trying to use nyc with ava and running into a small issue for good measure here's a link to the conversation at ava repo.

So essentially i have a few directories in node_modules directory that i would like to cover since i do unit tests for them.

Unfortunately a setup like so:

"config": {
    "nyc": {
      "exclude": [
        "node_modules/"
      ]
    }
  }

results in no coverage at all.

setting up more specific like node_modules/utils which are some of the files i need to cover results in a coverage containing too many files/packages such as babel, etc which i am not testing.

any suggestions on how to allow certain files from within node_modules directory?

thanks in advanced

empty test coverage

see https://github.com/jamestalmage/promisify-firebase

changing nothing but index.js and test.js to a minimal test case (i.e., not changing any of the build config / package.json / etc), I can get working coverage. This means that the code of my module itself is somehow disabling the coverage.

I've poked around trying to figure out exactly what line is causing it, so far no luck.

any help would be appreciated.

Allow specifying reporters with `nyc` directly

To get a report I currently run

nyc node test/index.js && nyc report --reporter text --reporter html

It would be great to just run

nyc node test/inedx.js --reporter text --reporter html

Also note I'm relying on the "--reporter" can appear multiple times feature which may be accidental :)

--all option fails with ES6 modules

--all causes NYC to fail immediately for code with ES6 modules. I am using Node @5.1.0, nyc @5.0.0 with mocha and babel-register.

The error looks like:

/Users/joe/code/nyc-all-issue/node_modules/nyc/node_modules/istanbul/node_modules/esprima/esprima.js:5703
            throw e;
                  ^
Error: Line 1: Unexpected token
    at constructError (/Users/joe/code/nyc-all-issue/node_modules/nyc/node_modules/istanbul/node_modules/esprima/esprima.js:2405:21)
    at createError (/Users/joe/code/nyc-all-issue/node_modules/nyc/node_modules/istanbul/node_modules/esprima/esprima.js:2424:17)
...

.babelrc:

{
  presets: ["es2015"]
}

package.json:

{
  "scripts": {
    "test": "nyc --require babel-register mocha test.js",
    "test-all": "nyc --require babel-register --all mocha test.js",
    "run": "node test.js"
  },
  "dependencies": {
    "babel-core": "^6.3.17",
    "babel-preset-es2015": "^6.3.13",
    "babel-register": "^6.3.13",
    "mocha": "^2.3.4",
    "nyc": "^5.0.0"
  }
}

$ npm test succeeds and $ npm run test-all fails where test.js uses import, or requires a file that uses import.

https://github.com/jwhitfieldseed/nyc-issue-91 reproduces. It runs NYC with/out --all on code with/out import.

It seems other syntax that needs transpiling is OK with or without --all. I tested with rest args, which are not in Node 5.

with import

$ npm run test       // success
$ npm run test-all   // fails unexpectedly
$ npm run run        // SyntaxError, as expected - no rest args or import in Node 5

without import

$ npm run test       // success
$ npm run test-all   // success
$ npm run run        // SyntaxError, as expected - no rest args in Node 5

Babel Support?

Is there a way to have nyc use babel-istanbul or isparta under the hood to support covering ES5+ code?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.