karma-benchmark
A Karma plugin to run Benchmark.js over multiple browsers with Jenkins CI compatible output.
Installation
npm install karma-benchmark --save-devKarma Configuration
Reporting results on the command line
To see jsPerf style results on the command line, install karma-benchmark-reporter:
npm install karma-benchmark-reporter --save-devThen, in karma.conf.js, add benchmark to the list of reporters:
module.exports = function(config) {
config.set({
// Other Karma config here...
frameworks: [
'benchmark'
],
reporters: [
'benchmark'
],
});
};Run Karma:
karma startThen, you'll then see output that looks like:
Chrome 32.0.1700 (Mac OS X 10.9.1) Array iteration: util.each at 19356910 ops/sec
Chrome 32.0.1700 (Mac OS X 10.9.1) Array iteration: Array.forEach at 2567531 ops/sec
Chrome 32.0.1700 (Mac OS X 10.9.1) Array search: util.contains at 12635982 ops/sec
Chrome 32.0.1700 (Mac OS X 10.9.1) Array search: Array.indexOf at 5828437 ops/sec
Chrome 32.0.1700 (Mac OS X 10.9.1)
Array iteration: util.each at 19356910 ops/sec (7.54x faster than Array.forEach)
Array search: util.contains at 12635982 ops/sec (2.17x faster than Array.indexOf)
See the examples folder for a full example.
Feeding Data Into Jenkins
To feed your data into Jenkins, install karma-junit-reporter.
npm install karma-junit-reporter --save-devIn karma.conf.js, add junit to the list of reporters and configure the reporter accordingly:
module.exports = function(config) {
config.set({
// Other Karma config here...
frameworks: [
'benchmark'
],
junitReporter: {
outputDir: 'reports',
outputFile: 'benchmark.xml'
},
reporters: [
'junit'
]
});
};Timeouts
As large suites of Benchmarks take a long time to run, you may need to increase Karma's timeout from it's default of 60000.
captureTimeout: 60000Writing Benchmarks
Suites and benchmarks are defined using a wrapper for Benchmark.js in the form of the suite and benchmark globals.
Typical
In this example, a suite is defined that pits _.each against the native Array.forEach method:
suite('Array iteration', function() {
benchmark('_.each', function() {
_.each([1, 2, 3], function(el) {
return el;
});
});
benchmark('native forEach', function() {
[1, 2, 3].forEach(function(el) {
return el;
});
});
});Suite options
Suite options are the same as in Benchmark.js with one exception: setup and teardown can be set at the suite level.
See the Benchmark.js Suite constructor API docs for a full list of options.
suite('Array iteration', function() {
benchmark('_.each', function() {
_.each(this.list, function(number) {
return number;
});
});
benchmark('native forEach', function() {
this.list.forEach(function(number) {
return number;
});
});
}, {
onCycle: function(event) {
var suite = this;
var benchmark = event.target;
console.log('Cycle completed for ' + suite.name + ': ' + benchmark.name);
},
setup: function() {
this.list = [5, 4, 3];
},
teardown: function() {
this.list = null;
}
});Benchmark options
Benchmark options are the same as in Benchmark.js. If setup and teardown are passed to benchmark(), they will override setup and teardown from the suite. Pass null or undefined to remove them.
See the Benchmark.js Benchmark constructor API docs for a full list of options.
suite('Iteration', function() {
benchmark('_.each with array', function() {
_.each(this.list, function(number) {
return number;
});
}, {
setup: function() {
this.list = ['a', 'b', 'c'];
},
teardown: function() {
delete this.list
}
});
benchmark('_.each with object', function() {
_.each(this.list, function(number) {
return number;
});
}, {
setup: function() {
this.list = {
0: 'a',
1: 'b',
2: 'c'
};
},
teardown: function() {
delete this.list
}
});
});Running only a specific benchmark or suite
To run only a specific benchmark, use benchmark.only() or bbenchmark() instead of benchmark():
benchmark.only(function() {
// Only this benchmark will run
// bbenchmark() does the same thing
});
benchmark(function() {
// This benchmark won't run
});The same applies to suites with suite.only() and ssuite().
Skipping benchmarks & suites
To skip a benchmark, use benchmark.skip() or xbenchmark() instead of benchmark():
benchmark.skip(function() {
// This benchmark won't run
// xbenchmark() does the same thing
});
benchmark(function() {
// This and all other benchmarks will run
});The same applies to suites with suite.skip() and xsuite().
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent