modeswitch / mozbench Goto Github PK

I removed all the native deps (zeromq, zeroconf) so the framework now runs in node.js only. Should be portable to Android assuming we can compile and run node.js there.

Currently the device type is hard-coded. When a worker joins, it needs to have a device type assigned to it so that work can be dispatched.

Linux packages for Firefox are .tar.gz. For Chrome they may be different. The code needs to recognize and handle different package types and unpack them correctly for the tests to run.

current commandline:
ode mozbench-client.js queue add linux firefox nightly octane-2.0 -r 2 -d x220-linux -i
https://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/firefox-30.0a1.en-US.linux-x86_64.tar.bz2 -l
http://10.242.24.62/~ack/work/mozbench/www/benchmarks/octane-2.0/

-r replicate
-d deviceType
-i installer path
-l url to benchmark

The manager keeps track of some state (device types, jobs, etc) that should be written to disk periodically and loaded when the manager starts.

Actual installers for other platforms (Windows, Darwin) and package types (.zip, .exe, etc).

This should cover things like unzipping packages, etc.

Results for finished jobs should be saved to disk or exported to elastic search.

Allow jobs to run until target confidence interval is reached rather than after a set number of replicates. For some benchmarks, we can use a smaller number of replicates this way.

If the worker doesn't complete its task in before the timeout, cancel the task and reschedule it.