If you run the ClojureScript compiler's unit tests in a loop and allow things to warm up, they end up running more slowly in Graal.JS than under Nashorn.
The numbers above were produced on a Mac, using GraalVM 1.0.0-rc4 (ee edition). Nashorn is 1.8.0_172-b11.
To reproduce using the actual ClojureScript compiler unit tests, the steps involve:
- Modify the ClojureScript compiler's tests so that they run in an infinite loop (optionally also capturing timing metrics)
- Build the tests (which involves producing a JavaScript file that can be executed in any engine)
- Run the tests, allowing some time (say 1/2 hour or so) for them to settle
To make things more convenient, I've attached a copy of the built tests, and attached them directly in the "Run the tests" section.
Modify the ClojureScript compiler's tests
The ClojureScript compiler is at https://github.com/clojure/clojurescript
Modifications are at mfikes/clojurescript@1c76cf6
and to make things easier, that branch could just be checked out and used as-is. Roughly, these modifications involve setting up some atoms (mutable state) to track timing information, and putting the tests themselves in an infinite loop ((while true ...)
in ClojureScript), doing some accounting for timing, and printing out the timing with the string DELTA
which can be grep
ped for.
Build the tests
To build the tests, run script/bootstrap
and then script/test
in the tree. This will cause it to build them and then after logging
Applying optimizations :advanced to 343 sources
it will automatically attempt to run them under various JavaScript engines if any are configured per https://clojurescript.org/community/running-tests. (No JavaScript engines need to be configured for this repro). If it starts running the tests in any engine, Ctrl-C them (because they are in an infinite loop).
This will result in a JavaScript file, builds/out-adv/core-advanced-test.js
that can be run in any engine.
Run the tests
For convenience, here is a built copy of the test JavaScript file:
core-advanced-test.js.gz
Then to run the tests, say, using Graal.JS, do
/path/to/js builds/out-adv/core-advanced-test.js | grep DELTA
You will see output like this
DELTA 3204 #queue [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3204] 100
DELTA 2083 #queue [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3204 2083] 165
DELTA 2016 #queue [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3204 2083 2016] 228
DELTA 1932 #queue [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3204 2083 2016 1932] 288
The first number is the number of milliseconds to run that test loop, the queue is the last 32 timings, and the very last number is the average of the 32 timings. (So, definitely let it fill the queue with timings, and go well beyond that to let things settle completely.)
(If you run them in Node, the tests want print
to be defined, so to do that, start node then var print = console.log;
and then require("./path/to/core-advanced-test.js");