Mozilla have a large base of tests cases for their SunSpider JS engine, see:
http://hg.mozilla.org/mozilla-central/file/d812f80a0f1d/js/src/tests

We would like to port these tests to work with our own test framework (test-runner.js, which is executed with the make test command). This will require cherry-picking tests which are currently applicable to Higgs as some of these will use features which we do not yet support. It will also require changing parts of the tests to use our own test library (lib/test.js) instead of their own idiosyncratic assertion functions.

Each ported test should ideally be a standalone JS file which can be run by Higgs directly from the console (e.g.: ./higgs tests/mozilla/mytest.js). The ported tests should go under source/tests/mozilla. New issues should be opened for the tests that currently fail under Higgs.

Note that it's important to include the license files associated with the tests that get ported.

Currently the make install target copies the Higgs binary to /usr/bin and the libraries to /etc/higgs.

These locations are configurable in the makefile, but the lib directory is hard coded - getLoadPath uses /etc/higgs as lib_path by default for loading libraries.

This should be an option configurable at compile time.

Higgs aims to ship with "batteries included". We'd like to find useful JS libraries out there, test them on Higgs, fix any bugs preventing them from running, and ship known working versions of these libraries along with Higgs. The goal here is to allow Higgs users easy access to these libraries with a single require('libname') command, no installation required.

The libraries we repackage obviously have to be open source (preferably BSD licensed). We would also like to write/include unit tests to make sure that the libraries work properly on the latest version of Higgs and detect any regressions.

Test case:

var x = 5;
x.hasOwnProperty("toString");

Causes a runtime error instead of evaluating to the expected false.

A library providing a very easy API to quickly create a window and begin plotting pixels. The API should be minimalist.

Example usage:

draw = require('lib/draw');
window = draw.window(512, 512, 'window title');
window.setPixel(20, 100, {r: 255, g:0, b:0, a:255});
window.drawLine(25, 22, 200, 350, draw.red);

Bonus if you can implement advanced methods such as fillRect() to fill quads with hardware acceleration, but please don't get too fancy. The library needs to be kept minimalist and highly portable. It needs to behave exactly the same on every platform, which isn't going to happen if we begin to rely on platform-specific features. Other Higgs graphics libraries can build on top of this very basic library to provide more advanced features such as text rendering, curves, GUI, etc.

This should probably be built using SDL or X11 bindings. Internal details of the underlying API should not be exposed to the user. It doesn't matter if the high-level API we expose doesn't have great performance, so long as it's fast enough for most tasks.

There are surely plenty of undiscovered bugs left in Higgs. Some source code might fail to parse, some programs might run but give an invalid result, or cause a segmentation fault. We want Higgs to be as bug-free as we can make it, but in order to fix these bugs, we need to find them. We encourage you to get Higgs and try running JavaScript code of your own. If you find bugs in Higgs, you can help us a lot by opening new issues and sending us pull requests with failing test cases exposing the bugs you found.

There seems to be an issue with the crypto-aes benchmark not producing the same output on Higgs as on V8. In particular, decryptedText.length does not match after the benchmark is done running. I believe this may be due to a bug in our regexp library implementation not handling some unicode characters properly.

The test-runner is failing when looking up test directories. This appears to be another OS-related problem; this time with readdir/dirent.

$ make test
...
gc/stackvm
gc/load
FFI
Starting test-runner.js...
 ---
run-time error: run-time error: Unable to open directory: ./tests/..
$rt_throwExc(2FA842) ("runtime/runtime.js"@128:1)
dir_prototype_open(32C434) ("lib/dir.js"@95:32)
dir(32C36E) ("lib/dir.js"@87:23)
anon(32C4F9) ("lib/dir.js"@173:39)
runTests(30A5E7) ("test-runner.js"@117:19)
anon(30A6F5) ("test-runner.js"@152:13)
Function_prototype_apply(2FF388) ("stdlib/function.js"@226:18)
Function_prototype_call(2FF3A1) ("stdlib/function.js"@245:18)
array_forEach(3011BD) ("stdlib/array.js"@578:9)
anon(30A528) ("test-runner.js"@149:5)
test_runner_js(30A522) ("test-runner.js"@43:2)
       52.90 real        51.93 user         0.61 sys
make: *** [test] Error 255

On Darwin, the struct's used to store the directory entries differ from their Linux counterparts:

struct dirent { /* when _DARWIN_FEATURE_64_BIT_INODE is NOT defined */
    ino_t      d_ino;                /* file number of entry */
    __uint16_t d_reclen;             /* length of this record */
    __uint8_t  d_type;               /* file type, see below */
    __uint8_t  d_namlen;             /* length of string in d_name */
    char    d_name[255 + 1];   /* name must be no longer than this */
};

struct dirent { /* when _DARWIN_FEATURE_64_BIT_INODE is defined */
    ino_t      d_fileno;     /* file number of entry */
    __uint16_t d_seekoff;    /* seek offset (optional, used by servers) */
    __uint16_t d_reclen;     /* length of this record */
    __uint16_t d_namlen;     /* length of string in d_name */
    __uint8_t  d_type;       /* file type, see below */
    char    d_name[1024];    /* name must be no longer than this */
};

Add a 'Manual' page to the Wiki.
This would be a simple guide for Higgs users, yet more in depth than the quickstart.

In the future, drafts of examples / various how-to stuff could grow into a kind of official Higgs programming manual/tutorial.

Reimplementing the FFI function call instruction in the new incremental JIT (currently in dev-maxime branch). Code should be somewhat similar to call_prim and eval_str instructions, which are both different kinds of call instructions. Generated assembly instructions should be near-optimal if possible (minimize moves) to maximize the throughput of the FFI.

Ideally we should cache the dmd package between builds to speed them up a bit. This can be accomplished by adding an appropriate cache: directive in the travis config and only wget'ing the package if the file does not already exist in the before_install: setup.

Higgs currently lacks support for Function.prototype.bind().

From MDN:

The bind() method creates a new function that, when called,
has its this keyword set to the provided value, with a given
sequence of arguments preceding any provided when the
new function is called.

It should be pretty easy to polyfill this and add it to /source/stdlib/function.js.

There seems to be issues when dynamically loading the bindings for higgs on OS X. This can be seen within the FFI test suite:

var higgs = $ir_load_lib("");

... which eventually results in:

runtime.vm.RunError: run-time error: dlopen(, 1): no suitable image found.

Full make test error output: https://gist.github.com/paulfryzel/11023330

(OS X 10.9.2, DMD64 D Compiler v2.065, Higgs@93c675f)

This error stems from differences in libc between OS's (BSD -vs- Linux). On OS X/BSD, dlopen requires null as its first parameter (path) to default to the main program.

auto lib = dlopen(null, RTLD_LAZY | RTLD_LOCAL); // jit/ops.d L#3967

There seems to be some error, possibly in the regex engine, which causes string-tagcloud to throw an exception. Seems to be the benchmark's own implementation of a JSON parser throwing an error, probably due to some non-spec-conformant behavior in Higgs. I suspect that our regexp implementation may be at fault.

It may be difficult to debug, the best strategy may be to find where the behavior of Higgs differs from that of V8. Another possibility would be to intercept the regexp match calls and log the results, see if the Higgs regexp implementation behaves properly.

Haven't really looked into this, but basically the following code:

function blah()
{

    while(true)
    {
        if (false)
            continue;
        else
            break;

        while(true)
        {
                // inner loop body
        }

        break;
    }

}

blah();

Will cause an assert failure during the peephole pass (line 116 in ir/peephol.d).

This issue doesn't appear if the inner loop body contains just break, however if the inner loop body is:

    if (true)
        break;
    else
        continue;

(or if it is empty as in the first code block) It does appear.

Higgs currently does not optimize tail calls. Properly optimizing them would require the implementation of a new call_tail IR instruction. The JIT needs to be modified to generate machine code for this instruction as well.

Example case:

var console = require("lib/console");

function bleh()
{
    this.message = "foo";
}

bleh.prototype.message = "";

console.log(Object.keys(new bleh()));

Expected result is ["message"], but instead this results in ["message", "message"].

The underlying cause is a duplication in for-in - if we replace the last line with:

for (var k in new bleh())
{
    console.log(k);
}

"message" will print twice instead of once.

The Octane 2 benchmarks need to be tested on Higgs. For the benchmarks that do not yet work properly, the cause of the issue needs to be identifier (is it a bug, is Higgs missing a required feature?). In the case of bugs, regression tests should be written and new issues should be filed so that these bugs can be fixed.

I've added the Octane suite to the Higgs repository (under source/benchmarks/octane). I'd be particularly interested in getting the zlib benchmark to work. The benchmark can be run with the following command:

./higgs --e "runZlib()" benchmarks/octane/base.js benchmarks/octane/zlib-data.js benchmarks/octane/zlib.js

We would like to fuzz test Higgs so as to find bugs in its implementation, and create new regression tests in consequence. This may require writing a custom fuzz tester that generates random JavaScript programs for which the output is known, runs the said programs on Higgs, and compares the output produced to the expected output.

Potential links of interest:
http://embed.cs.utah.edu/creduce/pldi12_talk.pdf
http://embed.cs.utah.edu/creduce/

We'd like to make sure that Higgs is tested on as much code as possible. As such, we want to find more JavaScript programs that Higgs can run, and add these to our test suite (run with the make test command). This will help discover new bugs and ensure that there are no feature regressions in future releases. Higgs already has support for most of the V8 and SunSpider benchmarks, and some of the language shootout benchmarks. What would be most helpful is finding JavaScript programs in the wild that we can repackage and include in the Higgs tests. Note that these programs must be able to run outside of a web browser (i.e.: they cannot rely on the DOM). Special care also needs to be taken to make sure that the programs found have a license that allows us to repackage and ship them with Higgs.

Ideally we should have a variety of packages with the Higgs binary and libs available in various formats (deb, rpm, etc) or even targeting specific distros (ubuntu, debian, arch, etc) so that users do not need to download/compile Higgs themselves. The packages should place the Higgs binary and libs in a location compatible with the target distros filesystem layout conventions (if applicable). (See also issue #35)

I would like to set up a D documentation engine for the D source code in Higgs. This code already contains javadoc-style comments, which should be parsed by most documentation engines.

One possibility is ddox:
https://github.com/rejectedsoftware/ddox

The Kraken benchmarks need to be tested on Higgs. For the benchmarks that do not yet work properly, the cause of the issue needs to be identifier (is it a bug, is Higgs missing a required feature?). In the case of bugs, regression tests should be written and new issues should be filed so that these bugs can be fixed.

Please note that the Kraken benchmarks are already included in the Higgs repository.

To allow higgs to be used for scripting type tasks, faster dev cycle, etc JS code should have access to the arguments passed to the process. One convention is to allow "--" to be used to separate args intended for Higgs and args that should be passed through.
e.g.

higgs script1.js script2.js -- 1 2 3

This is what d8 does. Another option is to simply limit the user to running one script with the higgs binary, and consider any other non option arguments as arguments for the script, which is what node does. Or we could do something like add another opt called "args" or "argstr" etc which gets exposed to the script.

There's also question of how to actually expose the args to JS. Node has this functionality in the process object, commonjs puts it in a module called system. We could also go another way.

The Function constructor is currently unimplemented in Higgs.

From MDN:

The Function constructor creates a new Function object. 
In JavaScript every function is actually a Function object.

The Function constructor can be found in /source/stdlib/function.js marked with a TODO comment.

The implementation should internally use "eval" to create a function with the source code passed as argument to the Function constructor.

Minimalist graph and bar chart plotting library based on lib/draw (X11 drawing library, already implemented). This could be loosely inspired from Python's matplotlib. Should allow updating/redrawing the graphs dynamically as new data is added. This could be used, for example, to chart mathematical functions, or stock prices over time.

Example usage:

plot = require('lib/plot')

// Draws a graph linking points by line segments
// values is an array of y values
// yMin and yMax are the range of the y axis
// xMin, xMax and xDelta determine x values corresponding to point indices
plot.graph(canvas, values, yMin, yMax, xMin, xMax, xDelta)

// Draws a bar graph with labeled bars
plot.barChart(/* TODO: determine argument format */)

I would strongly advise not trying to cover every possible use case for the first version of this library, as this may increase the difficulty and complexity significantly. Comments/feedback welcome.

Higgs currently does not support the with statement. The with statement changes the semantics of variable accesses, which now need to check if the variables are properties of the object passed as argument. Support for with in conjunction with "eval" is not needed at this point.

The parser needs to be modified to parse this kind of statement, and the AST-to-IR translation pass needs to generate IR for this new kind of statement.

getLoadPath uses /etc/higgs as lib_path by default for loading libraries.

So one problem ocurs when higgs is compiled as:

make LIB_DIR=/usr/lib/higgs/lib 
# In fact, libraries are installed in /usr/lib/higgs/lib
# but higgs doesn't find them.

I think #36 is related.

Parse errors cannot be caught:

var console = require('lib/console');
try
{
    eval(",");
}
catch (e)
{
    console.log("Err:", e);
}

Should catch and log the error, but it passes through.

This encapsulates the numerous outstanding issues preventing stable tests on OS X (Darwin). When this issue is fixed, all current tests should pass on both Linux and OS X.

• ☑ #99
• ☑ #101
• ☑ #102
• ☐ https://gist.github.com/paulfryzel/a3cfa21486823cd5ebb2

Fixes on branch paulfryzel/Higgs#osx-tests.

Higgs doesn't seem to correctly handle -0 and/or -Infinity in some cases. Ex:

    1 / -0

Should -Infinity but evaluates to Infinity.

Currently there are symbol lookup errors on OS X when running tests via make test.

run-time error: run-time error: dlsym(RTLD_DEFAULT, stdout): symbol not found

We should remove the OS-specific declarations from the stdio bindings (as the libc headers differ, e.g. FILE *stdout (linux) -vs- FILE *__stdoutp (darwin)) and instead build them from their POSIX-compliant file descriptor identifiers.

Multiple pages of the Wiki are incomplete. Completing existing documentation, adding links to relevant information and writing up explanations of specific parts the Higgs source code would be very much appreciated. A detailed Wiki full of useful information should help attract and retain potential contributors to this project.

The sunspider/date-format-tofte benchmark seems to be using very high amounts of memory, possibly due to string allocation. I stopped it after it had expanded the Higgs heap to 4GB on my system, which seems completely abnormal (these benchmarks are meant to run in a browser, on a computer from 1999).

Probably this is because this linux is 32bit instead of 64bit. Will experiment later.

!  ~/Higgs   master  source  make test Fri Jan 10 11:53:15 CST 2014
python interp/layout.py
python jit/encodings.py
dmd -m64 -O -L-ldl -unittest -debug -oftest-higgs util/error.d util/id.d util/string.d util/misc.d util/bitset.d analysis/typeset.d parser/lexer.d parser/ast.d parser/vars.d parser/parser.d ir/ir.d ir/ops.d ir/iir.d ir/ast.d ir/peephole.d ir/slotalloc.d ir/livevars.d ir/inlining.d interp/interp.d interp/ops.d interp/layout.d interp/string.d interp/object.d interp/gc.d interp/ffi.d jit/x86.d jit/encodings.d jit/codeblock.d jit/assembler.d jit/moves.d jit/regalloc.d jit/inlining.d jit/ops.d jit/peephole.d jit/jit.d repl.d options.d stats.d main.d jit/tests.d parser/tests.d interp/tests.d
util/error.d(68): Deprecation: function core.exception.setAssertHandler is deprecated
jit/codeblock.d(78): Deprecation: alias core.sys.posix.sys.mman.MAP_ANON is deprecated - Please use core.sys.linux.sys.mman for non-POSIX extensions
time ./test-higgs
./test-higgs: 1: ./test-higgs: �ELF����: not found
./test-higgs: 1: ./test-higgs: Syntax error: Unterminated quoted string
Command exited with non-zero status 2
0.00user 0.00system 0:00.19elapsed 0%CPU (0avgtext+0avgdata 528maxresident)k
48inputs+0outputs (1major+175minor)pagefaults 0swaps
make: *** [test] Error 2
!  ~/Higgs   master …  source 

This is the last/only V8 benchmark not working. This benchmark came from hell, and is an endless pit of bugs. Please help us fix all the bugs so that we may vanquish the demon once and for all.

Currently String.prototype.split() does not meet the ECMAScript spec.

• non-string separators are not handled correctly
• empty string separators are not handled correctly
• empty string as this not handled correctly

The second one was the cause for the memory consumption in issue #112 due to an infinite loop.

The global parseFloat function is not yet implemented. This should be added to stdlib/global.js, where parseInt already resides. This function is needed by the esprima JS parser test.

Consider a C Library with definitions like:

struct Baz {
    int num_foo;
    struct Foo *foos; /* array of length num_foo */
};

extern struct Baz *getBaz();
extern int openFoo( struct Foo* );

If one calls x = getBaz() there is no way to pass a pointer to x->foos[2] to openFoo except to do something like malloc a new chunk of memory, copy the data, and then pass a pointer to that (which is problematic). Ops for arithmetic which operate on rawptrs would make this easier.

Higgs doesn't currently allow the valid usage of ReservedWord as an IdentifierName.

The ES5 Grammar has both Identifier and IdentifierName. The difference is Identifier cannot be a ReservedWord but IdentifierName can.

Practical upshot:

var a = {};
a.new = "hellew";
// or
var a = { new : "hellew" };

Should be valid constructs as the "new" is an IdentifierName.

function new () {};
// or
var new = function(){};

Would not however, as the "new" is an Identifier.

Running the sunspider benchmark string-unpack-code.js currently segfaults with the message:

Assertion failed (runtime/gc.d@435): gcForward: object not in from-space heap
ptr   : 1
start : 7F39B6DB7000
limit : 7F39B7DB7000
header: ???
[1]    5830 segmentation fault  ./higgs benchmarks/sunspider/string-unpack-code.js

Writing a UDP/TCP socket library using the Higgs FFI. This would make it possible to write a simple HTTP client and/or server that runs inside of Higgs, as well as other networked applications. The library should internally rely on the BSD sockets API, which will be accessed through the FFI library.

I am trying to learn from this project. But make all returns with an error which I cannot solve by myself. At first, I thought it was because I was using a 32-bit linux, but the same thing happens in a 64-bit machine.
Is this the right place to go and ask for help (especially since my issue might not be due to Higgs itself but to my misunderstanding)?

Thanks! Sylvain

Error for reference:

ir/inlining.d(290): Error: cannot implicitly convert expression (arg) of type ir.ir.IRValue to inout(IRValue)
ir/inlining.d(307): Error: cannot implicitly convert expression (arg.value) of type ir.ir.IRValue to inout(IRValue)
ir/typeprop.d(196): Error: mutable method ir.typeprop.TypeVal.merge is not callable using a const object
ir/typeprop.d(216): Error: mutable method ir.typeprop.TypeVal.merge is not callable using a const object
jit/jit.d(1427): Error: template object.get cannot deduce function from argument types !()(BlockVersion[][IRBlock], IRBlock, void[]), candidates are:
/usr/local/include/dmd2/object.di(480):        object.get(K, V)(inout(V[K]) aa, K key, lazy inout(V) defaultValue)
/usr/local/include/dmd2/object.di(486):        object.get(K, V)(inout(V[K])* aa, K key, lazy inout(V) defaultValue)
make: *** [all] Error 1

The ES6 Draft Specification has a feature called template strings, or quasis - they are a syntactic sugar intended to make it easier to embed DSLs and templating into JS code.

They have a lot of fun features, but the upshot for me is we can get a syntax for multi-line string literals that is basically forward-compatible with ES6. When/if we move to full template string support the main issue with existing code that uses backticks purely as a multi-line string literal would be ${foo} constructs in the string would become significant as string interpolation. I don't think that's too likely to cause many problems though. To mitigate those issues I suggest we limit any discussion of them in the docs to usage in the FFI, and promote their usage only in that context (though not actually enforce this in Higgs or anything). Then when/if we add template strings we can update the docs and any existing code using them for FFI cdefs will still work.

So I suggest we add support for using backticks for string literals, treating them the same as other string literals currently are - excepting newline and other whitespace characters become part of the string value as in ES6 template strings.

The main reason I'd like this is for users of the FFI lib. It would make it easier to write/read/edit the embedded C definitions. Instead of something like

ffi.c.cdef("\
         int foo(int, int);\
         long bar(long, long);\
         typedef struct baz {\
             int x;\
             int y;\
         } baz;\
");

Or:

ffi.c.cdef("int foo(int, int);" + 
        "long bar(long, long);" +
        "typedef struct baz {" +
        "    int x;" +
        "    int y;" +
        "} baz;");

...etc, we could have the much nicer:

ffi.c.cdef(`
    int foo(int, int);
    long bar(long, long);
    typedef struct baz {
         int x;
         int y;
     } baz;
`);

The following code:

var x = Object.create(null);
x.foo = "bar";
Object.keys(x);

throws an error because Object.keys uses hasOwnProperty internally. It should use $rt_hasOwnProp or specially handle the case where the prototype is null.

If you run examples/pong.js (can be found in the PR in issue #92) after a short time running it will segfault with the message:

Assertion failed (runtime/gc.d@432): gcForward: object not in from-space heap ptr : 7F460FCE5FD8 start : 7F4619086000 limit : 7F461A086000 header: 2 [1] 5809 segmentation fault ./higgs ../examples/pong.js
I only just now hit this; I'll start debugging and update accordingly.

The sunspider benchmark string-validate-input.js currently segfaults when run.

The issue is on line 16:

(k%2)?email=name+"@mac.com":email=name+"(at)mac.com";

If you have such a construct in the top level like so:

var k = 0;
(k%2)?email=name+"@mac.com":email=name+"(at)mac.com";

...you will (incorrectly) get a parse error:
parse error: "../bin/snspdrfail.js"@2:2: invalid lhs in assignment

If it's placed inside a function:

var k = 0;
function foo()
{
    (k%2)?email=name+"@mac.com":email=name+"(at)mac.com";
}
foo();

...Higgs will segfault at the function call.

It would be quire useful to have a shorthand function in stdio that will read a whole text file into a string. The equivalent of:

var file = stdio.fopen(fileName, 'r');
var data = srcFile.read();
file.close();

A test should also be added to make sure that this function works properly.

Higgs allows users to pass arguments to scripts by including them after "--", like so:

./higgs script.js -- foo bar

However, this just provides an array of strings (in this case ["foo", "bar"]). Higgs should include a library that makes it easy for scripts to declare what arguments and options they take, which ones are required, etc. The library should then take an argument array like:

["plain arg", "second plain arg", "--opt", "value", "--secondopt", "value", "value"]

and produce an array of the arguments like:

["plain arg", "second plain arg"]

as well as an opts object like

{
    "opt" : "value",
    "secondopt" : ["value", "value"]
}

If all required args/options are not present an error should be thrown and/or a message displayed and the program exited.

A possible enhancement would be to take an optional description of what each option does, and use this to construct a "usage" message to display when no/incorrect args/opts are passed or the "-help" option is encountered.

Another possible enhancement would be to take an optional type for each arg/opt and then convert the argument/opt value to that type before passing it along to the library user, or an optional "converter function" to be applied to the opt/arg before passing it on to the library user.

Feedback for possible extra features, ideas for what the API should look like, and any other general thoughts are very welcome (^_^).