mitranim / jisp Goto Github PK

Many jisp expressions translate into multi-line chunks of JavaScript that aren't valid expressions. Jisp works around this by assigning results of multi-line expressions to reference variables and putting those variables into JS expressions. If the expression is a conditional, this can cause the hoisted multi-line code to be executed out of order, or be executed when it’s not supposed to.

Example.

Jisp source:

(= x 0)
(and false (for i `(1 2 3) (+= x i)))

Compiled JavaScript:

var x, i, _i, _res, _ref;
x = 0;
_res = [];
_ref = [1, 2, 3];
for (_i = 0; _i < _ref.length; ++_i) {
  i = _ref[_i];
  _res.push((x += i));
}
false && _res;

The loop is not supposed to be executed, yet it does, with unintended side effects. Similarly, things can be executed out of order.

One possible solution is to wrap hoisted blocks into self-executing lambdas with references as return values instead of hoisting them. Functions in JavaScript are expressions that are allowed to be multiline, so this sounds like an easy solution. Another possible solution is to wrap them into non-self-executing lambdas assigned to references, and embed references as function calls rather than values. This should produce more readable code than blunt function embedding, at the cost of more variable bindings in the code.

For prettier output, we should also check if a multi-line block consists purely of native JS expressions. In this case, we should compile it to a grouped (in parentheses) list separated with commas.

Right now, in a code returned from a macro, regex property notations like /regex/.text render as [native code]. ToDo fix. The current workaround is to use the (get /regex/ property) notation or chaining with (do /reg/ (.stuff)), the same can be used as an internal fix. (Alternatively, simply stop translating them into regexes when parsing, leave as strings.)

It will make debugging in the browser simpler. Here is the description of what source maps are.

Because the REPL is primary a tool for learning and testing the language, and because of all the differences between jisp and JavaScript, the REPL by default prints compiled code back before executing it. There needs to be an option to launch a non-verbose REPL that doesn’t print code back.

When compiling an (if), if there's no else branch and it would compile to the ternary form, instead compile it to the binary form:

if (<test>) <then>;

The #n and # notation needs to support the following:

• .dot and [bracket] property notation (the current workaround is the (get #n prop) or (do #n (.stuff)) notation) (fixed in 0.2.18)
• ... spreading

(Tentative.) We may want to distinguish between ;, ;;, and ;;; comments. ;; would quote the next form in parentheses, ignoring end-of-line. ;;; would be a multiline block comment.

Right now, regexes break if they contain literal whitespace (the tokeniser breaks them apart). ToDo fix.

Jisp uses a few methods that may not be supported in older JavaScript environments like IE8. Examples include Array.prototype.indexOf, Array.prototype.reduce, and probably a few others. There needs to be a compiler capability, with a compiler option (default off? definitely off in the REPL), to include polyfill for this stuff into the compiled source.

Jisp needs syntax highlighting for HTML and language modules for common code editors: Sublime Text, Atom, Emacs, and others.

It might be nice if parentheses were dimmed. Should make them more palatable for parenthesophobes without compromising the code structure, and generally improve readability.

In the meantime, you can use syntax highlighting for one of the other Lisp dialects.

The tokeniser sometimes fails to split off [ ] brackets from identifiers. Example:

form[,i]

In this example, it produces form[ as a separate token. ToDo reimplement bracket splitting in a way that makes this impossible.

In JavaScript, you can clump together arithmetic and logical operators that take a single argument:

+!"stuff"
!-~~Math.random()

Jisp needs to natively support clumping for all “operators” that can take a single argument (but not necessarily just one).

JavaScript allows Unicode letters and even many Unicode non-letter characters in identifiers. Right now, the syntax regexes only allow the $_A-Za-z0-9 set in names. ToDo implement support for all allowed symbols.

Operator-functions are things like + or <:

(+ 2 3)

We need to be able to use them in all the same ways as normal functions, including this following.

Pass by name:

(array.filter <)

Property notation:

(+.apply `(1 2 3))

Spread:

(* ...`(3 4) `(5 6))

Supporting this requires the following components:

• Impostor functions (trivial, partly implemented, see operators.jisp) to be embedded with .toString
• Hoisting and embedding mechanism: already implemented, will need a few tweaks for this
• Detect when the operator is referenced a non-standard way (one of the ways listed above), trigger replacement and embedding
• Replacement: replace the operator glyph with the name of the impostor function
• Treat impostor function names as service variables and guarantee that they don’t clash with user-defined names in any scope

So I can separate out my code over multiple files?

Smaller issue: right now, the multiline input is broken in the REPL. Worked until recently, probably a small bug.

Bigger issue: it counts parentheses inside strings and regexes, including escaped parens. ToDo fix.

I found jisp can't compile on window.(win7)
Is there any one same with me?
When I command jisp -c <file>,then it hold on, nothing can done.

Jisp needs a comprehensive compiler test suite. It turned out that recompiling itself is far from being a good enough test.

Conditionals like (if), (switch), (try) return their resolved value by assigning it to a reference variable in each branch of execution, and putting this variable after the form. When such a form comes last in a function, we should returnify each branch (prepend a return to the last expression) instead of referencing it and returning the reference.

As many built-ins as possible should support multiple arguments in a meaningful way. This especially includes operator-functions like comparisons or assignments: < +=.

This is related to issue #20: execution order control. Right now, <test> in (elif <test> <body>) can only be a single JS expression, the compiler throws an error if it compiles to multiple lines. We should check if those multiple lines can be included as comma-separated expressions, and when not, include them as self-executing lambdas or reference calls.

The REPL should save history and load it back up when restarted.

Right now, the tokeniser doesn’t properly account for escaped quotes in strings. In particular, it can’t handle more than one backslash immediately preceding a quote. In other cases, the output may not always match the input. ToDo fix.

I wanted to create a macros:

(mac http_call method url ...body
     `(do $http
         (,method ,url)                                                                                                                               (.success (fn data (do ,...body)))))

To be used like:

(http_call .get "/api/items/"
                      (= $scope.items data))

which should produce code like this:

(do $http
    (.get "/api/items/")
    (.success (fn data
                  (do (= $scope.items data)))))

But compiler says:

Error: unexpected .get in possible modes: emptyhash | hash | list | quote | unquote | spread | spread | atom

Tested against: (  :  )  (  isKey  :  (  `  ,  ...  …  isAtomString

Tokens: .get, "/api/permissions/", (, =, $scope.perms, data, ), ), (, http_get ...
    at Error (unknown source)
    at demand (/home/art/my-project/jisp/dev/jisp/lex.js:104:108)
    at lex (/home/art/my-project/jisp/dev/jisp/lex.js:170:22)
    at demand (/home/art/my-project/jisp/dev/jisp/lex.js:101:16)
    at lex (/home/art/my-project/jisp/dev/jisp/lex.js:170:22)
    at demand (/home/art/my-project/jisp/dev/jisp/lex.js:101:16)
    at lex (/home/art/my-project/jisp/dev/jisp/lex.js:161:21)
    at Object.compile (/home/art/my-project/jisp/dev/jisp/jisp.js:1571:20)
    at compileScript (/home/art/my-project/jisp/dev/jisp/command.js:170:25)
    at compilePath (/home/art/my-project/jisp/dev/jisp/command.js:127:15)

If i remove dot from get, then compilation succeed, but result is incorrect:

$http;
    ["/api/items/"]
      .success((function(data) {
        return ($scope.items = data);
      }));

Tiny things.

• Compiler error messages for when special forms like = and quote, as well as JS keywords, are met outside their destined place (first element in list).
• Lexer or parser error messages for common but hard to spot errors like infix +.

Either the command module or the repl module needs to automatically load register to register the file extension.

ToDo implement the # notation. It needs to be equivalent to arguments but be a true Array. In other words, like a rest parameter for all passed arguments:

{return #}

equivalent to:

(fn ...x x)

Right now, the hash macro : is compile-time only. It can be used to build hashes as lists inside macros, but not at runtime. ToDo implement a similar runtime feature. It should take a single list and build a hash out of it. Will probably need to be a different form (e.g. ::). If we want to tack it on :, the compiler needs to way to treat the same symbol in two different ways (a macro and a special form).

(Edit: will probably just add special clauses to the macro and implement gensym-like support for unique variables coming from macros.)

Right now, when you have a quoted list in a macro (say, the code you’re expecting to return), to make a quoted list inside of it, you have to quote it twice. This doesn’t seem to make sense. ToDo comprehend whether this is bad, and fix if yes.

Regex maintainability is easier when you can write them in a modular way, commenting the parts, without affecting the compiled code. See the CS version.

Right now, there’s a difference between macros and global functions in the way they are hoisted and embedded. Functions respect the file’s scope and allow the user to override them: if a function is reassigned in the top scope before being referenced by name, it’s not embedded. Macros don’t allow themselves to be overridden and are embedded even if “reassigned” (referenced like (= myMacro ‘stuff’)).

We need to allow the user to reassign and reuse macro names in a similar way as with global functions. If a macro’s name is reassigned before a macro call, the macro should be blacklisted and ignored for the duration of this particular call of the compile function or this REPL session.

It should look like this:

Jisp source:

(switch x
  (case val0 val1 val2 <body>))

Compiled JS:

switch (x) {
  case val0:
  case val1:
  case val2:
    <body>
    break;
}

I'm not sure if this is the best way to provide this feedback, but I'm very close to wanting to use this language for serious projects.

My hangup is a lack of good iterator forms. I don't want all my iterators to rely on arrays thus using extra memory, or make an awkward looking while loop(this requires additional let and do statements)

You describe a not yet implemented feature on the jisp.io homepage, where the for statement would accept an integer. That would be awesome.

In general I would like to see the iterable parameter to the for loop accept any function as an iterable(a closure, it would need to be stateful). So long as it returns a non-null value it continues iterating.

I honestly wouldn't mind a for loop syntax that was more of a direct mapping of javascript's for loop, even though that doesn't conform to functional paradigms.

There's some tricky language decisions involved and I respect the challenge of this problem.

I have been trying to figure out the best form I prefer for writing array comprehensions as well, but they use memory so I don't like them for some cases.

I don't know if you have that NYI integer for loop committed already to a branch, I would like to see that be a top priority because it is an important use case for me.

That's my feedback. Thanks.

There needs to be a way to return jisp code after parsing and expanding all macros, but before compiling. This must be available as a compiler option, and in the REPL.

This requires implementing jisp printing and beautification. Blindly printing the code back, as a single line, should be trivial. Indenting should also be simple: in the simplest form, it’s 2 extra spaces per each level of nesting. Not sure how to decide where to put linebreaks. ToDo figure it out and implement.

Right now, .dot and [bracket] notation has the following issues:

• Can’t be used with string literals (fixed in 0.2.17)
• Can’t be used with #n and (fixed in 0.2.18) # notation (NYI)
• Can’t be used with lists that resolve to JS expressions (e.g. ((mystring.replace /-/ '_').trim))
• Orphan property notation needs to be supported everywhere you can use the (get property) form (for instance, as arguments in macros) (fixed in 0.2.17)

In all of those cases, the current workaround is to use the (get object property) notation or chaining with (do object (.stuff)). ToDo fix.

With nested macros, it’s possible to have an array of things you want to give to an inner macro as arguments. ToDo implement (spread) support for macro calls.