danielxmoore / civet Goto Github PK

Parameter Properties

.= syntax is incorrectly highlighted in VSC:

Expected to see exact highlight as ::= has

• interface X extends Y does not parse
• interface X < Y does not parse

Currently, if you want a quick function that uses the input twice, e.g. ($) => $+$, you can write &+$. I find this syntactically inappropriate, as $ doesn't appear until you use it the second time; I don't find it at all intuitive that & binds $. And it's especially bad if you use $ yourself or nest & functions inside of each other.

I would rather find a notation that is similar in the first and second uses, or that doesn't use an existing identifier. Obviously &+& has a meaning though, namely, ($) => $ + ($$) => $$. And it could get more ambiguous with e.g. &(&).

Some possible proposals (which I think I mentioned previously in Discord):

1. A second & refers to the same variable as the first, so &+& would mean ($) => $+$. This would forbid nesting a & function shorthand inside another one, but we can use alternatives to just &. For example, if you want the current meaning of &+&, you could use something like &+&& or &+&2 or &1+&2.
2. Almost the opposite: && or &1 could mean "re-use the last & variable", similar to how $ behaves now. We could extend to &&& or &2 referring to two levels up, etc.
3. Similar to 2, we could use a special notation for the current & argument, like ^, or ^^ for two levels up etc. Or &^, &^^, etc.

It occurs to me that && is probably syntactically ambiguous with the and operator, so the above suggestions are only really serious with the &n notation for an integer n, or something else like &^.

FWIW, Ruby, Crystal, and Elm don't seem to have anything for this. So another option (other than leaving things as is) would be:

4. Forbid accessing the argument a second time. &+$ would become ($$) => $$ + $. If you want ($) => $+$, write that.

Also possibly related to pipe operator and #75 (comment) ...

class D<T = any> {}

Should compile to:

class D<T = any> {}

Instead of:

class Dextends(T = any> {}) {
}

https://coffeescript.org/#try:a%20%25%25%20b%0A

a %% b

function modulo(a: number, b: number):number { return (a % b + b) % b; };

modulo(a, b);

x = y == z fails to parse (even with coffeeCompat turned on).

x = (y == z) works, but the parentheses shouldn't be necessary.

The LSP should re-parse everything when tsconfig.json changes. Currently I need to restart. By contrast, .tsx files update when I change tsconfig.json.

Ideally the same for node_modules/@danielx/civet, for upgrading Civet more seamlessly.

import A, {B, type C} from 'library'

Should compile to:

import A, {B, type C} from 'library'

and should compile to:

import A, {B} from 'library'

when passed compiler option js: true.

Currently y := x => x + 1 compiles to const y = x(() => x + 1) which may be a source of confusion for people coming from js which allows parenthesis to be omitted for single argument.

Should we consider adopting the js behavior by default (unless coffeeCompat is set to true) ?

Perhaps to reduce ambiguity we enforce that arg list can never be omitted, so instead of y := -> 1 we'd have to write y := () -> 1

We should add support for this:

Civet

abstract class X
  abstract y

Transpiles to

abstract class X {
  abstract y
}

Transpiles to (--js):

class X{

}

Currently we don't handle this correctly: https://coffeescript.org/#try:x%20%2B%20a.b.c%3F

Safari 16.0
macOS 12.6

It works in Chrome on the same computer.

There's an error in the console, it may be related.
SyntaxError: Invalid regular expression: invalid group specifier name

https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_rename

@STRd6 points out we can delegate the renaming task to the TS LSP (or copy/paste implementations from it) on the transpiled code, and then remap the changes back through the sourcemapping — essentially the same thing we do for hover.

Livescript support piping partials and curry function. Should be nice have this in Civet

More of a question than an issue.
What's the import situation going to be (is?) with Civet?
Will we be able to import other .civet files as ES6 modules like in typescript?
Or is it going to be like in Coffee where you can import other .coffee files as Node modules, but not as ES6 modules?

Do we need the non-civet-file activation events here ?

I'd like civet extension to not take over for other non-civet related pure-ts/js projects. I can disable it per workspace but I juggle between a dozen plus repos quite frequently so its not ideal.

Given a simple program:

var something = "else"

when compiling, I get the following error:

Error: Cannot find module 'node:readline'
Require stack:
- /Users/zolo/project/ftl/node_modules/@danielx/civet/dist/civet
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:92:18)
    at Object.<anonymous> (/Users/zolo/project/ftl/node_modules/@danielx/civet/dist/civet:13:12)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:76:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/Users/zolo/project/ftl/node_modules/@danielx/civet/dist/civet' ]
}

Civet is installed locally (not globally) and is ran like so:

civet ./something.civet ./something.ts

Hello, thanks a lot for creating Civet - This looks great. I spent some time playing around with it yesterday and loved the DX.

Since this language is quite early, I was wondering if we could eliminate the need for closing xml tags in jsx, and adopt an indentation based syntax similar to imba. I think this will make it more coherent with the rest of the language as well.

Imba is tied to its own dom/rendering impl. where as I am trying to use civet with solidjs which has its own jsx preprocessor.

@IamIpanda posted this example:

class A
  a: number
  
class B extends A
  b: number

https://civet-web.vercel.app/#Y2xhc3MgQQogIGE6IG51bWJlcgogIApjbGFzcyBCIGV4dGVuZHMgQQogIGI6IG51bWJlcg==

Currently, this looks like and gets treated as a function call A({b: number}). But this is almost certainly not what the user meant. This is a special feature of classes where we can have bare type declarations like b: number (FieldDefinition within ClassElementDefinition within ClassElement).

@STRd6 What's the best way to communicate that ExtendsClause shouldn't, at the top level, use the implicit-function-call-with-implicit-object rule? Do we need to duplicate the whole rule tree from LeftHandSideExpression? Or set some state and disable caching? Pretty annoying...

It's not strictly necessary, but we ideally would still allow implicit function calls not at the top level, e.g.:

class B extends (A
  b: c
)

Update: Oh, I notice IndentedApplicationAllowed is exactly for this purpose. Just need to figure out which cache rules to invalidate...

Here's a small example from TypeScript:

namespace Animals {
  export interface Legged {
    numberOfLegs: number;
  }
  export class Dog {}
}

• namespace doesn't have an indentation form; it needs braces
• export interface doesn't parse in general
• export class Dog doesn't parse inside namespace Animals {...}

Use a ref in non-simple chained comparisons so not to repeat evaluations of complex expressions.

f1(x) < f2(x) < f3(x) < f4(x)

let ref
f1(x) < (ref = f2(x)) && ref < (ref = f3(x)) && ref < f4(x)

[
  1
  y: 2
  z: 1
  w: 2
]

currently compiles to

[
  1,
  {y: 2({
  z: 1,
  w: 2,
})}
]

Playground

Reported by Oren

When &blocks are combined with unary ops, binary ops, and call expressions the parse tree doesn't reflect the actual structure. It works by chance in simple cases but won't in complex ones.

I'm not sure what I'm missing from the readme, but I cannot seem to able to define multiline functions.

Given:

let bla = (one, two) ->
    console.log one
    console.log two

It compiles to:

let bla = function(one, two) {}
    console.log(one)
    console.log(two)

There is just a single line break after -> and the indentation is the same on the 2nd and 3rd line.
I'm using the VSC extensions, so the language server is already displaying me an error before compilation, saying
Cannot find name 'one'

Changing it to an arrow function:

let bla = (one, two) =>
    console.log one
    console.log two

Results in:

let bla = (one, two) =>
    console.log(one)
    console.log(two)

Here's the same with types:

let bla = (one: string, two: string) =>
    console.log one
    console.log two

It turns to:

let bla = (one: string, two: string) =>
    console.log(one)
    console.log(two)

Add support for implicit async function declarations.

json: (srcFileName, outFileName) ->
  version: 3
updateSourceMap: (outputStr, inputPos) -> outputStr

currently transpiles to

{json: function(srcFileName, outFileName) {
  return {version: 3,
updateSourceMap: function(outputStr, inputPos) { return outputStr }}
}}

(based on real code from source/util.coffee)

Playground

I'm really really happy about the existence of a real time checker in the form of the VS Code plugin.
I know it's in an early state.

Often times it gets stuck, showing errors for lines that don't exist anymore.
I noticed that its memory is bound to file names, so if I delete test.civet, but rename another file to test.civet, it will instantly start to show the phantom errors in that file.

Is there a way to ask the extension to forget everything and recheck the whole file from scratch? Maybe a keyboard shortcut? Is that possible?

PS: off topic, but love that we have a file icon now

CoffeeScript allows for indentation in the middle of e.g. an array literal:

[1
 2
 3]

Civet won't parse this, even after #26.

More generally, I think whenever we have indented forms, we often need to support switching from regular form to indented form, at least once. For example:

[1, 2, 3   # regular form
 4, 5, 6   # switch to indented
 7, 8, 9]

Oddly CoffeeScript (see ArgElisionList in grammar.coffee) supports additional indentation blocks in the middle of an array literal:

[1
 2
   3
]

This is probably less important to support. But one indentation level is pretty common I think.

There are no types for the esbuild-plugin submodule. This is useful when creating esbuild/tsup configs with typescript.

There should be some basic tests of the CLI and its options.

Trying to merge the TypeScript TextMate grammar and an updated CoffeeScript grammar seems like it would be an unspeakable nightmare. So it would be nice if the Civet parser / LSP could provide semantic tokens.

Essentially tokens that have {$loc} would be source tokens styled from the nearest ancestor that has some sort of {sematicToken} property. It might require passing a flag into the parser and skipping normal AST processing instead constructing a stream of semantic tokens.

https://code.visualstudio.com/api/references/vscode-api#DocumentSemanticTokensProvider (looks like it is very similar to how source maps work)

What is the Civet syntax for TS decorators?

Given:

const printable = (target: any, memberName: string) =>
    console.log memberName

class Person
    @printable
    name = 'Jon'

I get the warning Member 'printMemberName' implicitly has an 'any' type.typescript(7008) because @ is used for static members.

The Code Sample at the top of the readme contains the

experimentalDecorators: true

flag in the compiler options, so I assume they are supported.

Vue has a cli utility called vue-tsc, which can be used for type-checking in CI env etc. It would be good to have something similar for civet.

We need to keep up with additions to TS like satisfies. My initial feeling is that this will fit in similar to as which we already support.

https://devblogs.microsoft.com/typescript/announcing-typescript-4-9/#satisfies

class X
  y: => b

https://coffeescript.org/#try:class%20X%0A%20%20y%3A%20%3D%3E%20b

Thank you very much for your efforts on Civet. I am intrigued because I used to love CoffeeScript and wrote various projects with it.

I built a small library to make CoffeeScript support Mixins or Traits in an idiomatic way.

Maybe check out this code and tell me if you like the style of it and if it's worth porting it to Civet.

https://github.com/bennidi/coffee-latte/blob/master/src/mixins.spec.coffee

"civet coffeeInterpolation"
"Hello #{name}!"

compiles to "Hello #{name}!" (incorrect)

"civet coffeeInterpolation"
console.log "Hello #{name}!"

compiles to console.log(`Hello ${name}`) (correct)

Oren wanted something like this on Discord: (originally suggested without do keyword)

x = do
  y := 5
  y + 1

It does seem like a natural extension to do ... while currently supported, i.e., make the while optional. It also corresponds to this proposal (though we don't need that proposal for if/while expressions).

Related, I expected this to work, but it doesn't parse: (it works without the :=)

x = if true
  y := 5
  y + 1

Current workarounds, which work but are a bit inefficient as they create arrays:

x = (do
  y := 5
  y + 1
while false)[0]

x = (for nothing of [1]
  y := 5
  y + 1
)[0]

Parses:

class
  static a = 5

Doesn't parse:

class
  @a = 5

class
  constructor(x);

These examples work if the class has a name, as in class X.

This is a feature request rather than an issue. Originally I wanted to inquire about macro support, but to be honest what I'm mostly looking for is operator overloading and custom operators.

Swift has a wonderful feature where you can overload prefix, infix and postfix operators or create new ones to your heart's content and it's the best. It has the potential to make the code very succinct so such feature I believe is such an obvious fit in Civet and CoffeeScript.

For example, for a long time, Swift had an awfully verbose regex matching API, but I could just define the ~= operator on String and I could express with two characters that otherwise would have taken 4 lines to do:

url ~= ".*/account/.*/login"

Or how about using operators for matrices? Makes so much sense.
Or how about some reactive code where you could just add together multiple observables to get a combined signal?
Or how about a language that uses := as a readonly shorthand? 😉

Mathematical operators is one of the best known and most succinct language we have as humans and I feel like it's such a waste to use them only for numbers.

I would really really love a Swift-like solution. 🚀

On Windows:

$ npm install -g @danielx/civet

added 1 package, and audited 2 packages in 3s

found 0 vulnerabilities

$ civet
node:fs:221
  let mode = stats[1];
                  ^

TypeError: Cannot read properties of undefined (reading '1')
    at isFileType (node:fs:221:19)
    at Object.readFileSync (node:fs:470:16)
    at Object.<anonymous> (C:\Users\edemaine\AppData\Roaming\npm\node_modules\@danielx\civet\dist\civet:12:12)
    at Module._compile (node:internal/modules/cjs/loader:1120:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1174:10)
    at Module.load (node:internal/modules/cjs/loader:998:32)
    at Module._load (node:internal/modules/cjs/loader:839:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47

Node.js v18.7.0

On Linux:

$ npm install -g @danielx/civet

added 1 package, and audited 2 packages in 2s

found 0 vulnerabilities

$ civet
node:internal/fs/utils:348
    throw err;
    ^

Error: EAGAIN: resource temporarily unavailable, read
    at Object.readSync (node:fs:748:3)
    at tryReadSync (node:fs:448:20)
    at Object.readFileSync (node:fs:494:19)
    at Object.<anonymous> (/afs/csail.mit.edu/u/e/edemaine/.nvm/versions/node/v18.12.1/lib/node_modules/@danielx/civet/dist/civet:12:12)
    at Module._compile (node:internal/modules/cjs/loader:1159:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
    at Module.load (node:internal/modules/cjs/loader:1037:32)
    at Module._load (node:internal/modules/cjs/loader:878:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:23:47 {
  errno: -11,
  syscall: 'read',
  code: 'EAGAIN'
}

Node.js v18.12.1

Add support for implicit generator functions in coffeeCompat.

x?.y = 0

https://coffeescript.org/#try:x%3F.y%20%3D%200

So I frequently run into function application getting introduced at unexpected places. An example that currently behaves weirdly is:

 foo := [
  a: 10
  b: 20
]

const foo = [
  {a: 10({
  b: 20,
})}
]

There were a few previously discussed comments which were also variations of same problem: #84 (comment), #83 (comment)

I think crux of the problem is that our rules are evaluated inside out and indentation tracking (PushIndent, PopIndent) is interspersed in the same pass.

So when a: 10 is getting parsed, the rule for NestedElementList:

NestedElementList
  PushIndent NestedElement*:elements PopIndent ->

is yet to be evaluated. The PushIndent in this rule hasn't yet run so module.currentIndent is still same as previous level (in this case 0)

Because of that the next indented expression b: 20 appears to be indented further than current level, which is wrong.

This can probably be addressed by a smart rearrangement of rules on a case by case basis, but that is pretty hard to reason about (atleast for me).

A simpler solution is to potentially separate out indentation tracking in a separate previous pass (perhaps alongside lexing).

Alternatively the parser could apply rules left -> right and also provide a way to revert any shared state modifications if rules to right don't match.

• {func(){5}, x} compiles correctly as {func(){return 5}, x}
• {func(){}, x} compiles incorrectly as {func() {return {}, x }}
• {x, func(){5}} fails to compile at all; expected {x, func(){return 5}} which works in ES

• declare let $: JQuery doesn't parse
• declare global {}, declare module "hot-new-module"; and declare module "path" {} parse as function calls
• declare class

There are many cases where keywords cause the parsing to fail:

S.obj nullable: false

fails to parse because "able" is unexpected after null

Also tangentially, quoting nullable doesn't help

Similarly S.obj true: false fails to parse.

If I update the NullLiteral to be defined as :

NullLiteral
  /\b(null)\b/ ->
    return { $loc, token: $1 }

The nullable issue goes away, but things break if I add wordbreak boundaries to other literals like true/false. Perhaps there is a better/recommended way to handle this in hera ?