Introduce an `%import` pragma about compiler HOT 9 CLOSED

Hello :-),

During my PhD thesis, I mentionned this feature and addressing it is a very challenging task. You don't only import rules here, you are extending them or even worse: You are also overriding them. This is difficult. Hardly difficult. Extending would be difficult, refining would be much more difficult, but overriding… damn. This is too error-prone because: What if you are depending on a grammar that radically changes? Your grammar will be broken. We have to detect this, and this is very difficult.

from compiler.

So, it would be 👎 for me regarding the amount of work, but, let's the community decides. If this is something the community asks, I could work on it. Don't expect this feature to be availble in a week. Just the formalization could take months.

from compiler.

@Hywan fine ;) I already reverted my code in tml.

TBH I'm not aware of the problems but if you can explain them or give me a link to something giving more clue, I would be glad to work on this.

from compiler.

I really love the idea, but I understand how hard this could be.
Maybe some mitigation could be some kind of
%import: path/to/file.pp:%token|#rule
that would just work like computer-assisted copy paste (are you thinking about traits?)

This way, no overriding, no extending, and no huge copy paste, only a big list of imports, and some custom rules.

What do you think about this?

Note: I understand this would lead to pp files with a lot of imports and only some custom rules, making files much much bigger than a full gramma import, but the idea here is not to reduce grammar length, only to avoid huge parts to be duplicated.

On the parsing side, I think (but I do not know the compiler internals) that this does not change radically the way compiler works

from compiler.

@CircleCode I was also thinking of %importing only tokens or only rules but would propose it in a second time, but I was probably wrong.

A first step could be to support on token imports but we still have to take care of some things: if the imported grammar changes one of its token regex, the importing grammar could break.

from compiler.

@jubianchi right, but the break would be easy to detect since once the replacement (aka import) has been done, the grammar should be good on its own

from compiler.

It's really interesting, but I understand @Hywan, it seems hard ...

• We could assume than %import% does like an include in PHP, if you override tokens, rules, ... the 2nd occurence override the 1st.
• We could use %import% to load in background, then use xx_token; use yy_token; etc..., It would be more flexible but harder to implement for client.

from compiler.

Sorry for my dummy question.
Before building the AST, a file is like a key-value of "lines"?
Can we rebuild a cached .pp which is a merge of both files.

(I'm always curious about the context leading to write a new grammar).

from compiler.

@1e1 I don't understand your comment. A grammar is a set of lexemes and a set of rules. This is not simply “merging” them. A grammar describes a syntax. Merging syntax and still be coherent or detect conflict is not simple. Actually, I don't know if this is even automatically done.

from compiler.