The docs command should never execute any piece of Crochet code, as that is counter-intuitive (and will not work due to the VM not loading any capability or native module). Allowing it to execute preludes means that some packages are effectively undocumentable---the CLI will crash because the VM cannot complete the boot sequence---if they have any of those declarations.

The idea of DSL was to better support embedded DSLs where users would have more control over how to interpret the syntax, but the syntax itself was fixed. It turned out to be cumbersome, and at the same time made it hard to provide good debugging tools for them, even with it being more static. Crochet will go down the path of having only external DSLs (properly integrated in the IDE, build system, and VM).

Crochet separates declarations from expressions to avoid imposing an evaluation order for declarations (and to allow analysis of shapes without security nightmares). However, as currently implemented, declarations are evaluated in the order they appear. This is particularly problematic for types for two reasons:

1. It makes it harder to properly organise modules, leading to the current 0-types.crochet pattern; but more importantly
2. It makes it impossible to declare recursive or mutually recursive types, and that makes field contracts often useless. Though this is mitigated by separating cases in a sum type into independent declarations.

Anyway, this should be fixed in the VM so it can conform to the (not yet described) language spec. The simple solution is to allow referring to non-existing definitions with a placeholder, and then replace the placeholder with the correct implementation once we've seen all of its dependencies. We do, however, expect to be able to know some things about the type in several declarations (e.g.: in order to sort commands), so this change has quite some annoying ripple effects.

During the verification phase, if there are any placeholder entities that have not been fulfilled for the package, the package loader will stop with an error. Currently the error looks something like:

Cannot open <package> because it's lacking definitions for: <entity>...

This message lacks any indication of where these entities are used, forcing users to search for them. But it also does not suggest fixes. For example, forgetting to open a package's namespace and using types from that package would result in that message, but the fix would be to open the package's namespace, not to add a new local definition. Typos, similarly, should be reported as possible typos, not as a lacking definition.

This happens because the debugging representation of values uses the locale formats, but not all of them go well with the rest of the syntax around debugging representations. E.g.:

>>> 3 hours + 8900 minutes | balanced-to: du-seconds
(duration from crochet.time) [
  days -> 0,
  hours -> 0,
  minutes -> 0,
  seconds -> 544,800,
  milliseconds -> 0
]

Underneath they're converted to _ flat-map: _ calls, however they always produce a list back, and thus cannot be used with other types.

E.g.: the following fails:

for X in #set from: [1, 2, 3, 4] do
  X + 1;
end

It should yield the set [2, 3, 4, 5] instead.

#71 has added a new interactive playground which should replace the previous command-line REPL, even for non-browser projects. However, currently it only supports running browser packages. Future versions of Crochet will completely abandon the command line, and this is blocking that :)

The REPL currently executes code in a single environment, which is reused across all subsequent REPL pieces of code. This usually works correctly, but some expressions, like A | b are compiled with intermediate generated variables, and these get attached to this single shared environment during the execution as well.

As a result, it's not possible to evaluate something like: #result ok: 1 | map: (_ + 1) | value; #result ok: 1 | map: (_ + 1) | value in sequence.

One thing that can be done is to just spawn a new environment to execute the code, then copy all non-generated variables up to the shared one. A generated variable will always have a $ character in its name, and such character is not valid in Crochet variable names.

As a follow up on placeholder types for out-of-order definitions, we need to also forbid constructing placeholder types, since they don't actually make sense. We currently check for package consistency when loading, but that doesn't work with the playground, which loads definitions directly.

Packaging has been broken for a while. It does not consider transitive dependencies, and it knows not about capabilities and isolated filesystems (which it needs to maintain in order to make assets work). Packaging also does not really work for Node targets, meaning that you cannot build an executable from some Crochet package---this in turn makes it difficult to build the other necessary tooling.

If a native module contains the following:

ffi.defun("hello", () => ffi.text("Hello!"));
ffi.defun("hello", () => ffi.text("Goodbye!"));

One would expect the second declaration to crash the loader with a "duplicate entry" error, but the declaration is silently ignored.

Where is the "util" lib? Don't see it in package.json nor anywhere in "generated/" parent.

EDIT: Sorry, forgot to mention the file, it's https://github.com/qteatime/crochet/blob/main/source/generated/crochet-grammar.ts#L5

ps. Big fan of your art and all your projects!

The Crochet project started on the origamitower organisation, but was later moved to the qteatime organisation here on GitHub, but the packages remain published to origamitower on npm. This causes unnecessary confusion and suspicion; the npm packages, the GitHub packages, and the official website should all share the same organisation name.

When the simulation runs, it computes the list of reactions only once for the current turn. However, if a reaction causes any effect it might invalidate further reactions that have been scheduled to fire. This means that the reactions would be fired incorrectly.

Consider the tic-tac-toe example in this repository:

when X simulate-turn, if X won do
  fact state--won state;
  game log: "[X name] won!";
  game update;
end

when state--on-going state, X is player, if not (X won), not L at: C mark: empty do
  fact state--draw state;
  game log: "It's a draw!";
  game update;
end

Here, as part of the reaction to marking one of the cells, the game may either enter a winning or a draw condition. These separate reactions take care of that: if, at the end of one player's turn, the X won predicate succeeds, then we move the game to the winning state. But if we've exhausted all cells and found no winner, we move the game to the draw state---this further explicitness in checking for a winning condition is also necessary because Crochet does not guarantee any ordering between events.

Now, the second reaction is problematic. We have more than one player, so each player triggers a reaction here when the game reaches a draw state, because neither of them will have won. As a result of reaching a draw state we move the game to "state--draw", but we also cause effects that show this on screen. In the tic-tac-toe example, this causes It's a draw to be logged twice, because we run the reactions sequentially but we compute them eagerly, before we fire all of them.

The language could choose to not validate the preconditions after reactions, which would force users to rewrite their reactions with this in mind, and manually take care of multiple reactions firing. But that does not seem like a good direction, so it's quite likely that this will be solved by guaranteeing that reactions only fire if they're still valid.

Records are currently a "less useful" map type, they should be just an extensible record with no dynamic features. This will make it possible to extend dispatch to records as well, but it requires some work in converting some current misuses of records into uses of map.

Currently all of the packages in the standard distribution are part of the trusted base. Packages in the trusted base get more powerful FFI access, construction and projection capabilities over intrinsic types, and are able to load native code without requiring a native capability.

This makes the surface of dangerous attacks in Crochet too large. Bugs in an otherwise innocuous package, such as crochet.text.regex, could allow malicious code to get access to all of this power, effectively subverting most of the runtime safety mechanisms in Crochet.

This won't be addressed for the first experimental release, as it requires significant amount of work around safe native code support, but it's important that people are aware of this issue.

Currently it just grants some powerful capabilities to the root package, including native, which pretty much gets rid of all security measures Crochet could provide (there's a related issue of passing trusted FFIs to untrusted native modules here that also needs to be tackled) :/

This is a really unfortunate consequence of packages not supporting mutual dependencies. Everything depends on crochet.core, including crochet.debug, consequently crochet.core cannot have any dependencies. Due to the strict capabilities on open coupled with dependencies, it's not possible for core to open the debugging package.

This will require some more thinking around both debugging tools and the roles of dependencies, capabilities, and provided services.

Also marginally related to #10.

Currently the Playground runs both Node and Browser pieces of Crochet code in the main thread, which means that the Playground UI itself freezes in case the evaluation does not complete in reasonable amount of time. This is not acceptable for the stable release, but can be tolerated for the experimental releases. Fixing involves moving the kernels to an isolated worker.

Currently, Crochet does not do any changes to JavaScript's semantics. But any JavaScript code loaded after Crochet might do so, because the intrinsic objects are globally mutable in the default realm. This means that even semantics Crochet relies on for correctness, and security assumptions it makes based on JavaScript's default semantics might turn out to be incorrect. This is not acceptable.

Of course, the problem is that Crochet can't just lock down the semantics on its own: for example, in the Browser it is the user who decides when to load Crochet, and at that point in time several other pieces of JS might have been loaded---they might even have been injected there by random extensions the user has installed. At that point there's nothing Crochet can do, because there's nothing it can realistically assume from the environment. On the other hand, if Crochet does manage to be the first thing loaded and then lock down the realm (to guarantee some base semantics), any code being loaded after that does not know this semantic change has taken place will not work, causing a fair amount of frustration to users. If Crochet uses a separate realm to load itself FFI suffers. There's no winning in this game. And the user that is embedding Crochet needs to be, unfortunately, aware of all of this complexity.

This is not currently considered a pressing issue because Crochet is in experimental release, however, in the presence of thirdy-party untrusted code, this makes Crochet vulnerable to silly JavaScript attacks like prototype pollution. This is, to some extent, the case with Crochet's trusted code base. Though Crochet keeps external dependencies on the runtime minimal, none of them have been verified, and should not be considered secure.

This allows these documents to exist outside of their rightful boundary, which right now allows them to be confusing to the user (and hard to debug). Because interactive components are not a thing in the document language yet, this is a moderate issue, there are not enough compelling ways of using this for harmful attacks.

E.g.:

#document plain-text: "Annoying text"
  | position: (#doc-point2d x: 0 y: 0)

Non-size-suffixed type names should be reserved for arbitrary-precision types. If a type is bounded to a certain size, this should be reflected in its name. Doing so avoids confusing situations where people may not reason about the information bounds they're encoding in their programs, thus making programs unsafe.

Note that this also implies that a decimal type, which is arbitrary precision, must be provided by crochet.core.

This is an intended behaviour, so the aim of this ticket is not to remove it, but to add a bit more of controls to it so users can consent to it being done. Because packages can add commands to any type, they may be able to trick users into executing code that they would otherwise not do.

The issue is already mitigated with the use of capabilities. Even though, in theory, attackers can add commands to any types, and try to trick users into invoking their version of the command (likely a typo), they are still restricted in what they can do; they can do exactly what the capabilities granted to their package allows them to do. This reduces the amount of interesting attacks that can be done.

Hi, I'm reading the Crochet documentation. I"m using the dark theme and string literals are displayed on a bright background which makes them hard to read, mainly on my phone. I would expect them appearing on a dark background instead.

Crochet is currently not designed with accessibility in mind. What this means is that it will need to go through a redesign phase later to fix this (which will need to happen before a public release). This is otherwise just here to keep track of the issue.

Crochet's entire set of security guarantees rely on being able to control which packages have access to which types, as well as what they can do with these types. Types represent capabilities. But there are some more fine-grained capabilities in a type: using a type has different security implications from constructing a type or projecting fields from a type. That's why Crochet defines these as distinct capabilities.

Capabilities are handled at a "package" level. So packages define a trust boundary. In the specified semantics, constructing and projecting capabilities are granted to the defining package. Meaning that if package A defines a type T, then only package A is able to use new T or TValue.field. This is currently unchecked, which leads to these expressions succeeding outside of A as well.

This is a meta-issue to keep track of the security issues that are currently known, but not deemed immediate enough to fix during the experimental Crochet phase. That doesn't mean these security issues aren't dangerous, though. And they do certainly pose a threat to pretty much all security guarantees Crochet can provide.

• Node wrapper libraries don't use effects --- this makes it hard to provide safe experimentation with them.
• No IR validation --- this means attackers are able to craft IR binaries where define operations can execute arbitrary code. What this means is that attacks abusing capabilities can take place at load time rather than run time. The launcher currently assumes loading a package is safe, making opening packages dangerous.
• Non-audited dependencies --- the dependencies that make up Crochet's VM and tools have not been audited. It's possible that they contain undesirable code and capabilities.
• Non-verified JS compiler --- Browser packages are generated with Browserify, which includes its own runtime and isn't a verified compiler. It's possible that Browserify introduces bugs that weren't present in the original source code.
• Trusted FFI provided to non-TCB packages --- The trusted FFI would allow attackers to modify the VM semantics, and interact with the VM in unsupported ways. That said, native code isn't sandboxed at all to begin with, so there are bigger problems there.
• Some TCB dependencies are not audited --- And we rely on package managers to install them. This makes the VM, runtime, and tools vulnerable to dependency hijacking.
• Typed data supports dynamic projections --- If a piece of code uses X.[K] for their own internal typed structure, and K is provided by an attacker, they would be able to bypass projection capabilities and extract private data.
• No provenance checks on packages --- Crochet relies on package names being unique and readily identifiable (traceable to a source), however there's nothing that enforces this today. As a result attackers can abuse this by setting the name of their package to anything and trick users into granting them capabilities.
• Assets can be accessed by other packages --- Currently the URI for assets on the web includes no capabilities, so any package with the ability of making network requests can access other package's assets.
• Sensitive data may be leaked with debug representations --- As long as Crochet's debugging tools are in this transitionary state from rudimentary JS pretty-printing to capability-aware representations in Crochet code, the JS pretty-printing part will leak possibly boxed/secret data to standard output and other debugging outputs.
• Paths generated without semantic guarantees --- The assets code currently generates some paths without semantic guarantees about their composition. There might be cases where this results in paths that are unexpected. Also file systems are a mess so there's a lot else here that needs attention.

Though all strings should be parsed as interpolations, if they have at least one hole expression in them, having [] causes the expression to be parsed as a literal string---not an interpolation---even if other holes are well-formed. The parser should be more strict here, commit to the interpolation grammar, and just issue an error in these cases.

E.g.:

"this [should be an interpolation] but the following [] causes it to be parsed as a literal string"

Due to how PEG's first valid choice semantics work, this also affects everything that cannot be parsed as an expression. "this is a [type]" will also fallback to a literal string...