Giter Club home page Giter Club logo

as-wasi's Introduction

as-wasi

npm version npm downloads per month GitHub License

A high-level AssemblyScript layer for the WebAssembly System Interface (WASI). ๐Ÿงฉ

WASI is an API providing access to the external world to WebAssembly modules. AssemblyScript exposes the low-level WASI standard set of system calls. as-wasi builds a higher level API on top of the AssemblyScript WASI interface, at a similar level to the Node API. ๐Ÿš€

Installation

You can install as-wasi in your project by running the following:

npm install --save as-wasi

Quick Start

Example usage of the Console and Environ classes:

// Import from the installed as-wasi package
import { Console, Environ } from "as-wasi/assembly";

// Create an environ instance
let env = new Environ();

// Get the HOME Environment variable
let home = env.get("HOME")!;

// Log the HOME string to stdout
Console.log(home);

Here are some exported classes that are commonly used:

  • FileSystem - Reading and Writing the user's fileystem. ๐Ÿ“
  • Console - General logging to stdout and stderr. ๐Ÿ–ฅ๏ธ
  • Environ - Accessing environment variables, command flags, etc... ๐ŸŒ
  • Date - Getting the current system time. ๐Ÿ“…
  • Random - Accessing random numbers. ๐Ÿค”
  • Time - Allow sleeping and waiting for events to occur. โฐ
  • And More! See the Reference API in the next section for the full API.

Reference API Docs

Reference API documentation can be found in REFERENCE_API_DOCS. Documentation is generated using typedoc.

Projects using as-wasi

  • wasmboy - Game Boy / Game Boy Color Emulator Library, ๐ŸŽฎwritten for WebAssembly using AssemblyScript. ๐Ÿš€
  • wasmerio/io-devices-lib - Library for interacting with the Wasmer Experimental IO Devices API. Uses WASI for outputting graphics in a framebuffer, and handles mouse/keyboard input.
  • wasm-by-example - Wasm By Example is a website with a set of hands-on introduction examples and tutorials for WebAssembly (Wasm). Wasm By Example features as-wasi by default for the AssemblyScript WASI examples.
  • wasm-matrix - A Matrix effect in your terminal using AssemblyScript ๐Ÿš€ and WASI ๐Ÿงฉ . This project is a bit older, and uses an older version of as-wasi, but still creates a cool effect!

If your project is using as-wasi, and you would like to be featured here. Please open a pull request against the README with links to your project, and if appropriate, explaining how as-wasi is being used. ๐Ÿ˜Š

Contributing

Contributions are definitely welcome! Feel free to open a PR for small fixes such as typos and things. Larger fixes, or new features should start out as an issue for discussion, in which then a PR should be made. ๐Ÿฅณ

This project will also adhere to the AssemblyScript Code of Conduct.

License

MIT. ๐Ÿ“

as-wasi's People

Contributors

austinfrey avatar bnjbvr avatar dependabot-preview[bot] avatar dependabot[bot] avatar fgasper avatar jedisct1 avatar maxgraey avatar mendyberger avatar mhonert avatar radu-matei avatar tachyonicbytes avatar torch2424 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

as-wasi's Issues

Allocation error when getting environment variables and command line arguments

Just instantiating an Environ object when environment variables are passed results in an error:

index.ts:

import { Console, Environ } from "as-wasi";

export function _start(): void {
  let env = new Environ();
  //   let all_vars = env.all();
  //   all_vars.forEach(function (val) {
  //     Console.log(val.key + "=" + val.value);
  //   });
}

package.json:

  "dependencies": {
    "as-wasi": "^0.2.0",
    "assemblyscript": "^0.14.0"
  },
  "scripts": {
    "asbuild-test": "asc index.ts --use abort=wasi_abort --debug -b output/repro.wasm"
  }

Trying to use the resulting module with Wasmtime results in an allocation error:

$ wasmtime output/repro.wasm --env foo=bar
~lib/rt/tlsf.ts:461:30: error: allocation too large

And the same happens with the WASI implementation in NodeJS (14.5):

"use strict";

const fs = require("fs");
const { WASI } = require("wasi");
const wasi = new WASI({
  env: { foo: "bar" },
});

const importObject = { wasi_snapshot_preview1: wasi.wasiImport };

(async () => {
  const wasm = await WebAssembly.compile(
    fs.readFileSync("./output/repro.wasm")
  );
  const instance = await WebAssembly.instantiate(wasm, importObject);

  wasi.start(instance);
})();
$ node --experimental-wasi-unstable-preview1 --experimental-wasm-bigint node_wasi.js
~lib/rt/tlsf.ts:461:30: error: allocation too large

It turns out reverting to version 0.0.1 of as-wasi results in the complete example above working in both runtimes, and going through the changelist, @technosophos and I pinned this down to how StringUtils works. Specifically, this particular change to the fromCString method - passing the maximum integer value instead of the actual size of the string seems to be the one generating the error in allocation (the decodeUnsafe method tries to allocate based on the actual size parameter, and I suspect this is where the overflow happens). Reverting just the fromCString method to its previous version seems to fix the allocation issue.

Note that the same error occurs when trying to get command line arguments, since it is the only other place where fromCString is used.

A quick fix for this would be reverting fromCString's implementation to the version before that particular commit.

Environ class cannot be used: "env::abort" has not been defined

Using the Environ class causes the program to crash with the following error:

Error: failed to run main module `env-test.wasm`

Caused by:
    0: failed to instantiate "env-test.wasm"
    1: unknown import: `env::abort` has not been defined

How to reproduce the issue

Use the example provided by the README of this project:

// Import from the installed as-wasi package
import { Console, Environ } from "as-wasi";

// Create an envrion instance
let env = new Environ();

// Get the HOME Environment variable
let home = env.get("HOME")!;

// Log the HOME string to stdout
Console.log(home);

Build the code and then execute it via wasmtime:

$ asc env-test.ts -b env-test.wasm
$ wasmtime run --env HOME=flavio env-test.wasm
Error: failed to run main module `env-test.wasm`

Caused by:
    0: failed to instantiate "env-test.wasm"
    1: unknown import: `env::abort` has not been defined

This is my stack:

  • wasmtime: 0.21.0
  • AssemblyScript: 0.17.1
  • as-wasi: 0.4.0

Wasi context stdin truncate file if exist, with no ability writing to it

Is there a way to pass data into stdin via wasi context?

When file already exist it will be truncated during wasi context creation

final var stdinPath = Paths.get("./wasm/io/stdin.txt");

Files.writeString(stdinPath, "{\"name\": \"John\"}");

final var wasi = new WasiCtxBuilder()
        .stdin(stdinPath)
        .build();

Another case, when trying to write data after context creation it is ignored

final var stdinPath = Paths.get("./wasm/io/stdin.txt");
final var wasi = new WasiCtxBuilder()
        .stdin(stdinPath)
        .build();

Files.writeString(stdinPath, "{\"name\": \"John\"}");

You can find full working example here: https://github.com/amysyk-viax-io/wasmtime-java-stdin-stdout

readLine doesn't return null when the end of the file is reached

The following reads a file line, by line, with the assumption that readLine returns null at the end of file:

index.ts:

import "wasi";

import { Console, FileSystem, Descriptor } from "as-wasi";

const filePath: string = "data.txt";
const fileOrNull: Descriptor | null = FileSystem.open(filePath, "r");
if (fileOrNull == null) {
  throw new Error("Could not open the file " + filePath);
}
const file = changetype<Descriptor>(fileOrNull);

let line = changetype<string>(file.readLine());
while (line) {
  Console.log(line);
  line = changetype<string>(file.readLine());
}

package.json:

{
  "scripts": {
    "build": "asc assembly/index.ts -b build/index.wasm -t build/index.wat --runtime half",
    "start": "wasmtime --dir . build/index.wasm",
    "build:run": "npm run build && npm start"
  },
  "devDependencies": {
    "assemblyscript": "0.17.3"
  },
  "dependencies": {
    "as-wasi": "0.4.4"
  }
}

However, readLine never returns null.

I instead have to check for an empty line:

xlet line = changetype<string>(file.readLine());
while (line.length > 0) {
  Console.log(line);
  line = changetype<string>(file.readLine());
}

Compile Error: Cannot find name 'abort'.

Just following the Quick Start:

// Import from the installed as-wasi package
import { Console, Environ } from "as-wasi/assembly";

// Create an environ instance
let env = new Environ();

// Get the HOME Environment variable
let home = env.get("HOME")!;

// Log the HOME string to stdout
Console.log(home);

Then building asc assembly/index.ts --target release throws compile error:

ERROR TS2304: Cannot find name 'abort'.
     :
 935 โ”‚ abort();
     โ”‚ ~~~~~
     โ””โ”€ in ~lib/as-wasi/assembly/as-wasi.ts(935,7)

ERROR TS2304: Cannot find name 'abort'.
     :
 944 โ”‚ abort();
     โ”‚ ~~~~~
     โ””โ”€ in ~lib/as-wasi/assembly/as-wasi.ts(944,7)

FAILURE 2 compile error(s)

This error disappears if I don't useEnviron but only Console for example.

Maybe related to #127

Console.readAll blocks until EOF

The Console.readAll function blocks until the EOF character is entered (e.g. by pressing CTRL-D in a linux shell), so it is not that useful for a CLI application.

It would be great if there was a function which reads a whole line until a newline character is entered (e.g. Console.readLine).

For portability this should probably take different newline character sequences into account (e.g. CR+LF for Windows and only LF for Linux/macOS).

Use with wasi-shim?

I see you've updated to use wasi-shim, but it's not clear how usage/configuration has changed.

  • No more need to import "wasi", right?
  • How do I correctly configure my asconfig.json?

Adding nothing results in a module that won't execute:

Error: failed to run main module `build/debug.wasm`

Caused by:
    0: failed to instantiate "build/debug.wasm"
    1: unknown import: `env::abort` has not been defined

Following the instructions from wasi-shim results in a module that won't compile.

ERROR TS2300: Duplicate identifier 'wasi_abort'.
      :
 1100 โ”‚ export function wasi_abort(
      โ”‚                 ~~~~~~~~~~
      โ””โ”€ in ~lib/as-wasi/assembly/as-wasi.ts(1100,17)
    :
 19 โ”‚ export function wasi_abort(
    โ”‚                 ~~~~~~~~~~
    โ””โ”€ in ~lib/wasi_internal.ts(19,17)

Null strings are always read from STDIN

Note well: this is a copy of bytecodealliance/wasmtime#2373, as requested here.

I've written a simple "echo" program using AssemblyScript and as-wasi. The program reads the user input from STDIN and writes it back to STDOUT.

Unfortunately it looks like I can never get back the input I enter.

  • What are the steps to reproduce the issue?

This the source code of the AssemblyScript program I'm running:

import "wasi"

import {Console} from "as-wasi"

Console.log("type something");

let input: string | null;
let msg : string = "nothing";

input = Console.readAll();
if (input != null) {
  msg = input!
}

Console.log('I got: ' + msg);

The program can be compiled to a WASM binary by doing:

$ asc echo.ts -b echo.wasm

And it can be run in this way:

$ wasmtime run echo.wasm

The program will start and I'll be able to enter my text. The program will keep reading from STDIN until I send the EOF symbol (I'm on Linux -> CTRL-D).

Unfortunately the input variable is always null.

  • What do you expect to happen? What does actually happen? Does it panic, and
    if so, with which assertion?

I would expect the WASM program to be able to read data from stdin. The input object should hold the text I entered on my terminal.

  • Which Wasmtime version / commit hash / branch are you using?

This is my stack:

  • wasmtime: 0.21.0
  • AssemblyScript: 0.17.1
  • as-wasi: 0.4.0
  • If relevant, can you include some extra information about your environment?
    (Rust version, operating system, architecture...)

I'm running on a x86_64 Linux box that has openSUSE Tumbleweed

More information...

The as-wasi handles STDIN/STDOUT/STDERR by using instances of the Descriptor class. The library simply opens the file descriptor 0 for STDIN, 1 for STDOUT and 2 for STDERR. In my tests it looks like opening the file descriptor 0 always returns a null instance of Descriptor; this doesn't happen with STDOUT and STDERR.

The same issue happens also when running the program through a custom made Go program I wrote leveraging wasmtime-go.
In that case I even tried to start the WASM binary not by inheriting the parent STDIN, but instead by using a text file I previously created. Also in this case the WASM binary got a null object when reading from the STDIN.

Relevant: a similar WASM binary, generated by translating Rust -> WASM, just works as expected.

calling `.stat()` on a `Descriptor` does not include correct file details

details:

wasm runtime: wasmer v0.4.2
assemblyscript version: v0.6
wasa version: master branch, commit f4162e73b6cbc3efd59f5cadd26b8b5b579b268f
// The entry file of your WebAssembly module.
import "allocator/arena"
import {
  Console as console,
  FileSystem as fs 
} from '../wasa/assembly/wasa'

export function _start (): void {
  let file = fs.open('./package.json')
  let fileDetails = file.stat()
  console.log(fileDetails.file_size.toString()) // always prints 0
  console.log(fileDetails.file_type.toString()) // always prints 0
  console.log(fileDetails.modification_time.toString()) // always prints 0.0
}

a few things come to mind...

  • this could most certainly be user error...
  • this could be an issue with the underlying runtime, so if that's the case, please punt this issue :)
  • could it be an issue converting to strings for printing? (other tests i've done say probably not?)

or it could be something else! :)

Install from Npm?

Hello!

Was wondering if you were going to allow this to be npm-installable? ๐Ÿ˜„

It could be installed from the Github directly if we add a name field to the package.json.

Thanks! ๐Ÿ˜„

readLine returns null prematurely

I'm using the following to load a multi-line file into a string array:

  const fileOrNull: Descriptor | null = FileSystem.open(filename, "r");
  if (fileOrNull == null) {
    throw new Error("Could not open the file " + filename);
  }
  const file = changetype<Descriptor>(fileOrNull);

  const lines = new Array<string>();
  let line = changetype<string>(file.readLine());
  do {
    lines.push(line);
    line = changetype<string>(file.readLine());
  } while(line)
  return lines;

However, give the following input:

one
two
three

The result is an array ["one", "two"] - the readLine method appears to return null prematurely

Implementing "libpreopen" functionality

Hello! I've run a lot of a AssemblyScript Wasm modules while working on Wasmer's WASI implementation and noticed that AssemblyScript doesn't seem to do the same kind of filesystem setup work that Rust and C do in regards to preopened directories. This manifests as AssemblyScript Wasm modules behaving incorrectly (well, at least differently from Rust and C) when dealing with multiple preopened directories and in some cases with relative paths.

I have a lot of context on the WASI filesystem and would be happy to make a PR implementing that logic!

I haven't had a chance to take a good look at the code yet, but part of what we'll need is to execute logic in _start, before main, if we can't do that, then that's something else that we'll need to implement.

It looks like the primary thing to update is here. As part of this, I believe we'll need to do prefix search. I'll start with a linear string comparison based search which can be turned into iterative hashing and then a proper data structure like a prefix tree. Given the number of pre open directories in cases I've seen, linear search or iterative substring hashing (at the file component level) are probably the right trade off in terms of binary size vs speed here.

So it really should just come down to:

  • Hook into _start
  • Enumerate the files and prestat them to build up some global data structure
  • Implement dirfdForPath (finding the right fd with the data structure from the step above)

Program execution is stopped because proc_exit is invoked with a not allowed value

This is a follow up of #95 - the reproducer is always the same, but I'm going to write it.

I've written a simple "echo" program using AssemblyScript and as-wasi. The program reads the user input from STDIN and writes it back to STDOUT.

Unfortunately the program crashes right after the EOF symbol is received.

  • What are the steps to reproduce the issue?

This the source code of the AssemblyScript program I'm running:

import "wasi"

import {Console} from "as-wasi"

Console.log("type something");

let input: string | null;
let msg : string = "nothing";

input = Console.readAll();
if (input != null) {
  msg = input!
}

Console.log('I got: ' + msg);

The program can be compiled to a WASM binary by doing:

$ asc echo.ts -b echo.wasm

And it can be run in this way:

$ wasmtime run echo.wasm

The program starts and I can enter my text. The program keeps reading from STDIN until it gets the EOF symbol (I'm on Linux -> CTRL-D).

The program crashes as soon as the EOF, this is the output I get:

type something
abort:  in ~lib/rt/pure.ts(112:14)
Error: failed to run main module `stdin-test.wasm`

Caused by:
    0: failed to invoke command default
    1: exit with invalid exit status outside of [0..126)
       wasm backtrace:
         0:  0x555 - <unknown>!<wasm function 9>
         1:  0x1f6 - <unknown>!<wasm function 3>
         2:  0x209 - <unknown>!<wasm function 4>
         3: 0x2490 - <unknown>!<wasm function 54>
         4: 0x2584 - <unknown>!<wasm function 57>
         5: 0x25d8 - <unknown>!<wasm function 58>

This can be triggered also in an automated fashion:

$ echo "hello" | wasmtime run echo.wasm
  • What do you expect to happen? What does actually happen? Does it panic, and
    if so, with which assertion?

I would expect the WASM program to be able to read data from stdin. The input object should hold the text I entered on my terminal.

  • Which Wasmtime version / commit hash / branch are you using?

This is my stack:

  • wasmtime: 0.21.0
  • AssemblyScript: 0.17.3
  • as-wasi: 0.4.3
  • If relevant, can you include some extra information about your environment?
    (Rust version, operating system, architecture...)

I'm running on a x86_64 Linux box that has openSUSE Tumblewee

According to this comment from @peterhuene:

It looks like something is calling proc_exit with an exit code outside of the WASI-defined acceptable range of [0..126) (granted this range isn't part of the current WASI snapshot yet, so Wasmtime is being a little overzealous with this enforcement).

README gives wrong npm install

In the README it says to install by running npm install --save as-bind instead of npm install --save as-wasi, is this a mistake or am I missing something?

Filesystem.openForRead always returns a file descriptor of zero

I can not seem to open a regular file for reading. In this example, i call Filesystem.openForRead('./foo') on a local file in the directory, but this method always returns 0 or the stdin file descriptor. am I using these methods as intended or am i missing something?

Thanks!

import "allocator/arena";

import { Console, IO, Filesystem } from "./wasa";
import { wasiabort } from './env'

// entrypoint for wasmer
export function _start(): void {
  let fd = Filesystem.openForRead('./foo')
  Console.log(fd.toString()) // prints '0'
  let str = IO.readString(fd)
  Console.log(str.slice(0, str.length - 1).split('').reverse().join(''))
}

Questions: VSCode support

Hey guys, I'm using as-wasi in an AssemblyScript project using VSCode. The functionality works fine, but there is a minor issue that VSCode is complaining about Cannot find module 'as-wasi'.

Screen Shot 2020-08-19 at 9 00 54 AM

Is there anything on your roadmap to get it supported? Thanks

Environment variables whose rvalue contains an `=` gets truncated

As far as I can tell, it is legal to have an environment variable like this:

QUERY_STRING=bar=baz

This is equivalent to QUERY_STRING="bar=baz".

However, if I try to access the above in as-wasi's Envron, I get only bar.

import { Console, Environ } from "as-wasi";

let env = new Environ();

// export QUERY_STRING=bar=baz
let foo_env = env.get("QUERY_STRING")
let foo = foo_env === null ? "not set" : <string>foo_env;
Console.log("Value of QUERY_STRING: " + foo);

The problem seems to be that split("=", 2) splits all of the =, but then returns only the first 2 results (unlike similar functions in other languages that actually stop parsing once the limit is reached).

License?

Hello!

Was wondering if you were going to give a license the project? ๐Ÿ˜„

Thanks!

new release, please

Bonjour,

Would you please make a new release, to solve compatibility with AssemblyScript via removal of wasi_abort() ?

Merci!

Unable to use Environ

Hello,
I'm trying to use as-wasi but when trying Environ it fails with

~lib/rt/tlsf.ts:457:29: error: allocation too large
Exception: wasi proc exit

I have tried several wasi-compliant runtimes but with the same result.

I'm using the latest master branch in package.json.
However using

    "assemblyscript": "^0.8.1",
    "as-wasi": "0.0.1"

it works.

Any suggestions, please?

RFE/Question: Binary files?

Hello,

Is it possible to add API support for Descriptor to read and write binary strings (i.e., ArrayBuffer)? (If itโ€™s there, I havenโ€™t been able to find it.)

Thank you!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.