Giter Club home page Giter Club logo

svm's Introduction

SVM - Spacemesh Virtual Machine

build license gitter

Project Goals

  • Self-contained. Should be hosted by the Spacemesh Golang full-node and future Spacemesh Rust full-node
  • Built on top of wasmer
  • Future compatible for the SMESH programming-language (the language is still in a research stage).

Rust

SVM depends on the single-pass compiler of wasmer which requires the rust nightly release channel. More information about installing rust nightly can be found here

Build & Tests

The SVM project is organized into a couple of crates. The root crate is called SVM and it's a workspace crate.

In order to build the SVM project crates at once and run their tests:

cargo build
cargo test

If you want to build & test in release mode execute:

cargo build --release --no-default-features
cargo test --all --release --no-default-features

Docker

In order to build the Docker image

docker build -t svm-build .

Then, for spawning a new container

docker run -it svm-build

Supported Operating-Systems

  • Mac
  • Linux
  • Windows

Got Questions?

svm's People

Contributors

avive avatar dependabot[bot] avatar imerkle avatar jorropo avatar kobby-pentangeli avatar moshababo avatar neysofu avatar yaronwittenstein avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

svm's Issues

SVM CLI - Spawn App

Overview / Motivation

Encoding a spawn app transaction via a CLI will help to test SVM in a black-box manner.

The Task

Implement under the svm-cli crate the functionality of encoding a spawn app transaction.
The output should be saved to file. We'd also like to decode a raw template.

Implementation Notes

  1. For encoding use AppBuilder :

    pub struct AppBuilder {

  2. For interpreting a raw spawn app use:

    pub fn parse_app(bytes: &[u8], creator: &Address) -> Result<SpawnApp, ParseError> {

  3. Please use the clap crate.

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rustfmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

Implement the SVM-wallet template

Overview / Motivation

We want to have an SVM-wallet Template that will manage the coins of each investor.

The Task

Implement the template such that each app will be associated with one investor.

Implementation Notes

  1. The SVM-wallet spec can be found at:
    https://github.com/spacemeshos/product/blob/f288afc0c980c5d189b789fca4eab3e9d4963ea6/svm-wallet.md

  2. Spawning the apps should occur as part of the Genesis flow.

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rustfmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

svm-codec: More accurate APIs naming

Motivation

This issue has been opened as a result of this PR comments:
#153

In order not to block this PR, we've agreed it can wait and just make sure we document that task.

Add C-API testing via C client

Overview / Motivation

We want to simulate a failed exec app transaction. The returned receipt should indicate that the transaction failed along with an error message.

The Task

The task will require decoding a raw receipt of a failed transaction and printing the failure reason.

Implementation Notes

  1. The raw format of a receipt of a failed transaction can be found here:
    https://github.com/spacemeshos/svm/blob/develop/crates/svm-runtime-c-api/src/receipt.rs#L15
  2. Add a file named panic.wasm under examples/c/wasm

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rustfmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

Create a basic CLI for SVM

Overview / Motivation

We'd like to have a CLI for SVM. It'll help us exercise SVM functionality as a black-box with various configurations and inputs.

The Task

  1. The CLI should support deploying of a new contract.
    The input can be provided as one blob of data or field-by-field explicitly.

  2. The CLI should support the capability of executing transactions of an existing contract.
    The input can be provided as one blob of data or field-by-field explicitly

Implementation Notes

  1. This file contains documentation of the binary format for deploying a new contract:
    https://github.com/spacemeshos/svm/blob/develop/crates/svm-contract/src/wire/deploy/mod.rs

We'd like the CLI to receive the following fields:

  • name (string)
  • contract code: file path (string)
  • author address: a string representing 20 bytes in hexadecimal format (0xA18C...)
  1. And, this file contains documentation of the binary format for execution a new contract:
    https://github.com/spacemeshos/svm/blob/develop/crates/svm-contract/src/wire/exec/mod.rs

We'd like the CLI to receive the following fields:

  • contract address: a string representing 20 bytes in hexadecimal format (0xA18C...)
  • sender address: a string representing 20 bytes in hexadecimal format (0xA18C...)
  • function: a string
  • args: we'd guide you how to represent it as a CLI input
  1. Please use the Rust clap crate for implementing the command-line:

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rust fmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

svm-runtime-c-api: auto-generate "wasmer_svm.h" using cbindgen

Overview / Motivation

We'd like to auto-generate the svm_wasmer.h file for the svm-runtime-c-api

The Task

We'd like to auto-generate a svm_wasmer.h C-header file out of this Rust file:
https://github.com/spacemeshos/svm/blob/develop/crates/svm-runtime-c-api/src/c_api.rs

Implementation Notes

  1. Currently, the svm_wasmer.h C-header file is manually built:
    https://github.com/spacemeshos/svm/blob/master/crates/svm-runtime-c-api/examples/svm_wasmer.h

  2. Please use this link as a reference:
    https://doc.rust-lang.org/cargo/reference/build-scripts.html

  3. Please use cbindgen for the task.

You may want to take a look at this code of wasmer as a good reference for using cbindgen
https://github.com/wasmerio/wasmer/blob/master/lib/runtime-c-api/build.rs

  1. The svm_wasmer.h file should be auto-generated as part of cargo nightly build
    (SVM requires Rust nightly)

  2. If possible, please auto-copy the generated svm_wasmer.h C-header file also to:
    https://github.com/spacemeshos/svm/blob/develop/crates/svm-runtime-c-api/examples/

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rust fmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

svm-wasmer vmcalls tests segfault

Here is an example of a test that segfaults:
https://github.com/spacemeshos/svm/blob/master/crates/svm-wasmer/tests/storage.rs#L125

#[test]
fn vmcalls_mem_to_reg_copy_segfaults() {
    let module = wasmer_compile_module_file!("wasm/mem_to_reg_copy.wast");

    let import_object = imports! {
        lazy_create_svm_import_object!(0x12_34_56_78, MemKVStore, MemPages, MemPageCache, 5, 100),

        "svm" => {
            "mem_to_reg_copy" => func!(mem_to_reg_copy),
        },
    };

    let instance = module.instantiate(&import_object).unwrap();

    // initializing memory #0 cells `200..203` with values `10, 20, 30` respectively
    wasmer_ctx_mem_cells_write!(instance.context(), 0, 200, &[10, 20, 30]);

    // asserting register content is empty prior copy
    let reg = wasmer_ctx_reg!(instance.context(), 2, MemPageCache);
    assert_eq!([0, 0, 0, 0, 0, 0, 0, 0], reg.get());

    let do_copy: Func<(i32, i32, i32)> = instance.func("do_copy_to_reg").unwrap();
    assert!(do_copy.call(200, 3, 2).is_ok());

    // asserting register content is `10, 20, 30, 0, ... 0`
    let reg = wasmer_ctx_reg!(instance.context(), 2, MemPageCache);
    assert_eq!([10, 20, 30, 0, 0, 0, 0, 0], reg.get());
}

Optimize persisted `Template` and `App`

Overview / Motivation

We want to optimize the data of persisted Template and App.

The Task

Implement encoder & decoder for Template and App.

Implementation Notes

  1. Use the same encoding as the raw format transactions.

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rustfmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

SVM CLI - Execute Transaction

Overview / Motivation

Encoding an exec app transaction via a CLI will help to test SVM in a black-box manner.

The Task

Implement under the svm-cli crate the functionality of encoding an exec app transaction.
The output should be saved to file. We'd also like to decode a raw template.

Implementation Notes

  1. For encoding use AppTxBuilder :

    pub struct AppTxBuilder {

  2. For interpreting a raw exec app use:

    pub fn parse_app_tx(bytes: &[u8], sender: &Address) -> Result<AppTransaction, ParseError> {

  3. Please use the clap crate.

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rustfmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

Not leaving any `TODO` (s) in the codebase without related issue

Overview / Motivation

As part of the effort of making SVM 0.2 production-ready for Spacmesh Mainnet MVP,
we must make sure there are no left TODOs in the code.

Each existing TODO should be taken care of by creating new issues and addressing them one-by-one.

Fix the C example

Overview / Motivation

The current pure C example is broken due to recent changes in the raw format.

The Task

The example is implemented here:
https://github.com/spacemeshos/svm/blob/9b29364f31e52b4e341b81c0708d50181533fef0/examples/c/counter.c

Instead of manually crafting the raw transactions, we'd like to generate them using the SVM CLI
and load them into the code.

Implementation Notes

  1. Have the raw data under a new directory called raw.
  2. Replace the code synthesizing the raw transactions with read-made files generated by the SVM CLI.

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rustfmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

Flexible `App Storage` pages size

Motivation

The motivation for this issue is to support different pages size of different apps.
Within an App storage, each page may have a different size.

Upon committing an app's storage changes, we override each dirty page.
If we can define for each page it's minimum required we can save a lot of disk space.

This opens a room for more optimizations. For example, if we have variables that are read-only or rarely being modified, we can group them together under the same page.

Having that feature will save translate to storage saving.

Build pipelines

Overview / Motivation

Currently, we have a very basic build using Travis CI.

We'd like to create a build-pipeline for Mac and Linux.
SVM won't compile on Windows yet, but the preparation work can be done already.

The Task

We want to have the following build pipelines:

  • Linux
    Desired output: (both debug and release)
    • libsvm_runtime_c_api.so
    • svm_wasmer.h
    • wasmer
      • libwasmer_runtime_c_api.so
      • wasmer.h
  • Mac
    Desired output: (both debug and release)
    • libsvm_runtime_c_api.dylib
    • svm_wasmer.h
    • wasmer
      • libwasmer_runtime_c_api.dylib
      • wasmer.h
  • Windows (preparation since SVM doesn't work yet on Windows)
    Desired output: (both debug and release)
    • libsvm_runtime_c_api.dll
    • svm_wasmer.h
    • wasmer
      • libwasmer_runtime_c_api.dll
      • wasmer.h

Implementation Notes

  1. Please implement the CI using Travis CI. If you'd like to use GitHub Actions we might go for it.
  2. Please create separate branches for Mac / Linux / Windows.
    • SVM require rust nightly, please build using this command:
      cargo +nightly build
  3. Tests shouldn't run in parallel (we'll explain why on the Gitter chat).
    Please run the tests using this command:
    RUST_TEST_THREADS=1 cargo +nightly test --all --release
  4. In case not all tests pass, the build must fail abruptly

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rust fmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

auto-inject vmcalls imports right before compliation

the motivation is to reduce the size of the smart-contract wasm programs.

so instead of persisting all the svm vmcalls imports, we can add them on-the-fly
to an in-memory representation of the wasm code right before passing it to wasmer

SVM Applications

We'd like to introduce two concepts to SVM called an app-template and app.
The motivation behind is to encourage code reuse between different contracts.

It not uncommon to want to see copy-paste of code between different Smart-Contacts.
We'd like to have a new mechanism that will help to eliminate such duplications.

The SVM will have:

  1. app-template - a one-time declaration of a template code plus more metadata.
    an app-template isn't runnable. But can spawn live instances out of it.

  2. app - an app is the name of a spawned app-template instance.
    Each app will hold its own internal storage and will only share the same code with the other instances.

gas-estimation rewrite (linear complexity)

We want the gas estimation to be of linear complexity,
so that's why we rewire the current implementation which can have O(n^2) on the worst-case when n is #instructions

`host_ctx` feature: Runtime `exec_app` should receive info about the current host context

Overview / Motivation

We want to give the running apps fast access to the host current running context.

For example:

  • sender address
  • sender balance
  • app address
  • app balance
  • layer number
  • block id
  • transaction id

The Task

The Runtime trait execute app transactions via the exec_app method.
This method should receive the host context and wrap each field in a read-only method that
will be accessible to the running wasm apps.

plain C tests for `svm-runtime-c-api`

Motivation

Since crate svm-runtime-c-api emits svm.h and svm object file. The ideal tests should be written in C (instead of the current Rust ones).

Basic observability measurements

  • Deploy a new contact using the c api
  • Execute the same contract transaction 1000 times sequentially (using the c api again)

And then:

  • Calculate the mean execution time
  • Plot histogram for CPU time
  • Plot histogram for Disk I/O time
  • Plot a flame graph for executed code
  • Compare process memory size before and after the test.
    We want to make sure there are no memory leaks

add `capacity` to `svm_byte_array`

Motivation

In case the bytes pointer of svm_byte_array points to data of a Vec,
we need in the Rust side to use Vec::from_raw_parts

This is required mainly for managing Receipts.

Since the length of a Vec might be strictly less than its capacity, it's better not to assume anything about the relation of these two fields and add explicit capacity field to the svm_byte_array.

use the BLAKE3 as the default Hash algorithm

Overview / Motivation

Use a fast Hash algorithm in SVM.

The Task

Use BLAKE3 as the default Hash algorithm in SVM.

Implementation Notes

  1. Use the Rust crate:
    https://github.com/BLAKE3-team/BLAKE3

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rustfmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

Integrate `svm-gas` into the flow

Overview / Motivation

We want executed app transactions to the amount of gas that has used.

The Task

We need to use measure the amount of gas being used while executing the app transaction.
If we exceed the maximum amount of given gas (a.k.a the gas limit), we immediately halt executing the app transaction.

Implementation Notes

  1. Pass in SVM exec app API a gas limit
  2. Return gas_used as part of the receipt.
  3. use wasmer metering middleware.

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rustfmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

SVM CLI - Deploy Template

Overview / Motivation

Encoding a deploy template transaction via a CLI will help to test SVM in a black-box manner.

The Task

Implement under the svm-cli crate the functionality of encoding a deploy template transaction.
The output should be saved to file. We'd also like to decode a raw template.

Implementation Notes

  1. For encoding use AppTemplateBuilder:

    pub struct AppTemplateBuilder {

  2. For interpreting a raw deploy template use:

    pub fn parse_template(bytes: &[u8], author: &Address) -> Result<AppTemplate, ParseError> {

  3. Please use the clap crate.

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rustfmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

svm-runtime-c-api: expose build raw transactions API

Overview / Motivation

Expose build raw transactions methods under the svm-runtime-c-api crate that will be used
by used by the go-svm client (and future clients).

The Wallet will dispatch trigger raw transactions by using this flow:

  1. Wallet -- gRPC --> Spacemesh Full-Node
  2. Spacemesh Full-Node --> go-svm
  3. go-svm -- cgo --> svm-runtime-c-api

Contribution Guidelines

Important: Issue assignment to developers will be by the order of their application and proficiency level according to the tasks complexity. We will not assign tasks to developers who haven't introduced themselves on our Gitter dev channel

  1. Introduce yourself on svm dev chat channel - ask our team any question you may have about this task
  2. Fork branch develop to your own repo and work in your repo
  3. The code must be formatted using rustfmt
  4. You must write unit tests for all types and methods when submitting a component.
  5. When ready for code review, submit a PR from your repo back to branch develop
  6. Attach relevant issue to PR

svm-codec: use `Buffer` in the js tests

Motivation

Use Buffer instead of TextEditor for the js tests of svm-codec.
Otherwise, people using older versions of node and npm will get compilations errors.

Windows support

Overview / Motivation

We need to talk with wasmer and see what's the current gap of having wasmer single-pass compiler support Windows.

@avive @moshababo

Get / Set Template API

Motivation

The template raw data that is saved on-chain (inside a deploy-template transaction),
needs to be saved within SVM internal storage too. To avoid this on-disk data-duplication,
we'd like SVM to expose external API to the SVM host (a.k.a Full-Node).

The proposed Get/Set Template API is:

  1. SetTemplate - will receive a blob of data and return a TemplateHash.
  2. GetTemplate - will receive a TemplateHash and return the template blob.

The full flow:

  1. Deploy template - when deploying a new template, the original transaction will contain a slice of data that contains the deploy template blob. This data need to be passed to the Runtime deploy_template

  2. We need to take now the bytes of 1. and call the SetTemplate described above and get TemplateHash back.

  3. The Host will save on-chain the original transaction and replace the deploy template bytes with the TemplateHash as part of the transaction.

  4. When the Host is retrieving transactions (as part of syncing another network peer), he will get to the deploy transaction , grab the TemplateHash and call GetTemplate against SVM in order to fetch the original deploy template data. Then the Host will substitute TemplateHash with deploy template data. Now it'll have the raw data that has been sent on-the-wire originally.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.