gear-tech / gear Goto Github PK

RPC should do the following:

1. Send message to the particular program with some huge gas
2. Wait for reply
3. See how much gas left in the reply message
4. Figure out how much actual gas is needed to process a message and report it as a result

Database state should not change (since no actual transactions are involved in the process)

Given we have syscalls to write message by parts, we can write something like Vec<impl codec::Encde> by parts

cc @breathx

https://docs.rs/parity-scale-codec/2.2.0/parity_scale_codec/trait.Output.html

RPC tests are disabled at the moment.

• should verify log and program memory: #135
• should never hang: #73
• should suppress node log and show it in case of error: #74
• should show pretty diagnostic
• #353

To use the same sophisticated runner as gtest

Use isolated heap memory instead of shared memory.
This change will transfer the memory limit (4GB) from the entire runtime to a separate program, also allowing parallelism to be used in the future.
The state of the memory heap must be preserved as well as static memory and must be accessible from other programs via API.

• Heap memory management. (89d9952)
• Use one memory for each program. (#41)
• Implement memory access API.

There should be none except which will never fire. The reason why it will never fire must be explained in the .expect argument.

Allocator should utilize static area to store configuration and free even first page after no more memory in heap required.

Problem to Solve
Custom payload in tests.

Possible Solution
Parse metadata from .meta.wasm and encode/decode data provided from .yaml with SCALE.

Notes
Something like this:

title: meta

programs:
  - id: 1
    path: examples/target/wasm32-unknown-unknown/release/demo_meta.wasm
    meta: examples/target/wasm32-unknown-unknown/release/demo_meta.meta.wasm
    init_message:
      kind: custom
      value:
        value: 1
        annotation: "hello"

fixtures:
  - title: meta
    messages:
      - destination: 1
        payload:
          kind: custom
          value:
            value: 1
            annotation: "hello"
    expected:
      - messages:
          - destination: 1
            payload:
              kind: custom
              value:
                old_value: 0
                new_value: 1

Problem to Solve

The smart contract should have the possibility to break execution and reschedule the original message processing.

Possible Solution

Add gr_wait() syscall that should:

• Break the current smart contract execution with a custom trap (or something similar).
• Push back the original message to the end of the queue.
• Take into account the gas spent by program and update gas limit (or add gas spent field).

File Location
src/lib.rs

Proposal
It is better to use a newtype pattern for gas limit in send function:

pub struct Gas(u64);

pub fn send(program: ProgramId, payload: &[u8], gas_limit: Gas, value: u128) {
    ...
}

Using:

msg::send(0.into(), "payload", Gas(1000000000), 0);

This avoids confusing gas_limit and value which can be quite dramatic.

Further improvement is to add gas! macro that can accept multiples values instead of counting zeroes:

assert_eq!(gas!(1 K),   Gas(1000));
assert_eq!(gas!(100 K), Gas(100_000));
assert_eq!(gas!(1 M),   Gas(1000_000));
assert_eq!(gas!(10 M),  Gas(10_000_000));
assert_eq!(gas!(1 G),   Gas(1000_000_000));
assert_eq!(gas!(1 T),   Gas(1000_000_000_000));

Should cover every branch with mocked gear-core, as it a critical runtime code

Should cover:

• Ability or inability to pay for submitting programs
• Ability or inability to pay for submitting messages
• Chain gas limit and correspondent message processing capacity

Problem to Solve

Need to create an ability to push extra data into reply's payload to make api more flexible.

Possible Solution

• Implement the feature into MessageContext from core/message.rs
• Implement the feature into core-runner
• Implement the feature into wasmi and wasmtime from core-backend

Notes

Also need to add the feature usage into tests

• Update core/message.rs unit tests

• Upgradable schedule of particular costs
• Rent fee of memory usage
• Benchmarking

Using custom executor and waker, we can achieve something like this in contracts:

fn handle() {
  block_on(handle_async);
}

fn generate_reply() { .. }

async fn handle_async() {
  let next_message = msg::next().await;
  
  let reply = msg::send_with_reply().await;
  
  return generate_reply(next_message, reply);
}

Single send

async fn handle_async() {
     let another_reply = msg::send_and_wait_for_reply(0.into(), b"async_result", u64::MAX, 0).await;
     ...
     msg::reply(...)
}

Multiple send

async fn handle_async() {
    let r1 = msg::send_and_wait(1, b"PING", ...).await; // If `await` it will wait. If no, we need to join?
    let r2 = msg::send_and_wait(2, b"PING", ...).await;
    let r3 = msg::send_and_wait(3, b"PING", ...).await;
    if r1 == b"PONG" && r2 == b"PONG" && r3 == b"PONG" {
        msg::reply("SUCCESS", ...);
    } else {
       msg::reply("FAIL", ... );
    }
}

• Add a simple executor.
• Add async example.
• Add send_and_wait_for_reply function.

Problem to Solve

JSON is quite difficult to edit by humans.

Possible Solution

Add support for YAML (or even TOML) configurations for test smart contracts. These configurations may be distinguishable by the file extension (.json vs .yaml/.yml or .toml).

Notes

YAML can be quite simply and univocally converted to JSON. TOML is insensitive to the indents.

gear-core should be available in no-std context and put inside wasm runtime blob. The less logic we put outside of wasm blob the better. Functions called inside guest code should reenter wasm runtime blob or even dynamically linked to it (run in the same wasm Store.

Since it is intended to be used in browser context, would be great to test it on demo-meta example to see if it returns metadata expected

This would be also be an example of how to use it.

• Every message on the wait list is available to any program at the moment.
• Need to split the wait list by program ID for addressing only by the dedicated program.

File Location(s)

gear-core/gear-backend/src/wasmtime/env.rs

Proposal

• Rethink variable names (first of all, resolve ambiguity in value and value_ptr)
• Use uniform casting rules (for example, cast every number into i32 only)
• Improve code readability by using Result::{map, map_err} constructions

• only authorities can submit programs
• initially only authorities can send messages
• authorities can have ACL of who can send messages to what programs

Problem
If incorrect program bytes presented in initialisation then node can't construct MessageError.

Steps

1. Upload random bytes as a program.
2. Try to get info about the next block.

Unable to retrieve the specified block details. Unable to decode storage system.events:: createType(Vec<EventRecord>):: decodeU8a: failed at 0x00010000000a0268… on : {"phase":"Phase","event":"Event","topics":"Vec<Hash>"}:: decodeU8a: failed at 0x0a0268e3d3875cef… on event: {"index":"EventId","data":"Null"}:: Struct: failed on data: :: decodeU8a: failed at 0x010000010000000a… on :: Cannot construct unknown type MessageError

Those are fuzz-like tests that check invariants like

1. Gas+balance is never lost
2. Gas+balance is never created

Possible flows emulated by the fuzzying framework should include at least the following:

• Simple flow built around submitting programs through submit_program() and triggering messages processing (including sending outgoing messages to programs and random addresses and generating traps) by sending a number of send_message() extrinsics distributed across a range of blocks;
• Flow with users replies;
• Flow with mailbox provessing;
• Flow with wait list rent collection by external players sending arbitrary inputs.

UPD by @SabaunT

• Gas tree invariants to be fuzzed: #82 (comment)

Proposal

The demo should reflect the scenario of interaction with the reply logic.

File Location
src/lib.rs

Proposal
We have one function for message sending with a quite long and non-obvious signature:

pub fn send(program: ProgramId, payload: &[u8], gas_limit: u64, value: u128);

According to use cases, the value is often unused and set to 0.

The good idea is to have two available functions:

pub fn send(program: ProgramId, payload: &[u8], gas_limit: u64) {
    send_with_value(program, payload, gas_limit, 0);
}

pub fn send_with_value(program: ProgramId, payload: &[u8], gas_limit: u64, value: u128);

This will make using send a little bit easier and convenient. Also, it will simplify the contract of a function.

Reply can be either (1) Regular reply (2) Trap

Trap can be automatically generated by embedder when message processing panicked or deliberately returned by abort syscall

cargo +nightly test -p tests-btree

running 3 tests
test tests::binary_available ... ok
test tests::program_can_be_initialized ... FAILED
test tests::simple ... FAILED

failures:

---- tests::program_can_be_initialized stdout ----
thread 'tests::program_can_be_initialized' panicked at 'failed to init program: Error loading program: can't find memory export', tests/btree/src/lib.rs:139:14

---- tests::simple stdout ----
thread 'tests::simple' panicked at 'failed to init program: Error loading program: can't find memory export', /Users/nikolayvolf/git/gear/tests/common/src/lib.rs:26:10
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

failures:
tests::program_can_be_initialized
tests::simple

test result: FAILED. 1 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

Problem to Solve

We don't have a full vision of desired API available for smart contracts.

Possible Solution

Add a table to README.md with functions signatures and a short description. Later we will add using examples that will be moved to the docs.

The table may look like this (in alphabetical order):

We will add issues and track the implementation of every new feature.

Like we do return meta information on what input and output of the program is, we can introduce couple of more metas

• meta_state_type that will return json-type of the meta_state
• meta_state that will return scale-encoded payload that should be interpreted as type returned by meta_state_type

To actually run meta_state, memory for the program program.meta.wasm should be extracted from the current chain.

Problem to Solve

Currently program state is saved regardless of execution result. It makes possible following unwanted conditions:

• Allocated memory pages may not be freed upon exit (#80)
• Program may end up in corrupted state

Possible Solution

Prevent saving program state, outgoing and incoming messages if trap is produced. Another option is to provide default panic handler that would roll back state. Users will be able to opt-out from it and preserve state on trap.

Notes

Following cases have to be considered:

• Should gas be deduced for unsuccessful executions? If so, how much? If not, could it be a potential DOS vulnerability?
• Program must not loose any incoming messages if state was not preserved. How should infinite loop of handling incoming message and producing a trap be prevented?
• Mechanism to recover from such errors should be present. E.g. updating submitted code with preserved state and/or messages.

Initially, program is instantiated in "uninitialized" state. Until it achieves "initialized" state, all messages to it should not be handled.

Program can achieve "initialized" state, once it replies to the initialization message with non-trap reply

see #79

To properly allow to reply messages that were addressed to the user, runtime must keep track of what messages were destined for the user

The mailbox enumerates, per user, all unreplied messages created

That would be great:

   let mut program_2_id: ProgramId;
   let mut program_1_id: ProgramId;
   let runner = runner_builder()
      .program(wasm_code()).init(Data { a: 2, b: 4 }).build(|p| program_1_id = p.id)
      .program(wasm_code()).init(Data { a: 9, b: 1 }).build(|p| program_2_id = p.id)
      .build();

  /// run some message asserts on runner

Problem to Solve

Implement functions of sending messages and replies by parts in gstd.

Possible Solution

Add new functionality in mod sys in gstd/msg.rs

Notes

It is better to do this as soon as the #36 is closed

Problem to Solve

Currently programs can share the same code because the same code would produce the same key in the key-value store.
In case all programs referencing a piece of code are removed, the code itself should be cleared from storage, as well.

Possible Solution

Use reference counting pointers.

Notes

No response

Once reply messages done, program initialisation should also be effective only once the initialisation message got replied

Errors should be common for Ext and WASM traps with exit codes.

Should be exponential (quadratic?) of total allocations

If there were a message destined to the particular user, it can send a reply to this message

Something along this lines should be good to go:

gear login <private_key/uri>
gear deploy <wasm or path to Cargo.toml>