Construct a build system similar to https://nx.dev/ or https://hardhat.org/ which covers basic usage of a framework

• create a project: #38
• create a workspace: a project could consist of multiple dapps and each dapp should work in its own workspace: hold on until MVP is ready
• create a config file: #38
• task/plugin system @homura @PainterPuppets #13
  • inject dapp templates
  • #8
  • run tests: #190
  • compile: #180
  • deploy
  • build a docs website for the project: #244

https://github.com/ckb-js/kuai/blob/develop/packages/samples/mvp-dapp/src/main.ts#L35

Running on Node.js 18, the address will be http://:::3000 because an IPv6 address is returned by server.address()

Ref: nodejs/node#40537

We can reference the design of #85

• Message define
• Manager define
• tests

This feature is similar to Exception filters, which filters known errors and wrapped exceptions in the desired format.

Now in the MVP dapp, unknown exceptions, like those thrown from a third-party dependency, will be returned directly, then the response is unpredictable(string, custom structure, etc.). With the filter, all exceptions will be converted into a uniform error format.

At the early stage, we have a MVP DApp(#54) to verify data manipulation on a cell.

Now we are going to support on-chain contracts in kuai, so the MVP DApp should also cover the on-chain verification.

Ref: add contract module in kuai project

CI: https://github.com/ckb-js/kuai/actions/runs/4599392372
Seems that rejected by the server? Need help from @PainterPuppets

1. add a format npm script and run it with staging files on committing;
2. update prettier to remove semicolons.

It's an issue that will achieve #189

• as an actor factor: #4
• default storage layout
• pattern to gather cells
• interfaces
  • new(pattern): fetch cells by the specific pattern and initialize a Store model
  • destroy(): remove data in the Store model
  • duplicate(): create a new Store model from plain cells with the same data
  • sync(blockNumber?): load data with pattern defined on initialization at a specific block number
  • get(path): get a value at the specific path
  • set(path, value): update a value at the specific path
  • delete(path): remove key-value pair at the specific path
• metadata of a storage
  • schema
  • serialization
  • how to share/propagate metadata
• lazy evaluation of updates

1 Overview

• Language

  C, Rust

• Compile

  You can use any tool you like to compile the contract. However, capsule can make deployment easier.

• Deploy & Upgrade

  You can use tools such as capsule and ckb-cli to deploy / upgrade the contract. You can also write your own tools using sdk such as lumos, ckb-sdk-rust, etc.

2 Examples

2.1 Compile

Using capsule:

capsule build

2.2 Deploy & Upgrade

There are several ways to deploy or upgrade contracts.

• capsule [todo]

  • deploy

    capsule deploy ...
    

  • upgrade

• ckb-cli [todo]

  • deploy

    ckb-cli
    wallet transfer ...
    

  • upgrade

• write your own tools using sdk

  • using lumos
    https://github.com/felicityin/ckbox/blob/main/examples/contract-client/contract.ts

  • using ckb-sdk-rust
    https://github.com/felicityin/ckb-contract-rs

If Store's set function can return inputs and outputs, it can't deduct the tx fee when returned. How about letting the tx fee calculate by developers in business code?

On the other hand, shall we let the set function can be calling with chains? Like store.set().set().set()? If so, we maybe need to provide another function to get the inputs and outputs, ex. store.set().set().set().submit()

Can you help me provide any suggestions?

1. Front-end: https://kuai-mvp-dapp-ui.vercel.app/
2. Backend

Backend is under deployment

Design of CKB-independent component: #17

• Add the most basic actor model for over components to interact with: #29
• Add the registry of actor model: #39
• Add message queue: #62

Ref:

• #7

Now the route in the mvp dapp is declared in an imperative way(

It could be adjusted in a declarative way, as nest.js did(https://docs.nestjs.com/controllers#routing). This optimization could be implemented in IO layers

The .bit project currently has four main features

.bit Alias: https://docs.did.id/developers/dotbit-alias
.bit Namespaces: https://docs.did.id/developers/records-key-namespace
.bit schemas: https://github.com/dotbitHQ/das-types/tree/master/schemas
subDID: https://www.did.id/subdid

General docs: https://github.com/dotbitHQ/das-contracts/tree/master/docs

At the stage of MVP, we need to verify whether the capacity abstraction provided by kuai can satisfy the DApp design, i.e., if the developer could add, delete, modify, and read cells of the dapp solely by Store. Thus we choose the namespace feature as the main function of the MVP DApp.

The detail of namespace could be found in .bit document(https://docs.did.id/developers/records-key-namespace), while its schema can be found at https://github.com/dotbitHQ/das-types/blob/master/schemas/cell.mol#L263-L277

flowchart TD
    input([Input]) --> is_store_live{Is store live?}
    is_store_live -->|True| is_reading{Is reading a key?}
    is_store_live -->|False| A(Kuai) --> C[/Activate a store with pattern/] --> is_reading
    is_reading -->|True| reading[/reading the key from the live store/]
    reading --> is_found{Is the key found in the store?}
    is_found -->|True| output([Output])
    is_found -->|False| D[/Activate a store with pattern/] --> E{Is new store activated?}
    E --> |True| reading
    E --> |False| output
    is_reading -->|False| is_adding{Is adding a key?}
    is_adding -->|True| F{Is store capacity enough?}
    F --> |True| add_key[/Add key in the store/]
    F --> |False| G[/Expand the store/]
    G --> H{Is the store expanded?}
    H --> |True| F
    H --> |False| output
    add_key --> output
    is_adding -->|False| is_deleting{Is deleting a key?}
    is_deleting --> |True| I[/Locate a store contains the key, similar to reading a key from store/]
    I --> J[/Clear the specified key/] --> output
    is_deleting --> |False| is_updating{Is updating a key?}
    is_updating --> |True| K{Is store capacity enough?}
    K --> |True| L[/Update the specified key/] --> output
    K --> |False| M[/Expand the store and update the key/] --> output
    is_updating --> |False| output

Please add more info on how to implement the storage component in this issue, e.g. tech stack, schedule, and the PR of adding the database component could be referenced here.

Ref:

• #7
• #16

@kuai is the most ideal scope for the kuai project, with that the cli could be referenced as @kuai/cli.

However, the scope is unavailable on npm registry so we have to pick another one. Now @ckb-js is accessible, how about naming our packages in the pattern @ckb-js/kuai-*(only in package.json) @yanguoyu @felicityin @PainterPuppets

Document: #52

This idea describes a way to store all states off-chain, but as UTxO, developers may have the need to store states directly in cells, such as NFT, which is more suitable for UTxO

We can try to explore some variants of the merkle-like solution, so that states also exist directly in the on-chain as cells

• extends from Contract model
• extends from metadata of Contract model
• interfaces
  • metadata(tokenId, metadata?): a getter/setter to the metadata of a specific token id, including name, symbol, tokenURI, totalSupply
  • mint(tokenId, amount): mint a token
  • send(from, to, tokenId, amount): transfer amount token tokenId from from to to
  • stake(address, dapp, tokenId, amount): stake into a dapp and get staking tokens
  • redeem(address, dapp, tokenId, amount): unstake from a dapp and burn corresponding staking tokens
  • lock(address, tokenId, amount, locktime): lock tokens, for cases like pre-sale, bounty of a team
  • unlock(address, tokenId, amount): unlock tokens
  • combine(token): comebine two Token models into one
  • getBalance(address, tokenId): get token balance of the given address
• token-specific attributes

For merging the cells' data in Store, I think we should consider the following things.

1. Provider default merge functions to merge cells, use the latest cell's data, or merge all cells' data by lodash.merge
2. When calling get, developers can get with merged data or appointed OutputPoint
3. When calling set, developers can merge all cells or update appointed OutputPoint

In my mind, here is my design to achive these:

1. Add a merge cells' data property mergeState in Store

class Store<
  StorageT extends ChainStorage,
  StructSchema extends StorageSchema<GetState<StorageT>>,
  Option = never,
> extends Actor<GetStorageStruct<StructSchema>, MessagePayload<StoreMessage>> {
  protected mergeState: GetStorageStruct<StructSchema>
}

2. Provide a function to merge cells' data that developers can overwrite.

class Store<
  StorageT extends ChainStorage,
  StructSchema extends StorageSchema<GetState<StorageT>>,
  Option = never,
> extends Actor<GetStorageStruct<StructSchema>, MessagePayload<StoreMessage>> {
    useLatest: boolean = false
    mergeCellsData(current: GetStorageStruct<StructSchema>, last: GetStorageStruct<StructSchema>): StructSchema {
        if (this.useLatest) {
           return current
        }
        const customizer = <T>(objValue: T, srcValue: T) => {
          if (Array.isArray(objValue)) {
            return objValue.concat(srcValue)
          }
        }
    
        return mergeWith(current, last, customizer)
    }
}

3. update mergeState when addState or removeState

  private addState({ cell, witness }: UpdateStorageValue) {
    if (this.cellPattern && !this.cellPattern({ cell, witness })) {
      // ignore cells from resource binding if pattern is not matched
      return
    }
    if (cell.outPoint) {
      const outPoint = outPointToOutPointString(cell.outPoint)
      const value = this.deserializeCell({ cell, witness })
      if (this.schemaPattern && !this.schemaPattern(value)) {
        return
      }
      this.states[outPoint] = value
      this.chainData[outPoint] = { cell, witness }
      this.mergeState = this.mergeCellsData(value, this.mergeState)
    }
  }
  private removeState(keys: OutPointString[]) {
    keys.forEach((key) => {
      delete this.states[key]
      delete this.chainData[key]
    })
   Object.values(this.states).reduce((pre, cur) => this.mergeCellsData(pre, cur), {})
  }

4. get and set support calls without delivering OuputPoint

get(paths?: StorePath): GetFullStorageStruct<StructSchema> {
     // get value from this.mergeState
}
set(value: any, paths?: StorePath) {
      if (this.useLatest) {
      // get the latest cell to update and add capacity from old cells
      return
      }
      //if update all, merge all cells to a new cell
      // if updating one key, find the cell and use updated data, if capacity is not enough maybe use other cells.
}

This question comes from https://github.com/ckb-js/kuai/pull/181/files/37a82372b9fe57f1dbde0eda4764f0656d15b217#r1162301306

The schemaOptions in store is not mandatory(https://github.com/ckb-js/kuai/blob/develop/packages/models/src/store/store.ts#L53) and {} will be returned if it's undefined or not an object(https://github.com/ckb-js/kuai/blob/develop/packages/models/src/store/store.ts#L128)

In what case will the schema be empty

Runtime of Kuai has been divided into 5 components(#7) and this issue is to elaborate the architecture and technical design of the Input/Output component, better to include use cases, interfaces, MVP or PoC description, internal architecture, and technical design.

Feel free to add any updates in this issue and the final document will be appended in the top message.

• Concise design doc: #166
• Online demo: https://kuai-mvp-dapp-ui.vercel.app/
• Code Snippets: #166
• Front-end and backend repos:
  • Front-end: https://github.com/Magickbase/kuai-mvp-dapp-ui
  • Back-end: samples/mvp-dapp directory in https://github.com/ckb-js/kuai/

We can reference the design of #2

• Schema define
• JSON serialize and deserialize
• molecule serialize and deserialize. We can use https://github.com/ckb-js/lumos/tree/develop/packages/codec to implement it.
• Store implement
• Pattern define

This issue is from #1 and will add a default compile npm script to build the application. With this instruction, all applications generated by kuai-cli share the same workflow of compilation.

Runtime of Kuai has been divided into 5 components(#7) and this issue is to elaborate the architecture and technical design of the CKB-independent component, better to include use cases, interfaces, MVP or PoC description, internal architecture, and technical design.

Feel free to add any updates in this issue and the final document will be appended in the top message.

We can reference the design of #3

I have some ideas about the error code for mpv or kuai, can we use something like the following to define it? I would like to see what people think.

Application ID (three-digit number) - Module ID (three-digit number) - Error Type (one-letter) - Error Code (five-digit number)

For example: 001-001-E-00001

The meaning of each category is as follows:

Application ID: Indicates which application the error belongs to and can be represented by a three-digit number. For example, 001 represents application MVP1, and 002 represents application MVP2.

Module ID: Indicates which functional module the error belongs to in the application, and can be represented by a three-digit number. For example, 001 represents the login module, and 002 represents the market module.

Error Type: Indicates which type of error the error belongs to and can be represented by a one-letter code. For example, P represents parameter errors, B represents business errors, and O represents other errors.

Error Code: Represents the specific error under the error type and can be represented by a five-digit number. For example, 00001 represents a signature error, and 00002 represents a cell not found error.

The benefit of using this error code is that it can quickly locate errors and display them in a friendly way.

Reference

We can reference the design of #18

• Implement CoR: #53
• implement Listener: #66
• [TBD]Implement a Router like koa-router: #66
• implement adapter for koa: #66
• implement adapter for express
• implement adapter for fastify

Runtime of Kuai has been divided into 5 components(#7) and this issue is to elaborate the architecture and technical design of the Database component, better to include use cases, interfaces, MVP or PoC description, internal architecture, and technical design.

Feel free to add any updates in this issue and the final document will be appended in the top message.

PRD: #54

UI will be a replicated data.did.it working with omnilock

https://data.did.id/magickbase.bit

Now in the mvp dapp, an instance of model is getting by method defined in the controller

It returns an instance if one has been registered or instantiates one if there's no live model.

This logic could be encapsulated in the registry, so users don't have to write it in different dapps repeatedly.

Cell model is UTxO-like, and it is hard for us as application developers to get working with it because of shared state, capacity management, and script combination.

An initial idea is that the Kuai framework provides a runtime that is used to help developers write CRUD-style programs that manage off-chain state and generate proofs to update the merkle roots on-chain, i.e., state is stored off-chain and only one merkle root is stored on-chain.

To implement the feature mentioned above, we first need to do some feasibility study, like

• dApp user identity via P2SH instead of P2PK(H)
• off-chain state transition replayed in the contract without a vm
• cross-dApp (contract) communication without a global sequencer
• and other...

The registry in #29 is roughly implemented with a Map for fast delivery so other developers won't be blocked by this model.

The registry is expected to be done by a container and will be implemented with inversify

• extends from Store model
• default storage layout
• interfaces
  • deploy(): deploy the contract onto the chain
  • extend(): extends from an existing Contract model for overriding
  • upgrade(): upgrade the contract on the chain
  • run(method: string, params: Array): call a method of the contract to update or get its state(structured data)
  • link(interfaces: Array): instantiate a contract SDK from interfaces
• metadata extends from that of Store
  • API

As this issue said, the goal of the database layer is to allow dapps to migrate between databases at a lower cost (dapps won't migrate databases often but it's a prominent characteristic of this layer), so it's better to support multi databases, not just PostgreSql.

In addition, we should not decide which database to use for developers. Developers should decide it according to their own scenarios, because there are many factors that will affect the database selection.

Resource binding service in #7

There is currently some code in kuai-core that is not covered by the tests. More tests need to be added to cover these code

Some things that should be doing:

• add kuai init cli test
• add test coverage report in ci

The design of kuai task system

1. Why we need a task system

As a build system, kuai needs to be able to perform tasks such as compiling, testing, starting nodes, etc. To better manage and expand these tasks, kuai needs a task system. In order to manage and expand these tasks, kuai needs a task system.

2. Design target (What)

To achieve the above goals, the task system needs to have the following features

• dynamically load tasks to facilitate expansion/new custom tasks
• The task runtime needs to be like an onion model to implement pre/post-processing operations when multiple task handlers are used
• has a KuaiContext, which may have config/anything, usually the context is a singleton (providing the ability to create multiple contexts manually)
• task can be executed directly from the cli via npx kuai TASK_NAME
• task can be executed manually in the code to facilitate doing e2e tests
• support subtask

3. How it works

Clarifying how many modules there are and what the system architecture looks like first helps us understand how the system works

• KuaiContext: Load configurations, plugins, tasks, etc. and initialize other modules.
• TaskSystem
  • RuntimeEnvironment: Execution/management of task status, and task lifecycle.
  • Task: Specific logic
  • OverrideTask: extends from Task, for override task
• Kuai CLI: Parsing command line arguments, and providing good cli interaction and output.

  C4Component
  title Architecture of kuai project context
    
  Container(ctx, "Kuai Context", "", "Load configurations, plugins, tasks, etc. and initialize other modules.")
  Container(cli, "CLI", "", "Parsing command line arguments, and providing good cli interaction and output.")

  Container_Boundary(st, "Task system") {
      Component(kre, "Kuai Runtime Environment", "", "Execution/management of task status, and task lifecycle.")
      Component(task, "Task", "", "Specific logic")
      Component(otask, "OverrideTask", "", "Task override")

      Rel(kre, task, "Manage & Run")
      Rel(otask, task, "Override")
  }

  Rel(ctx, kre, "init")
  Rel(cli, ctx, "call")

3.1. KuaiContext, the entry of the kuai

KuaiContext is the entry point for the whole system, responsible for loading config, task, etc... And to prevent multiple KuaiContext at runtime, it should be a singleton pattern, and a new instance can be created by KuaiContext.createKuaiContext().

  classDiagram
  class KuaiContext {
    + RuntimeEnvironment env
    + TaskLoader tasksLoader
    + Extender[] extenders

    + setRuntimeEnvironment(RuntimeEnvironment env)
    createKuaiContext()$
    getKuaiContext()$
    loadConfigAndTasks()$
  }

Possible usage

// entry.ts
import { KuaiContext, RuntimeEnvironment } from 'kuai/core'

const ctx = KuaiContext.createKuaiContext();
const config = loadConfigAndTasks();

const env = new RuntimeEnvironment(
  config,
  ctx.tasksLoader.getTasks(),
  ctx.extenders,
);

ctx.setRuntimeEnvironment(env)

3.2. Task system

classDiagram
  class TaskParam {
    String name
    String type
    String description
    T defaultValue
    Boolean isOptional

    validator(value): Boolean
  }

  class Task {
    String name
    String description
    Record~string, TaskParam~ params
    Boolean isSubTask

    action(args, env, runSuper)
  }

  class OverrideTask {
    Task parentTask
  }

  class RuntimeEnvironment {
    - Extender extenders
    + Record~string, Task~ tasks

    + run(name, args)
  }

  Task --o TaskParam: has
  OverrideTask --|> Task : extends
  RuntimeEnvironment --o Task : has

3.2.1. Task

In the task system, any logic can be a Task, which should have a name as the task identifier, a description , params defined, and a task handler.

In order to implement task override, an OverrideTask is needed, which has an additional property of parentTask compared to Task. When the action of OverrideTask is run, the action of parentTask will be passed as runSuper, so that the override of task can be implemented

3.2.2. RuntimeEnvironment

We now have a simple task definition that allows us to run some logic, but the goal of the task system is to have some simple tasks that combine to accomplish a complex task, so we need a struct that manages all the tasks

RuntimeEnvironment is a module to help us manage the execution of Task and to share the context between Task. When a task is executed, this is passed into the action of task as env so that the context can be shared between tasks

How to load tasks

When KuaiContext calls loadConfigAndTasks to initialize the project, it will run the kuai.config.ts file. you can put the custom task in it, or refer to the custom task's file in kuai.config.ts, so that the task can be injected into KuaiContext. and these injected tasks will be passed as parameters when KuaiContext initializes RuntimeEnvironment.

Some helper functions are also provided in kuai/core to help simplify the code

Possible usage

// kuai/core/helper
function task(name, description) {
  const ctx = KuaiContext.getKuaiContext();
  const loader = ctx.tasksLoader;
  return loader.addTask(name, description)
}

// custom-task.ts

import { task } from 'kuai/core'

task(TASK_NAME)
  .addParam("paramA", "a custom param", 'default value')
  .action(async (args, env, runSuper) => {
  // do something…
  })

// kuai.config.ts

import 'custom-plugin.ts'

When a Task with the same name is registered multiple times, the subsequent Task is loaded as an OverrideTask, and the previous Task is saved as a parentTask

// custom-override.ts

import { task } from 'kuai/core'

task(TASK_NAME)
  .action(async (args, env, runSuper) => {
    // do something before parent task…
    await runSuper()
    // do something after parent task…
  })

How to run task

There is a run method in RuntimeEnvironment, which will find the matched task from tasks, and when running it will check if the Task is an OverrideTask, if it is, it will pass the action of its parentTask as runSuper parameter to the action of the task

How to extends RuntimeEnvironment

Extending RuntimeEnvironment is to add/change some values of RuntimeEnvironment when initializing Kuai.

For example, we add a hi method to RuntimeEnvironment, and then call this hi through the task, so that we can achieve the effect of extending the Kuai.

Possible usage

// kuai/core/helper
extendsRuntimeEnvironment(extender: (env: RuntimeEnvironment) => void) {
  const ctx = KuaiContext.getKuaiContext();
  ctx.extenders.push(extender);
}

// kuai.config.ts / custom-plugin.ts
import { extendsRuntimeEnvironment } from 'kuai/core'

extendsRuntimeEnvironment((env) => {
  env.hi = () => {
    console.log('hi')
  }
})

task("hi", async (args, env) => {
  env.hi()
});

Will yield:

$ npx kuai hi
hi

Using TypeScript

// custom-type.ts
import "kuai/types/runtime";

declare module "kuai/types/runtime" {
  export interface RuntimeEnvironment {
    hi: () => void;
  }
}

3.3. CLI

CLI facilitates users to run some scripts and functions in kuai environment, such as test / start node / deploy contract etc.

The cli is mainly implemented by commander.js, and the way to use cli in kuai is like this

$ npx kuai help        
Kuai version 0.0.1

Usage: kuai [GLOBAL OPTIONS] <TASK> [TASK OPTIONS]

GLOBAL OPTIONS:
  -h, --help     Show help
  -v, --version  Show version number

AVAILABLE TASKS:
  help                  Prints this message
  node                  Starts a JSON-RPC ckb server
  test                  Run tests

In this article we focus on how the task system will integrate with cli

Our task itself has a name and we want to be able to run the task by typing kuai TASK_NAME directly, so this requires the cli module to dynamically load commands based on the task

The name of the Task will be used as the name of the command, and the params will be parsed as options

Possible usage

// kuai/core/cli.ts
import { Command } from 'commander';
import { loadCommandFromTasks } from './helper';


const program = new Command();

const ctx = KuaiContext.getKuaiContext();
const commands = loadCommandFromTasks(ctx.tasksLoader.getTasks());

commands.forEach((command) => {
  program.addCommand(command);
});
program.parse();

From the tsconfig.json we can see that output directory is set to dist(https://github.com/ckb-js/kuai/blob/develop/packages/samples/mvp-dapp/tsconfig.json#L6), which is listed in .gitignore. While there's an empty directory named build(https://github.com/ckb-js/kuai/tree/develop/packages/samples/mvp-dapp/build) reserved in samples/mvp-dapp. My question is that what's the difference between build and dist, and why is dist ignored in git(https://github.com/ckb-js/kuai/blob/develop/packages/samples/mvp-dapp/.gitignore#L161) but build is not.

Same question to the libs empty directory(https://github.com/ckb-js/kuai/tree/develop/packages/samples/mvp-dapp/libs)

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• chore(deps): update dependency @types/dockerode to v3.3.29
• chore(deps): update dependency husky to v9.0.11
• chore(deps): update dependency ts-jest to v29.1.5
• fix(deps): update dependency koa to v2.15.3
• fix(deps): update dependency path-to-regexp to v6.2.2
• fix(deps): update dependency reflect-metadata to v0.2.2
• fix(deps): update dependency tslib to v2.6.3
• fix(deps): update rust crate serde to v1.0.203
• fix(deps): update rust crate serde_json to v1.0.120
• chore(deps): update dependency @ckb-lumos/rpc to v0.23.0
• chore(deps): update dependency @types/lodash to v4.17.6
• chore(deps): update dependency eslint to v8.57.0
• chore(deps): update dependency prettier to v3.3.2
• chore(deps): update dependency typescript to v5.5.3
• chore(deps): update nervos/ckb docker tag to v0.116.1
• fix(deps): update dependency @ckb-lumos/base to v0.23.0
• fix(deps): update dependency @ckb-lumos/bi to v0.23.0
• fix(deps): update dependency @ckb-lumos/codec to v0.23.0
• fix(deps): update dependency @ckb-lumos/config-manager to v0.23.0
• fix(deps): update dependency @ckb-lumos/experiment-tx-assembler to v0.23.0
• fix(deps): update dependency @ckb-lumos/hd to v0.23.0
• fix(deps): update dependency @ckb-lumos/helpers to v0.23.0
• fix(deps): update dependency @ckb-lumos/lumos to v0.23.0
• fix(deps): update dependency ioredis to v5.4.1
• fix(deps): update dependency jsonrepair to v3.8.0
• fix(deps): update dependency pg to v8.12.0
• fix(deps): update rust crate molecule to 0.8.0
• chore(deps): update dependency eslint to v9
• chore(deps): update docker/build-push-action action to v6
• fix(deps): update dependency commander to v12
• fix(deps): update dependency path-to-regexp to v7
• 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update actions/checkout digest to 692973e
• chore(deps): update dependency lint-staged to v15.2.7
• chore(deps): update dependency lerna to v8.1.5
• chore(deps): update dependency typedoc to v0.26.3
• chore(deps): update typescript-eslint monorepo to v6.21.0 (@typescript-eslint/eslint-plugin, @typescript-eslint/parser)
• fix(deps): update dependency dotenv to v16.4.5
• fix(deps): update rust crate ckb-std to 0.15.0
• chore(deps): update typescript-eslint monorepo to v7 (major) (@typescript-eslint/eslint-plugin, @typescript-eslint/parser)
• fix(deps): update dependency chalk to v5
• fix(deps): update dependency find-up to v7
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• fix(deps): update dependency env-paths to v3

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

An actor in actor model is the most basic computing unit to run business logic insulated. It is self-contained that receives messages and processes them sequentially.

Actor runtime is the moderator which decides how, when, where each actor runs, and routes messages transported in the actor system.

Under the control of actor runtime, a large number of actors can perform simultaneously and independently to achieve parallel computation.

Kuai provides the basic implementation of an actor for developers to extend business logic and dominates these actors with the actor runtime for scalability and reliability.

1. Actor Runtime

1.1 Supervision

The main idea delivered by the actor model is Divide-and-Conquer. If a heavy task is assigned to actor_a, the task would be split into sub-tasks and delegated to actor_a's child actors, recursively, until sub-tasks are small enough to be handled in one piece.

Within the runtime, an actor will be activated(created) or deactivated(destroyed) automatically so developers don't have to pay attention to the lifecycle of a single actor. Once a message is sent to an actor located by identity, the actor runtime will activate one to handle the message. And if an actor has been idle for a while(no messages, no internal state), the actor runtime will deactivate it. In some cases, which we will mention later, an actor should be kept alive, it could send messages to itself periodically as a reminder.

To achieve this, a supervision tree is going to be introduced because actor model has a tree-like hierarchical structure. More about supervision tree could be learned from supervision principles

1.2 Router

Actors communicate with each other exclusively by sending messages to targets while targets are not referenced directly, instead, messages are routed by the runtime. With this, actors are decoupled and their lifecycles could be taken over by the runtime.

To deliver a message, the recipient should have an identity that is transparent to the sender. For simplicity, address is used as the identity in Kuai runtime.

There're several ways to get a recipient's address. The simplest way is to register the newly activated actor in a registry. A registry is a local, decentralized, and scalable key-value address storage. It allows an actor to look up one or more actors with a given key. The actors registered in the registry will be managed by the runtime under different supervision/monitoring strategies(strategies could be found in supervision principles too)

The other way is more modular, an actor will only send messages to addresses it received, e.g. a parent spawns a child actor and gets a response of the child actor's address, with the response, the parent could start communicating with its child actor.

2. Actor

2.1 State

/*
 * field decorated by @State will be registered as an internal state
 */
class CustomActor extends Actor<CustomState> {
  @State()
  state_a: CustomState['state_a']
}

2.2 Behaviour

type Status = ok | error | continue | timeout | stop

class Actor<State, Message> {
  
  /*
   * functions to send messages
   */

  // make a synchronous call
  function call (
    address: ActorAddress,
    message: CustomMessage,
    timeout?: number
  ): {
    status: Status,
    message: CustomMessage,
  }

  // make an asynchronous call
  function cast (
    address: ActorAddress,
    message: CustomMessage,
  ): void

  // send message synchronized to named actors running in specified nodes
  function multi_call (
    nodes: Array<Node>,
    name: string, // registered actor name
    message: CustomMessage,
  ): {
    responses: Array<{
      status: Status,
      message: CustomMessage,
    }>
  }

  // broadcast messages to named actors running in specified nodes
  function abcast (
    nodes: Array<Node>,
    name: string, // registered actor name
    message: CustomMessage,
  ): void

  // reply to the sender in a synchronized call
  function reply (
    client: ActorAddress,
    message: CustomMessage,
  ): void

  /*
   * functions for lifecycle
   */

  // start an actor outside the supervision tree
  function start (
    actorConstructor: ActorConstructor,
    init: CustomState,
    options?: Record<string, string>
  ): {
    status: Status,
    address: ActorAddress,
  }

  // start an actor within the supervision tree
  function startLink (
    actorConstructor: ActorConstructor,
    init: CustomState,
    options?: Record<string, string> 
  ): {
    status: Status,
    address: ActorAddress,
  }

  // stop a actor
  function stop (
    actor: ActorAddress,
    reason: string,
  ): void

}

class CustomActor extends Actor<CustomState, CustomMessage>{

  /*
   * callback behaviors are injected by decorators, and matched by Symbols
   */

  // invoked when the actor is activated
  @Init()
  constructor (
    init: CustomState
  ): { 
    status: Status,
    state: CustomState,
  }

  // invoked when the actor is deactivated
  @Terminate()
  function (
    status: stop,
    reason: string,
    state: CustomState,
  )

  // invoked to handle synchronous call messages, the sender will wait for the response
  @HandleCall(pattern: Symbol)
  function (
    message: CustomMessage,
    from: ActorAddress,
    state: CustomState
  ): {
    status: Status,
    message: CustomMessage,
    state: CustomState,
  }

  // invoked to handle asynchronous call messages, the send won't wait for the response
  @HandleCast(pattern: Symbol)
  function (
    message: CustomMessage,
    state: CustomState,
  ): {
    status: Status,
    state: CustomState,
  }

  // invoked to handle `continue` instruction returned by the previous call
  @HandleContinue()
  function (
    message: CustomMessage,
    state: CustomState,
  }: {
    status: Status,
    state: CustomState,
  }

  // invoked to handle all other unmatched messages
  @HandleInfo()
  function (
    message: any,
    state: CustomState,
  ): {
    status: Status,
    state: CustomState,
  }

  // invoked to inspect internal state of the actor
  @FormatStatus()
  function (
    reason: string,
    state: CustomState,
    context: Context,
  ): void
}

The state field in the parameters is the state of actor before the action and the state field returned by callback is the state of actor after the action(framework will update the internal state of the actor, quite similar to useState hook in React: useAction(preState => curState)).

2.3 Mailbox

// TODO:

3. Basic Models

3.1 Registry

It's actually a Store model we will design later.

3.2 Supervisor

interface ActorSpec {
  name?: Symbol
  id: Symbol
  address: ActorAddress
  strategy: SupervistionStrategy
}

class Supervisor extends Actor {
  @State()
  actors: Map<ActorSpec, Actor>

  constructor (
    children: Array<{
      child: ActorConstructor,
      spec: Pick<ActorSpec, 'strategy'>
    }>,
    options: Record<string, string>,
  ): {
    status: Status
    children: Array<ActorSpec>
  }

  // stop the supervisor
  function stop (
    reason: string,
  )
  
  // add actor children
  function startLink (
    children: Array<{
      child: ActorConstructor, 
      spec: Pick<ActorSpec, 'strategy'>
    }>,
    options: Record<string, string>
  ): {
    status: Status,
    children: Array<ActorSpec>,
  }

  // add a single child
  function startChild (
    actorConstructor: ActorConstructor,
    spec: Pick<ActorSpec, 'strategy'>,
  ): {
    status: Status,
    child: ActorSpec,
  }
    

  // restart a child spec which has been terminated
  function restartChild(
    spec: ActorSpec,
  ): {
    status: Status,
  }

  // delete a child spec which has been terminated
  function deleteChild (
    spec: ActorSpec
  ): {
    status: Status,
  }

  // terminate but keep the child spec
  function terminateChild (
    spec: ActorSpec
  ): {
    status: Status,
  }

}

3.3 Store

// TODO: Design Store model

3.4 Contract

// TODO: Design contract model

3.5 Token

// TODO: Design token model

PRD: #54

The message queue in #29 is roughly implemented with a Map for fast delivery so other modules won't be blocked.

It's expected to be done with redis stream as designed in #17

Architecture of Kuai runtime at the components level

    C4Context
      title Architecture of Kuai runtime

      Container_Boundary(b0, "Kuai runtime") {


        Boundary(io_comp, "Input/Output", "Component") {
            System(ckb, "CKB Node", "Chain data")
            System(input, "input", "Requests from users/dapps")
            System(output, "output", "Responses to users/dapps")
            System(main_entry, "main_entry", "Entry of DApp that launches components")

            Rel(input, main_entry, "")
            Rel(output, main_entry, "")
            Rel(main_entry, token, "manage")
            Rel(main_entry, contract, "manage")
            Rel(ckb, resource_binding, "bind")
        }

        Boundary(ckb_comp, "CKB related", "Component") {
          Boundary(b5, "CKB models", "System") {
            Component(store, "Store model")
            Component(contract, "Contract model")
            Component(token, "Token model")

            Rel(token, contract, "extend")
            Rel(contract, store, "extend")
            Rel(store, actor_models, "extend")

          }
          System(resource_binding, "Resource binding", "Trigger state update by on-chain transactions")
          Rel(resource_binding, store, "trigger")
        }

        Boundary(basic_comp, "CKB-independent", "Component") {
          System(actor_models, "Actors", "Encapsulate code and data in a reusable actor as the most basic computation unit")
          System(secrets, "Secrets", "Limit access to secrets, adopt secret scope policy with restriction permission")
          System(configuration, "Configuration")      
          System(observability, "Observability", "Metrics, logs, and data tracing")   
    
          Rel(secrets, actor_models, "implement")
          Rel(configuration, actor_models, "implement")
          Rel(observability, actor_models, "extends")
        }

        Boundary(db_comp, "Database", "Component") {
          System(storage, "Storage", "With pluggable DBMS for storing and querying data")
          Rel(storage, store, "data query and update")
        }

        Boundary(external_services, "External Services", "Component") {
          SystemDb_Ext(database, "DBMS")
          System_Ext(secrets_manager, "Secret manager")
          System_Ext(config_manager, "Configuration manager")
          System_Ext(metrics, "Metrics service")
          System_Ext(logs, "Logs service")
          System_Ext(health_check, "Health check service")

          Rel(database, storage, "connect")
          Rel(secrets_manager, secrets, "")
          Rel(config_manager, configuration, "")
          Rel(metrics, observability, "")
          Rel(logs, observability, "")
          Rel(health_check, observability, "")
        }

        UpdateLayoutConfig($c4ShapeInRow="10", $c4BoundaryInRow="1")
      }

There're 5 main components running in runtime

• Input/Output Component: there're only two directions of data flow on CKB: chain -> user and user -> chain, so they are grouped together as an IO component. This component is to handle messages/data from the two endpoints.

• CKB Related Component: this component is a group of dapp business rules. There're 3 basic abstractions of CKB's unit, store, contract, and token to be extended for dapp specific rules. Each model extended from the basic abstraction could be regarded as a use case.

• CKB-independent Component: this component provides general functions of a framework, especially those that are not coupled with CKB cell models. From this perspective, the dapp could be platform-agnostic.

• Database Component: database component is outlined for flexibility, with this component, state storage could be decoupled from an explicit database.

• External Services Component: some third-party services could be used to enhance functions of kuai.

Description

As a dApp user, I want the local network to be simple enough to use out of the box.

We can take a cue from hardhat node

> npx hardhat node
Started HTTP and WebSocket JSON-RPC server at http://127.0.0.1:8545/

Accounts
========

WARNING: These accounts, and their private keys, are publicly known.
Any funds sent to them on Mainnet or any other live network WILL BE LOST.

Account #0: 0xf39fd6e51aad88f6f4ce6ab8827279cfffb92266 (10000 ETH)
Private Key: 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80

Account #1: 0x70997970c51812dc3a010c7d01b50e0d17dc79c8 (10000 ETH)
Private Key: 0x59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d

Account #2: 0x3c44cdddb6a900fa2b585dd299e03d12fa4293bc (10000 ETH)
Private Key: 0x5de4111afa1a4b94908f83103eb1f1706367c2e68ca870fc3fb9a804cdab365a

1. out of box, no config required
2. 20 key pairs with assets (10000 ETH) are provided by default

Since CKB does not have a tool like Ganache (Tippy is written in C# and it is outdated) now, we can use CKB with a default config for the testing purpose as local network at first

Kuai is likely to write contracts in C/Rust in the long term, so we can't live without the RISC-V toolchain, which means we will be depending on Docker, so the local network can also depend on Docker directly, which will require Docker on the developer's machine

Reference

• Docker image
• dev config

The MVP DApp is expected to be work with MetaMask at this early stage.

Meanwhile, a browser extension wallet of Nervos CKB, named Nexus(https://github.com/ckb-js/nexus) is under development.

Once Nexus is ready for tests, we will try to support it in the MVP DApp.

As the proposal said

Note that a Store model could be a group of cells matching the same pattern and working as an entity, so it could be regarded as a virtual DApp. Say we have 2 DApps in School Roster Store models, each of them consists of many Student Store models. And we are going to build a Sports Meeting DApp, a new Sports Meeting Store model could be created and consist of partial Student Store from School A and B, respectively, it should work as same as a School Roster DApp.

A store model should have a pattern that is used to bind to specific cells. It's defined in the architecture as Resource binding between CKB Node and Store model.

With the binding, a store model maps itself to specific cells and parses data from those cells into a structured data model as the state of a dapp.

Any updates on the chain should be delivered to the store model and keep its state up with on-chain data.

Please add the technical design of resource binding between store and ckb node in this issue, and then we can make a plan for the implementation.