Description

Unit tests help when code needs to be refactored. When new changes are introduced unit tests provide assurance that refactored code still works correctly.

• discover untested corner cases in the code and implement unit tests for them

Acceptance Criteria

• new unit tests are implemented and they successfully pass

Expected behavior

Closing DB should be completely close the DB without leaving the trace. (lock file or open handles)

Actual behavior

After closing the DB, it leaves the LOCK file and it cannot re-opened

Steps to reproduce

1. create DB
2. close DB
3. check the folder

Which version(s) does this affect? (Environment, OS, etc...)

0.3.0-debug.20-94522fbd51ec040e1040cdccd1b0572d805a2825

last working version checked was 52ffbeac2c1b0b188e8df995a1dc6a1634426a27

Description

• iterative functions have better execution performance
• easier to find an opportunity for their optimization and possible parallelism

Acceptance Criteria

• there is no recursive functions in a code

Additional Information

• only perform this action on a places in the code where and if it makes sense

Description

To check coding format style a rustfmt will be used in a project.

• we will follow diem coding guideline
• rustfmt.toml file needs to be created with following attributes:

edition = "2021"
comment_width = 120
wrap_comments = true
format_code_in_doc_comments = true
normalize_comments = true
normalize_doc_attributes = true
max_width = 120
group_imports = "StdExternalCrate"
imports_granularity = "Crate"

• GitHub action needs to be created to check code formatting when a PR is created and when a PR is merged

Acceptance Criteria

• rustfmt.toml file exists in project's root directory
• GitHub action is created

Expected behavior

With long running state DB, memory usage should not bloat.

Actual behavior

while running the state_long.js, check the process memory usage (using instruments leak option for Mac), it detects the memory leak, on StateWriter::cache_new and the memory usage constantly increase

Steps to reproduce

Run the state_long.js and check the memory usage

Which version(s) does this affect? (Environment, OS, etc...)

0.3.2

Expected behavior

When running unit tests in report_misbehavior_transaction.spec.ts in lisk-sdk, there should be no Jest warnings regarding open handle for database connection.

Actual behavior

After running unit tests, Jest reports a warning regarding open handles for database connection:

Jest has detected the following 1 open handle potentially keeping Jest from exiting:

  ●  neon threadsafe function

      159 |
      160 |     removeDB(systemDir.data);
    > 161 |     const moduleDB = new Database(path.join(systemDir.data, 'module.db'));
          |                      ^
      162 |     const stateDB = new StateDB(path.join(systemDir.data, 'state.db'));
      163 |
      164 |     const validatorsModule = new ValidatorsModule();

      at new Database (../node_modules/@liskhq/lisk-db/database.js:85:20)
      at Object.getBlockProcessingEnv (src/testing/block_processing_env.ts:161:19)
      at Object.<anonymous> (test/integration/node/processor/report_misbehavior_transaction.spec.ts:35:30)

Steps to reproduce

In lisk-sdk/framework directory execute:
$ yarn test:integration report --detectOpenHandles

Which version(s) does this affect? (Environment, OS, etc...)

Current main branch.

Description

Update the build to support node 18 as well since node 16 will be finishing the LTS in 2023.
However, since Lisk core v3 still uses node 16, we should keep the node 16 support.

Description

• Explain how to use the library
• Update license year

Motivation

Readme should describe basic interface and how to use them

Expected behavior

When calling StateDB range, it should never expose the internal prefix

Actual behavior

In some case, range returns state db prefix 00

Steps to reproduce

Call range with key in cache/out of cache/deleted

Which version(s) does this affect? (Environment, OS, etc...)

0.3.1

Description

• currently we have some identical code blocks on a different places in the code
• duplicated code tends to be more error prone
• possible solution is to replace duplicated code with a function call

Acceptance Criteria

• there is no duplicated code in a project

Additional Information

• only perform this action on a places in the code where and if it makes sense

Motivation

• In order to use more feature from rocksdb, new native binding was required
• Sparse merkle tree calculation requires good performance, which cannot be achieved by node.js

Tasks

• #2
• #3
• #9
• #10
• #12

Description

• when calling Rust bindings from JavaScript code we get some execution performance penalty due to an additional code which needs to be executed because of a bridge between two different programming languages
• A good example are functions StateReadWriter.set() and StateReadWriter.get() in state_db.js file. In these two functions there are multiple Rust binding calls. Because these two functions are heavily used, overall code execution performance is slow because of a lot of bridge calls between JavaScript and Rust code.
• develop an idea how such JavaScript functions could have less Rust function calls

Acceptance Criteria

• JavaScript code has less Rust function calls

Additional Information

• one idea is that JavaScript part of lisk-db will have it's own temporary database/cache to store and read data inside it and only call Rust code in case that expected data won't be present in its database/cache

Description

In order to implement snapshot/restore using InMemoryDatabase, it is required to have clone method on InMemoryDatabase
It is mainly used for mocking the stateDB behavior using InMemoryDatabase.

Acceptance Criteria

• Implement test in Node.js to show cloned instance does not affect the original data
• • Implement test in Node.js to show cloned instance does get affected by the original data

Description

• Create a github action to publish npm package automatically
  • when merged to main, it should be able to publish to the npm.lisk.com registry

Acceptance Criteria

• Built package should include binary for all the platforms

Additional Information

Example:

• https://github.com/IronCoreLabs/recrypt-node-binding

Description

GitHub action needs to be created to check Rust compiler warnings when a PR is created and when a PR is merged.

Acceptance Criteria

• GitHub action is created
• source code in a repository is compiled without any compiler warning

Description

Add smt.verify function for both SparseMerkleTree and SMTDB

Acceptance Criteria

• Add tests using fixtures from lisk-sdk

Description

in lisk-sdk use-case, state execution only happens once in batch, so the transactional operation is not critical.
However, if execution and read from another process happens at the same time, it might return in consistent data.

Therefore, when creating StateReader or StateReadWriter, it should create rocksdb snapshot and use that for reading data

Acceptance criteria

• Add test where read and write happen simultaneously
• All other test should pass

Description

Implement 2 different kinds of databases. Append or remove only data (Block) and mutating data with possibility of rollback (states).

Both are supported by native binding with Rust implementation using “neon” framework.

Database

Properties

• Batch write
• get/set/del

Interface

interface DBOptions {
    readonly gte?: Buffer;
    readonly lte?: Buffer;
    readonly reverse?: boolean;
    readonly limit?: number;
}

interface Batch {
    set(key: Buffer, value: Buffer): void;
    del(key: Buffer): void;
}

interface DBReader {
    get(key: Buffer): Promise<Buffer>;
    has(key: Buffer): Promise<boolean>;
    createReadStream(options?: DBOptions): NodeJS.ReadableStream;
    close(): void;
}

interface Database {
    newReader(): DBReader;
    get(key: Buffer): Promise<Buffer>;
    has(key: Buffer): Promise<boolean>;
    set(key: Buffer, value: Buffer): Promise<void>;
    del(key: Buffer): Promise<void>;
    write(batch: Batch): Promise<void>;
    createReadStream(options?: DBOptions): NodeJS.ReadableStream;
    clear(options?: DBOptions): Promise<void>;
    close(): void;
}

StateDB

Properties

• Provide snapshot of a current state, which will not be changed
• Compute SMT fast
• Provide SMT proof
• Rollback to previous state

Sparse Merkle Tree

General direction is to batch 4 heights (16 nodes) as a subtree to reduce database call. Also, batch key-value updates to eliminate duplicate paths while traversing the tree.

Interface

interface DBOptions {
    readonly gte?: Buffer;
    readonly lte?: Buffer;
    readonly reverse?: boolean;
    readonly limit?: number;
}

interface StateDBReader {
    get(key: Buffer): Promise<Buffer>;
    has(key: Buffer): Promise<boolean>;
    createReadStream(options?: DBOptions): NodeJS.ReadableStream;
}

interface StateDBReadWriter {
    get(key: Buffer): Promise<Buffer>;
    has(key: Buffer): Promise<boolean>;
    createReadStream(options?: DBOptions): NodeJS.ReadableStream;
    snapshot(): void;
    restoreSnapshot(): void;
    set(key: Buffer, value: Buffer): void;
    del(key: Buffer): void;
}
interface Proof {
	readonly siblingHashes: Buffer[];
	readonly queries: Query[];
}

interface Query {
	readonly key: Buffer;
	readonly value: Buffer;
	readonly bitmap: Buffer;
}

export interface SparseMerkleTree {
    update(root, keys, values): Promise<bytes>;
    proof(queryKeys: bytes[]): Promise<Proof>;
}

export interface StateDB {
    newReader(): StateDBReader;
    newReadWriter(): StateDBReadWriter;
    get(key: Buffer): Promise<Buffer>;
    has(key: Buffer): Promise<boolean>;
    revert(prevRoot: Buffer): Promise<Buffer>;
    proof(queryKeys: Buffer[]): Promise<Proof>;
    commit(prevRoot: Buffer, dryRun?: boolean, expected?: Buffer): Promise<Buffer>;
    finalize(height: number): Promise<void>;
    close(): void;
}

Acceptance Criteria

• All interfaces are implemented and tested with fixtures

Description

Expose SparseMerkleTree::calculate_root through SparseMerkleTree and StateDB classes.

Acceptance Criteria

• Type is defined on types.d.ts
• Add unit test on js

Additional Information

• Complex unit test wont be required because the function is tested through verify function

Description

To be able to successfully resolve issue #39, slow parts of the code needs to be detected.
We can use Valgrind to profile execution of a program.

Acceptance Criteria

• it is possible to create a profiling result file and open it with KCachegrind
• a list of top functions which take the most time exists
• a list of top functions by number of calls exists

Additional Information

Useful resource:

• How-to Optimize Rust Programs on Linux

Expected behavior

If there are prebuilt on remote, it should download instead of building locally

Actual behavior

It always build the package

Which version(s) does this affect? (Environment, OS, etc...)

0.3.3

Description

Rust files should be structured that first everything is declarated and after that implementations follow.

Order of sections within Rust file:

• use statements
• const and static variables
• type declarations
• trait declarations
• enum declarations
• struct declarations
• static functions
• trait implementations
• enum implementations
• struct implementations
• public functions

Acceptance Criteria

• all Rust files follow above file structure convention

Description

Add interface to return root and version (rename from height) to StateDB.
Both values should be save atomically with commit and revert

Motivation

• root / version can be used for checking the current state of the stateDB

Acceptance Criteria

• root and height should be stored consistently
• Add tests to check root and height saved with commit

Description

In the rust implementation, when passing the object, often it is using clone so that borrow constraint does not get violated.
However, instead, it should carefully check where it can be passed by reference and try to re-use the allocated data for performance and efficiency.

• One idea is to increase the usage of Rc

Acceptance Criteria

• Reduce number of copy happens in the code
• All tests and public interfaces should remained the same

Description

state_writer_snapshot needs to be able to hold multiple snapshot and revert to a particular one

interface should be

const snapshotIDFirst = createSnapshot();
const snapshotIDSecond = createSnapshot();

revertSnapshot(snapshotIDFirst);

• Also, once revert is called, all snapshot can be discarded

Motivation

When executing a state change, it is required to revert more than 1 layer

Acceptance Criteria

• Add test on nodejs to verify above scenario is possible

Description

Introduce parallel processing on places in the code where it is applicable and make sense from a code execution performance perspective. An average CPU nowadays has a multiple cores so it makes sense to transform CPU intensive calculations into parallel processing to gain code execution speed

• a crate called Rayon might be used for parallel processing
• among other features, Rayon has parallel iterators which may be used to transform code from sequential processing into parallel processing
• example code how it's possible to make a code to run in parallel using Rayon parallel iterators:

// instead of writing code like this
let mut sum = 0;
for element in vec_array {
  sum += element*element
}

// it would be better to translate above code into this
vec_array.iter().map(|&el| el * el).sum()

// because the above code can be trivially converted to be executed in parallel with Rayon
vec_array.par_iter().map(|&el| el * el).sum()

When using parallelism in a code, shared resources between different threads need to be protected by not having simultaneous operations on them. If two different threads want to change a shared resource it might occur that this resource will be in invalid state if it won't be locked before any write occurs.

• a crate called parking_lot might be used for locking a shared resources
• where applicable RwLock may be used because multiple threads can read a shared resource at the same time and only one thread can lock a shared resource for writing purpose

Acceptance Criteria

• where applicable code is running in a parallel
• some tests before and after introducing parallelism are taken and measurements are documented

Additional Information

• some functions in smt.rs file are CPU intensive with a lot of calculations
• concentrate on those functions

Expected behavior

shelljs should be in dependencies because it is used in npm install hook

Actual behavior

shelljs is in devDependencies

Steps to reproduce

try to install with production

Which version(s) does this affect? (Environment, OS, etc...)

0.3.4

Description

Project will be using Clippy as linter for Rust code.

• clippy.toml file needs to be created with following attributes:

# cyclomatic complexity is not always useful
cognitive-complexity-threshold = 100
# types are used for safety encoding
type-complexity-threshold = 10000
# we won’t allow too many function arguments
too-many-arguments-threshold = 13

• GitHub action needs to be created to check Clippy warnings when a PR is created and when a PR is merged

Acceptance Criteria

• clippy.toml file exists in project's root directory
• GitHub action is created

Description

Make a code more generic by using traits and implementing them on structs or types.

• currently some functions like key_hash(), value_hash(), etc. are used only on &[u8] object type
• we may introduce a trait for those functions and implement it for an object type
• example: instead of a function call key_hash(&value), we could call function value.key_hash() on an object
• reference implementation:

pub trait MyHash {
  fn my_hash(&self) -> Vec<u8>;
}

impl MyHash for &[u8] {
  fn my_hash(&self) -> Vec<u8> {
	let mut hasher = Sha256::new();
	hasher.update(self);
	let result = hasher.finalize();
	return result.as_slice().to_vec();
  }
}

Acceptance Criteria

• traits are used in a code
• methods are implemented for an objects where it makes sense

Additional Information

• only perform this action on a places in the code where and if it makes sense

Description

• Investigate if replacing diff calculation with rocksdb checkpoint make sense or not in terms of complexity and performance
• if above make sense, update the revert to use checkpoint

Acceptance criteria

• Provide benchmark of before/after
• Provide reasoning why checkpoint would be better/worse than the current diff calculation
• All existing test should pass

Expected behavior

For example, with a data like

const data = [
        {
          key: 'f15513bb375856a100000000',
          value: '0a087472616e7366657212135472616e73666572204576656e74204e616d651a204bfd60536797462760b7c76207c48ee87a544c7d402e92172e43698f73dfdd472220eee7b5908f179969b0287ac8f0e609777448a66a53e53880085d7ed9287a8c9722205fcbde4eaa4b9e7e89c6053c9dc8f6f865bdb26d962b3b10f7360cc3dfa53c41280c3000'
        },
        {
          key: '8ee6c245dd8ef30300000001',
          value: '0a087472616e7366657212135472616e73666572204576656e74204e616d651a204bfd60536797462760b7c76207c48ee87a544c7d402e92172e43698f73dfdd472220eee7b5908f179969b0287ac8f0e609777448a66a53e53880085d7ed9287a8c9722205fcbde4eaa4b9e7e89c6053c9dc8f6f865bdb26d962b3b10f7360cc3dfa53c41280c3000'
        },
        {
          key: 'b1096238b75b1fd300000004',
          value: '0a087472616e7366657212135472616e73666572204576656e74204e616d651a2001cc08d1321e35a268ff070dee34ec02bb833b791f6c0ec12366abb34bb910552220da42ca6720a26c5a83e36ac0484e80e1c7a09099b08dad5ef7e3583052ae62c6280c3001'
        }
      ]

It should be able to provide root and proof for the key

Actual behavior

With above, it returns

const queryBytes = Buffer.from('f15513bb375856a100000000', 'hex');
const root = await eventSMT.update(Buffer.alloc(0), data);
const proof = await eventSMT.prove(root, queryBytes);

prove will throw Invalid input: Invalid data. key prefix is invalid.`.

It could be a bug in update or prove

Which version(s) does this affect? (Environment, OS, etc...)

c8c7111

Description

Update and optimize lisk-db to be production ready

Tasks

• #6
• #28
• #29
• #30
• #31
• #32
• #33
• #34
• #35
• #36
• #37
• #38
• #39
• #40
• #41
• #45
• #50
• #53
• #54
• #57

Description

• .clone() function call takes some time and space penalty because it creates a new object
• try to borrow objects between function calls instead of cloning them

Acceptance Criteria

• code has less .clone() function calls

Additional Information

• only perform this action on a places in the code where and if it makes sense
• Rc or Arc could be useful for this task

Expected behavior

The behavior seems to be correct, but variable name and the usage seems to be invalid

Actual behavior

https://github.com/LiskHQ/lisk-db/blob/main/src/state_db.rs#L583 this variable is version (ie: block height) as mentioned in the comment here

However, it is used as key_length here

Description

• if a function has slice as parameter, it could accept both slices or Vectors as its argument
• Check if any function could accept a fixed size array as a parameter instead of a slice (eg. &[u8; 32] instead of &[u8]). In this case a compiler could verify if a function call has a valid value as argument.

Acceptance Criteria

• functions have slices &[u8] as parameters instead of vectors Vec<u8>

Additional Information

• only perform this two actions on a places in the code where and if it makes sense

Expected behavior

All the test should pass constantly

Actual behavior

Node.js test
Database › Database › DatabaseReader › iteration › should return empty if no data exist

sometimes fail on the CI.

Steps to reproduce

Run the node.js test multiple times

Which version(s) does this affect? (Environment, OS, etc...)

development-

Description

Update smt.prove function for both SparseMerkleTree and SMTDB

Acceptance Criteria

• Add tests using fixtures from lisk-sdk

Expected behavior

publish-github (self-hosted, macOS, ARM64, aarch64-apple-darwin, 16) should have process.arch as arm64

Actual behavior

publish-github (self-hosted, macOS, ARM64, aarch64-apple-darwin, 16) is returning x64 for process.arch

Steps to reproduce

Check the action for the above https://github.com/LiskHQ/lisk-db/runs/7548457552?check_suite_focus=true

Which version(s) does this affect? (Environment, OS, etc...)

main-

Description

Join variables into structs in places where it is applicable and make sense from a clean code perspective. With this technique a number of functions parameters could be smaller.

• if some variables are naturally connected/depended between each other it would be better to join them into a struct
• an example of joining two connected variables:

// instead of having two independent variables
let key = vec![1,2,3,4];
let value = vec![5,6,7,8];

// it is better to join them together into a struct
struct KVPair(Vec<u8>, Vec<u8>);
let kv_pair = KVPair(vec![1,2,3,4], vec![5,6,7,8]);

• check if any struct could be present as a field in some other struct
• check if any struct could be joined with some other struct into a new struct

Acceptance Criteria

• variables are joined into structs or tuples where it makes sense
• different structs don't have the same fields present inside them but in this case one struct is embedded inside another struct

Additional Information

• possible candidates for this type of refactorizations are QueryProof, QueryProofWithProof and Proof structs
• In a lot of places in the code we have key-value pairs. Maybe join them in a tuple or struct?

Expected behavior

types.d.ts includes correct typing for getCurrentState

Actual behavior

types.d.ts has currentState, but exposed function is getCurrentState

Expected behavior

StateDB.verify and StateDB.proof should be the same interface as SparseMerkleTree.verify and SparseMerkleTree.proof.

Unit test on JS should be added to ensure the interface is correct. Note that StateDB key is by default.

Actual behavior

The internal smt.verify and smt.proof is used for both in memory and state db, but bridge between js and rust is not working.

Steps to reproduce

describe('proof', () => {
            it('should generate proof and verify the result', async () => {
                const queries = [getRandomBytes(38), getRandomBytes(38)];
                const proof = await db.proof(root, queries);

                const result = await db.verify(root, queries, proof);

                expect(result).toEqual(false);
            });
        });

above unit test should pass in the state_db.spec.js

Description

• We have some variable names which are abbreviation and Hungarian Notation in our code that should be rename.
• Also we need to restructure the modules of the code for better organization.

Acceptance criteria

• Variable name should be more descriptive.
• Eliminate all Hungarian notation
• Rename abbreviation variable to a good name
• Review functions name to make a better naming if it is possible.
• Review modules to make a better structure or even separate some parts of them into new one.

Additional Information

• We can follow the following roles for our naming style: Naming in rust

Description

• they will be easier to maintain
• easier to find an opportunity for their optimization and possible parallelism

Acceptance Criteria

• all functions have below 50 lines of a code

Additional Information

• only perform this action on a places in the code where and if it makes sense
• we may agree on a different number of allowed lines of a code per function

Expected behavior

It should fail on smt verify if bitmap includes zero byte at the begging or the end, smt.verify should fail.

Actual behavior

it fails on verify if zero byte is appended but not when prepended

Steps to reproduce

Modify tests to prepend and append zero bytes to proof bitmap, and call verify

Which version(s) does this affect? (Environment, OS, etc...)

0.3.4

Description

Currently SMT calculation is running on background thread, but it might be able to improve performance by splitting the task to multiple threads.

Acceptance criteria

• Provide benchmark result for before and after
• All existing tests should pass

Description

• Add checkpoint(folderPath: string) interface for Database and StateDB
• Expose the checkpoint interface from JS library
• checkpoint should be async

Acceptance Criteria

• Add a test on node.js side

Description

Add option to open Database and StateDB as a secondary DB.
catchup method should be added for secondary db which runs in background thread.
If this function is called for the non-secondary DB, it should throw error.

Motivation

During migration or running third party command, it would be nice to be able to read the database from different process

Acceptance Criteria

• Add test for secondary db case in node.js

Expected behavior

These two unit tests should be included in a test suite and all tests should passed.

Actual behavior

If any of these two unit tests is included into test suite, sometimes below error occurs:

#
# Fatal error in , line 0
# Check failed: result.second.
#
#
#
#FailureMessage Object: 0x7ffe996f06b0
 1: 0xb6ca81  [/usr/bin/node]
 2: 0x1bef1a4 V8_Fatal(char const*, ...) [/usr/bin/node]
 3: 0xfb2e31 v8::internal::GlobalBackingStoreRegistry::Register(std::shared_ptr<v8::internal::BackingStore>) [/usr/bin/node]
 4: 0xd048a8 v8::ArrayBuffer::GetBackingStore() [/usr/bin/node]
 5: 0xab8d00 napi_get_typedarray_info [/usr/bin/node]
 6: 0x7fc008f230a2  [/home/matjaz/projects/work/lisk-db/bin-package/index.node]
 7: 0x7fc008a47080  [/home/matjaz/projects/work/lisk-db/bin-package/index.node]
 8: 0x7fc008a21af9  [/home/matjaz/projects/work/lisk-db/bin-package/index.node]
 9: 0x7fc008a0ce3c  [/home/matjaz/projects/work/lisk-db/bin-package/index.node]
10: 0x7fc008a4f819  [/home/matjaz/projects/work/lisk-db/bin-package/index.node]
11: 0x7fc008a3dd9a  [/home/matjaz/projects/work/lisk-db/bin-package/index.node]
12: 0x7fc0089ef904  [/home/matjaz/projects/work/lisk-db/bin-package/index.node]
13: 0xaaf64d  [/usr/bin/node]
14: 0xd396ae  [/usr/bin/node]
15: 0xd3aacf v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/usr/bin/node]
16: 0x15d5519  [/usr/bin/node]`

Steps to reproduce

Include any of two unit tests into a test suite and execute:
$ yarn test:node

Which version(s) does this affect? (Environment, OS, etc...)

Current main branch.

Expected behavior

if I'm building lisk core, or any other package that uses lisk-db as a dependency, and there's no prebuild for my platform, npm should attempt to build lisk-db from source

Actual behavior

No attempt is made

~/lisk-core # npm ci
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
npm ERR! code 1
npm ERR! path /root/lisk-core/node_modules/@liskhq/lisk-db
npm ERR! command failed
npm ERR! command sh -c -- node-pre-gyp install
npm ERR! response status 404 Not Found on https://js.lisk.com/lisk-db/0.3.0-debug.20-52ffbeac2c1b0b188e8df995a1dc6a1634426a27/index-v0.3.0-debug.20-52ffbeac2c1b0b188e8df995a1dc6a1634426a27-node-v93-linux-x64-musl.tar.gz
npm ERR! node-pre-gyp info it worked if it ends with ok
npm ERR! node-pre-gyp info using [email protected]
npm ERR! node-pre-gyp info using [email protected] | linux | x64
npm ERR! node-pre-gyp info check checked for "/root/lisk-core/node_modules/@liskhq/lisk-db/bin-package/index.node" (not found)
npm ERR! node-pre-gyp http GET https://js.lisk.com/lisk-db/0.3.0-debug.20-52ffbeac2c1b0b188e8df995a1dc6a1634426a27/index-v0.3.0-debug.20-52ffbeac2c1b0b188e8df995a1dc6a1634426a27-node-v93-linux-x64-musl.tar.gz
npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://js.lisk.com/lisk-db/0.3.0-debug.20-52ffbeac2c1b0b188e8df995a1dc6a1634426a27/index-v0.3.0-debug.20-52ffbeac2c1b0b188e8df995a1dc6a1634426a27-node-v93-linux-x64-musl.tar.gz
npm ERR! node-pre-gyp ERR! install error
npm ERR! node-pre-gyp ERR! stack Error: response status 404 Not Found on https://js.lisk.com/lisk-db/0.3.0-debug.20-52ffbeac2c1b0b188e8df995a1dc6a1634426a27/index-v0.3.0-debug.20-52ffbeac2c1b0b188e8df995a1dc6a1634426a27-node-v93-linux-x64-musl.tar.gz
npm ERR! node-pre-gyp ERR! stack     at /root/lisk-core/node_modules/@mapbox/node-pre-gyp/lib/install.js:67:15
npm ERR! node-pre-gyp ERR! stack     at processTicksAndRejections (node:internal/process/task_queues:96:5)
npm ERR! node-pre-gyp ERR! System Linux 5.15.90-gentoo
npm ERR! node-pre-gyp ERR! command "/usr/local/bin/node" "/root/lisk-core/node_modules/.bin/node-pre-gyp" "install"
npm ERR! node-pre-gyp ERR! cwd /root/lisk-core/node_modules/@liskhq/lisk-db
npm ERR! node-pre-gyp ERR! node -v v16.19.0
npm ERR! node-pre-gyp ERR! node-pre-gyp -v v1.0.10
npm ERR! node-pre-gyp ERR! not ok

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2023-02-02T14_42_11_559Z-debug-0.log

Steps to reproduce

The easiest way is to try this in docker.

docker pull node:16-alpine
docker run -it node:16-alpine /bin/sh
cd ~
apk update
apk add automake autoconf libtool linux-headers python3 rust cargo clang clang-libs clang-dev
git clone https://github.com/LiskHQ/lisk-core.git
cd lisk-core
npm ci

Which version(s) does this affect? (Environment, OS, etc...)

Any where there's no prebuild. Not that we make any promises about unsupported platforms, but npm should at least attempt to build before failing

I think it's just this guy that needs to be changed:

diff --git a/publish.js b/publish.js
index ddceedd..274c566 100644
--- a/publish.js
+++ b/publish.js
@@ -44,7 +44,7 @@
  if (canarySHA) {
     originalPackageJSON.version = `${originalPackageJSON.version}-${canarySHA}`;
  }
- originalPackageJSON.scripts.install = "node-pre-gyp install";
+ originalPackageJSON.scripts.install = "node-pre-gyp install --fallback-to-build";
  fs.writeFileSync("./package.json", JSON.stringify(originalPackageJSON, null, 2));

  //Use a fully qualified path to pre-gyp binary for Windows support

Description

• comments help that future code maintenance/extension is easier and faster, and reviewers and newcomers can understand the code better
• there should be a good balance regarding number of lines between code and comments

Acceptance Criteria

• any public fields, functions and methods should be documented

Additional Information

• Rustdoc comments guideline