birthdayresearch / jellyfishsdk Goto Github PK
View Code? Open in Web Editor NEWDeFiChain Jellyfish SDK & Ecosystem for building modern lite DeFi Applications at scale.
Home Page: https://jellyfishsdk.com
License: MIT License
DeFiChain Jellyfish SDK & Ecosystem for building modern lite DeFi Applications at scale.
Home Page: https://jellyfishsdk.com
License: MIT License
Automatically create a pull request to jellyfish/packages/testcontainers with the update docker version.
Refactoring of jellyfish-address
code/logic to be lighter, currently, it's too complex. Dangerous technical debt.
@@ Coverage Diff @@
## main #189 +/- ##
========================================
Coverage 96.21% 96.22%
========================================
Files 66 74 +8
Lines 1718 1958 +240
Branches 235 264 +29
========================================
+ Hits 1653 1884 +231
- Misses 65 74 +9
I would expect around <15
code logical branches for jellyfish-address
but there were +29
which is 10% more than the current combined mono repo code base.
#237 has only +7
for comparison and it has a much more complex implementation design. Clearly, there is much to optimize, the more code there is, the more there is to maintain. Less is more.
Not to use readBigUInt64LE
and bigint
as it's not support in many platform.
Ability to create hierarchical deterministic wallet with JavaScript/TypeScript that can be used for the DeFiChain ecosystem.
Likely to follow the principles behind bip32
& bip39
but not yet decided.
To reduce the complexity of private key and wallet management for the developers of DeFiChain ecosystem.
Improve front matter of the documentation website. This include:
jellyfish.defichain.io
?Currently the documentation website is populated with basic relevant docs of jellyfish development. But not much attention is paid to the "front matter" of the docs website.
Service provider like Netlify
, Vercel
& AWS Amplify
can be used.
This tracks all the RPC methods and categories being implemented in
jellyfish-api-core
. Will list all the methods here, and strike out those that are not applicable.
Feel free to update this and tag yourself if you want to work on a feature. You will need to reference DeFiCh/ain code base, https://github.com/DeFiCh/ain/tree/master/src/rpc to be exact.
This is purely for documenting all features of DeFi Blockchain.
Everything that is required has already been completed.
src/rpc
blockchain.cpp
mining.ccp
misc.cpp
net.cpp
rawtransaction.cpp
src/spv
spv_rpc.cpp
src/wallet
rpcwallet.cpp
rpc_accounts.cpp
- COMPLETEDrpc_masternodes.cpp
- COMPLETEDrpc_poolpair.cpp
- COMPLETEDBlocked by #187
To refactor #139 after merged
We need a utility for DeFi Blockchain address for easy address interfacing for DeFi JS ecosystem.
export type Network = 'mainnet' | 'testnet' | 'regtest'
export const DeFiAddress {
from(address: string): {hex: string, network: Network}, // return hex with prefix stripped
to(hex: string, network: Network): string, // return address with network prefix
}
At the same time, we should refactor all address related code into @defichain/jellyfish-address
. Putting it into package/jellyfish-address
and importing jellyfish-crypto
and jellyfish-network
as dependencies.
Allow sandbox environment in the documentation website.
ocean.defichain.com
https://github.com/DeFiCh/ocean/issues/1Its not critical but also a note.
Currently the precision is loose on
while:
parse (obj, type): any
parse({ foo: { bar: 123 }, bar: 123 }, { bar: 'bignumber' }
result:
{ foo: { bar: { BigNumber }}, bar: { BigNumber }}
expected:
{ foo: { bar: 123 }, bar: { BigNumber }}
note:
Two issues will be facing while implement the strict precision
ApiClient.call
's resp object structure - { result, error, id }
// better
getBlock(...args) {
return await this.client.call(m, [], { number: 'bignumber' })
}
// Not so friendly
getBlock(...args) {
return await this.client.call(m, [], { result: { number: 'bignumber' }})
}
precision
by empty object {}
// remap.ts ln: 108-110
case MappingAction.DEEPLY_UNKNOWN:
-- remapLosslessObj(losslessObj[k], precision)
++ remapLosslessObj(losslessObj[k], {})
break
this causes the nested object cannot be parsed due to empty precision.
{foo: { bar: 123 }} !== { bar: 123 }
But its also convince-able to remain the loose precision
parse({foo: {bar: 123}, bar: 123}}, {bar: 123})
assume bar
is a crypto amount, so why not parsing both??
parse({foo: {bar: 123}, bar: 123}}, { foo: {bar: 'number'}, bar: 'bignumber' })
// foo.bar remains number
// bar is converted to bignumber
This tracks all the DeFi scripting (AKA CustomTx) that is to be implemented in jellyfish-transaction.
packages/jellyfish-transaction/src/script/defi/*
All CustomTX has a starting signature of DfTx
in utf8 or 0x44665478
. It is as it is, don't need to convert from BE to LE.
They are structured as a
VarUInt
+ [OP_RETURN
+ DfTx
+ TYPES[CxTxType]
+ ...]VarUInt
+ 6a
+ 44665478
+ Type + ...It's a char, UTF8 -> HEX
'C'
@ivan-zynesis #283'R'
@ivan-zynesis #283'T'
@canonbrother #208'M'
@canonbrother #170'N'
@canonbrother #300'n'
@canonbrother #300'p'
@canonbrother #233'u'
@canonbrother #314's'
@fuxingloh #143'l'
@fuxingloh #145'r'
@fuxingloh #146'U'
@ivan-zynesis #175'b'
@ivan-zynesis #175'B'
@ivan-zynesis #175'G'
@ivan-zynesis #298'a'
@ivan-zynesis #181'A'
, @ivan-zynesis #279jellyfish-core
implementation must be:
Promise.reject()
/triage accepted
/area core
Currently in @defichain/testing
only accountToAccount
is supported. That function requires manual selection of tokens to send for testing. We need sendtokenstoaddress
for testing for a better/easier setup process.
A test within jellyfish-api-core
to ensure that DFI can only be sent to others in the form of UTXO.
expect().toThrow()
Additional regression capability for api-core
.
Rename jellyfish-*
to transport-*
or api-*
. or protocol-*
jellyfish-core
to transport-core
or api-core
jellyfish-jsonrpc
to transport-jsonrpc
or api-jsonrpc
While working on the wallet-core
RFC for #25 jellyfish
name should be reserved for the project as it bundles related project for a single distribution. Renaming jellyfish-core
to api-core
or such will reduce the confusion of what jellyfish does.
/triage accepted
/area core jsonrpc
/priority important-soon
Invalid PR such as #148 and #147 are created. Although the version updates state in the PR title are valid, but the PR created to update the version and wrong.
packages/*/package.json
& package-lock.json
should be created to update just that deps. Min of 2 files and at least 2 LOC to change. There might be lack of support for NPM7 workspaces, hence we might need to individually indicate each package.json in .github/dependabot.yml
This is an example of a correct PR that should be created by dependabot: https://github.com/DeFiCh/oss-governance-bot/pull/71/files
Currently, only bech32 is supported in jellyfish-crypto, we need to add base58check encode/decoding support in
@defichain/jellyfish-crypto
. This is for address type: p2sh, p2pkh.
The implementation should follow the design pattern of https://github.com/DeFiCh/jellyfish/blob/main/packages/jellyfish-crypto/src/bech32.ts
Test fixtures/data should be taken directly from running testcontainers. https://github.com/bitcoinjs/bs58check can be used for the implementation. Our network prefix are here: https://github.com/DeFiCh/jellyfish/blob/main/packages/jellyfish-network/src/index.ts
Although for the light wallet implementation we don't intend to support receiving via non-bech32 address, we still need to support sending. We also need this for DeFi whale and DeFi explorer.
npm i @defichain/jellyfish --save --include=dev
npm ERR! code ETARGET
npm ERR! notarget No matching version found for @defichain/jellyfish-core@^0.0.3.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.
Successful installation
npm i @defichain/jellyfish --save --include=dev
Similar to #108
Implementation of 1.7.x Custom DfTx
https://github.com/DeFiCh/ain/blob/6428ac0ca470dd16f973d62b74a1ec0b9e5444a8/src/masternodes/mn_checks.h#L62-L67
enum class CustomTxType : uint8_t
{
AppointOracle = 'o',
RemoveOracleAppoint = 'h',
UpdateOracleAppoint = 't',
SetOracleData = 'y',
}
in
packages/jellyfish-transaction/src/script/defi/*
All CustomTX has a starting signature of DfTx
in utf8 or 0x44665478
. It is as it is, don't need to convert from BE to LE.
They are structured as a
VarUInt
+ [OP_RETURN
+ DfTx
+ TYPES[CxTxType]
+ ...]VarUInt
+ 6a
+ 44665478
+ Type + ...Refer to #108 for a detailed implementation guide
Moved out of scope from #17 due to waiting for upstream fixes and implementation.
import jellyfish from 'jellyfish.ts'
window.jellyfish = jellyfish
To allow browser users to do this:
<script src="https://unpkg.com/@defichain/jellyfish@latest/dist/jellyfish.umd.js"/>
<script>
var client = new jellyfish.Client('http://ocean.defichain.com')
client.mining.getMintingInfo().then(info => {
console.log(info)
})
</script>
#229 getMintingInfo is used as an example in some places, let me know if this PR should replace the examples as well
Following the deprecation of getMintingInfo
, getMiningInfo
is added to support multiple masternode support from within one defid instance. We need to update all documentation that uses getMintingInfo
as an example to use getMiningInfo
.
Inconsistency in naming convention noticed by @monstrobishi in jellyfish-transaction/script/mapping.ts
.
To refactor as it is best to follow the original naming convention.
OP_DEFI_TX_TOKEN_MINT
OP_DEFI_TX_POOL_REMOVE_LIQUIDITY
DEFI_OP_UTXOS_TO_ACCOUNT
DEFI_OP_ACCOUNT_TO_UTXOS
Currently, the CI only test on linux platform. We should test on windows and mac to guarantee edge and regression don't show up.
--> CI Build & Test (Windows)
CI Build & Test with Coverage (Linux) |
--> CI Build & Test (MacOS)
In the process of dogfooding this library and building up and out the DeFi ecosystem, we have to write tons of tests. The test that is written is pretty much consistent with a minor variable difference to test an aspect. They are not fixed hence your typically fixture paradigm is not effective. But rather we need a testing module (e.g. @defichain/testing
) with utilities to set up the testing conditions.
As this library is written in jellyfish and to prevent circular dependant, jellyfish components should not use @defichain/testing
components for testing. This is for projects that are external to jellyfish, DeFi whale for an example.
async function createSignedTxnHex (
aAmount: number,
bAmount: number,
a: EllipticPair = Elliptic.fromPrivKey(Buffer.alloc(32, Math.random().toString(), 'ascii')),
b: EllipticPair = Elliptic.fromPrivKey(Buffer.alloc(32, Math.random().toString(), 'ascii'))
): Promise<string> {
const aBech32 = Bech32.fromPubKey(await a.publicKey(), RegTest.bech32.hrp as HRP)
const bBech32 = Bech32.fromPubKey(await b.publicKey(), RegTest.bech32.hrp as HRP)
const { txid, vout } = await container.fundAddress(aBech32, aAmount)
const inputs = [{ txid: txid, vout: vout }]
const unsigned = await client.rawtx.createRawTransaction(inputs, {
[bBech32]: new BigNumber(bAmount)
})
const signed = await client.rawtx.signRawTransactionWithKey(unsigned, [
WIF.encode(RegTest.wifPrefix, await a.privateKey())
])
return signed.hex
}
Currently, RegTestContainer
offers a variety of testing functions and boilerplate to ease testing. E.g. container.generate(1)
and container.waitForCoinBaseMaturity()
. We need to add spv
testing functionality into RegTestContainer as well.
spv_fundaddress
as container.spv.fundAddress()
spv_setlastheight
as container.spv.setLastHeight()
class RegTestContainer {
readyonly spv = new Spv(this)
}
Additional testing against testcontainers for jellyfish-transaction
for better compatibility. This can be done on a RegTestContainer
without masternode configured for a quick round of tests. __testcontainers__
directory can be used to separate the tests that require testcontainers to be up.
This provide additional guarantee the txn generated by jellyfish-transaction will all always in compliance with DeFi Blockchain.
Run extended end to end test on testnet and mainnet for extra quality.
Honestly seems highly redundant, but note worthy.
packages/*/
โโโ __tests__/**.test.ts <- fast tests, spins up regtest chain
โโโ __tests_testnet__/**.test.ts <- slow tests, spins up test chain
โโโ __tests_mainnet__/**.test.ts <- slow tests, spins up main chain
Ability to setup and run multiple nodes in @defichain/testcontainers
for distributed decentralized mocking.
For scenario that require testing transaction send from another node.
Documentation to be implemented along side with code in the docs/
directory. Docs to be deployed automatically on merged.
Implement measures to make the test less flaky.
Spending too much time manually re-triggering the tests as they are too flaky.
An API gateway/proxy service for DeFiChain jellyfish users.
Currently, the JSON-RPC
channel in DeFiCh/ain
is password protected by default. Important and intentional by design. However, this makes it difficult to setup password-less service for public to uplink to. It does not make sense to implement API gateway/proxy features in C++. This will also offload the networking layer to a purpose built provider. (๐ NGINX & Related)
DeFiCh/ain
without wallet featuresChange txid
in jellyfish-transaction
when encoded as a string to be formatted as BE order.
jellyfish-api-core
hence BEjellyfish-transaction
uses LE when stored at REST.We should change to BE for consistency.
DeFi whale is a stateless stateless indexer for DeFi Blockchain. jellyfish-api-whale is hence the typed api client for interacting with jellyfish-api-whale. This will be developed together with the DeFi whale project.
DeFi Whale has entered into project inception phase.
WhaleWalletAccount implements the WalletAccount from jellyfish-wallet; a stateless account service for DeFi.
This is required for WhaleWalletAccount.
Built-in JSON/Object parser must parse number as bignumber
, and development of library must take a beginner friendly stance of interoperability with big-number and number.
While most developers are aware of the IEEE 754 double standards, JSON specification does not say anything about number precision. Jellyfish implementation must be accommodating to various developer experience while preventing number precision mistake at large.
THIS IS VERY IMPORTANT FOR THE LIBRARY TO BE ACCOMMODATING TO VARIOUS DEVELOPER EXPERIENCE.
> parseFloat("0.000008") * 10
0.00007999999999999999
/area core
/triage accepted
/priority important-soon
"smart revive" blockchain amount as bignumber, the rest numeric remains as number.
strict type and better view for user
Spin off tx_segwit.ts
and tx_signature.ts
that is part of jellyfish-transaction into a separate package called jellyfish-signature
or jellyfish-transaction-signature
.
To remove "@defichain/jellyfish-crypto"
dependencies out of jellyfish-transaction for lite applications that want to just use jellyfish-transaction
as it further depends on this dependencies. This will eventually make it easier to bundle jellyfish-transaction
into jellyfish, as jellyfish-transaction
only has smart-buffer as a dependency.
"@defichain/jellyfish-crypto"
dependencies:
"bech32": "^2.0.0",
"bip66": "^1.1.5",
"create-hash": "^1.2.0",
"tiny-secp256k1": "^1.1.6",
"wif": "^2.0.6"
#25 jellyfish-wallet
implementation should support ledger hardware wallet.
Ability for the test env to automatically spin up defi/defichain:latest
for testing without any manual setup.
This can be implemented directly into https://github.com/DeFiCh/ain/tree/master/.github
as a workflow to build docker images ready for use without command override. Command interface is not friendly for automation, preference of convention over configuration.
Allow upstream services of DeFiChain Blockchain
[this project included] to utilize conventional defaults when running integration automation, tdd, testing and such. Allowing projects to have the ability to setup fully-featured test env for their various needs.
jellyfish-wallet-mnemonic
with encrypt
and decrypt
function extended above 24 word.The 24 words mnemonic seed should serve as a cold storage of your HD seed.
However the 24 words should not sit in your device (phone/browser/desktop) unencrypted as rest.
Encryption can be done with biometric authentication provided by device, jellyfish-wallet-mnemonic
just need to provide the interface to do it.
Reference example: https://github.com/bitcoin/bips/blob/master/bip-0038.mediawiki
Similar to #48 This tracks all the RPC methods and categories being implemented in jellyfish-api-core
. Please tag update this if you want to work on a feature.
As a final docker build is not produced please use the latest 1.7.x docker build for your testcontainers
, #184 for more information.
src/masternodes/
rpc_icxorderbook.cpp
rpc_oracles.cpp
src/spv
spv_rpc.cpp
jellyfish-network
or jellyfish-wallet
should implement a validator and guess what kind of address this is.
For client/wallet app this provides an utility to present the type of address user are sending their crypto assets to.
This tracks all the OP_CODES class and their categories to be implemented in jellyfish-transaction. DeFi blockchain custom transaction will be tracked in another issue.
packages/jellyfish-transaction/src/script/*
data.ts
constants.ts
control.ts
stack.ts
splice.ts
bitwise.ts
arithmetic.ts
crypto.ts
expansion.ts
invalid.ts
A callback interface for jellyfish-wallet
transaction signing. When a user were to perform any signing action, it triggers a callback to the signing interface. This allow presentation of information to user before they sign the transaction.
const signingInterface = {
unsigned(buffer: Buffer, tx: Transaction): Promise<void> {
console.log('show client')
},
signed(buffer: Buffer, tx: TransactionSegWit): Promise<void> {
console.log('show client')
}
}
const provider = new WalletHdNodeProvider(signingInterface)
To provide the user/client the information of what they are actually signing.
Jellyfish implementation must be platform agonistic and support all possible use case.
<script src="jellyfish.js">
for browser with bundlingTo support the developer ecosystem, jellyfish implementation must plug and play for all platform.
/area core
/triage accepted
Interfaces in WalletAccount for
stateless features:
stateful features
jellyfish-transaction
should validate that it can actually spend the transaction with the given elliptic pair before signing transaction.
Only for p2wpkh
as it is the only one that is implemented.
Currently jellyfish-transaction
will sign any transaction regardless of what prevout is given to it.
It should check it can actually spend the given transaction by, taking the prevout and HASH160
it.
A bit steep to this.
Requires Docker networking layer knowledge in order to proceed.
Available in NPM >v12.7.0
Custom subpaths can be defined along with the main entry point by treating the main entry point as the "." subpath:
{
"main": "./dist/index.js",
"exports": {
".": "./dist/index.js",
"./submodule": "./dist/bech32.js"
}
}
This allow loading a sub path
import submodule from '@defichain/jellyfish-crypto/bech32';
// Loads ./node_modules/@defichain/jellyfish-crypto/dist/bech32.js
As the modules and packages grows, it is becoming increasingly complex to manage imports and exports. Bundling all exports into one index.js
file for small modules make senses but for large more complex modules it is harder to effectively manage conflicts. E.g. jellyfish-api-core
, hence it is wise to implement subpath exports vs named exports for larger packages.
A new package called jellyfish-transaction
, this package will use a builder-like mechanism to facilitate the creation of transaction. Basic P2SH
, P2PKH
, P2WSH
and P2WPKH
are expected. All custom transaction should be implemented as well. They don't need to be implemented in 1 pr but should allow easy iterative implementing.
static const std::vector<unsigned char> DfTxMarker = {'D', 'f', 'T', 'x'}; // 44665478
enum class CustomTxType : unsigned char
{
None = 0,
// masternodes:
CreateMasternode = 'C',
ResignMasternode = 'R',
// custom tokens:
CreateToken = 'T',
MintToken = 'M',
UpdateToken = 'N', // previous type, only DAT flag triggers
UpdateTokenAny = 'n', // new type of token's update with any flags/fields possible
// dex orders - just not to overlap in future
// CreateOrder = 'O',
// DestroyOrder = 'E',
// MatchOrders = 'A',
//poolpair
CreatePoolPair = 'p',
UpdatePoolPair = 'u',
PoolSwap = 's',
AddPoolLiquidity = 'l',
RemovePoolLiquidity = 'r',
// accounts
UtxosToAccount = 'U',
AccountToUtxos = 'b',
AccountToAccount = 'B',
AnyAccountsToAccounts = 'a',
//set governance variable
SetGovVariable = 'G',
// Auto auth TX
AutoAuthPrep = 'A',
};
Signature signing should be supported as well. SIGHASH_ALL
should be implemented first and the rest if required in another PR.
Signature.SIGHASH_ALL = 0x01;
Signature.SIGHASH_NONE = 0x02;
Signature.SIGHASH_SINGLE = 0x03;
Signature.SIGHASH_ANYONECANPAY = 0x80;
In light wallet implementation, you need to be able to create transaction without relying on a node. This package will facilitate that.
Jellyfish to return Promise<BlockInfo>
instead of using callback.
To prevent "callback hell" problem where code get nested and messy, making it harder to understand. As async/await
is very mature, it brings better quality of life to development.
/area core
/triage accepted
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.