Comments (5)
Idea for Unpack
When an error occoured on unpack process, a tree view of that tells developer how program works can take more useful information for debugging.
Here I will give some examples.
Example 1
The type schema is vector Bytes <byte>
# there are 6 items
06 00 00 00
# six items
23 01 00 00, 56 04 00 00, 90 78 00 00, 0a 00 00 00, bc 00 00 , ef 0d 00 00
# ↑
# this section is missing one byte
The error message is
vector<Byte>
|-- _size: 6 (06 00 00 00)
|-- [0]: 0x123 (23 01 00 00)
|-- [1]: 0x456 (56 04 00 00)
|-- [2]: 0x7890 (90 78 00 00)
|-- [3]: 0xa (0a 00 00 00)
|-- [4]: 0xef0000bc (bc 00 00 ef)
|-- [5]: Failed, try parse 4 bytes but got 3 bytes: 0d 00 00
When the developer see this error message, it's expected result is [0x123, 0x456, 0x7890, 0xa, 0xbc, 0xdef]
.
So he can find the bytes in vec[4]
position have missed a 00
byte.
Consider a more complex condition.
The type shema is table MixedType { f1: Bytes, f2: byte, f3: Uint32, f4: Byte3, f5: vec byte }
And the input is
# the full size is 43 (0x2b) bytes
2b 00 00 00
# five offsets (20 bytes in total)
18 00 00 00, 1c 00 00 00, 1d 00 00 00, 21 00 00 00, 24 00 00 00
# five items (19 bytes in total)
# the full size is 43 (0x2b) bytes
2b 00 00 00
# five offsets (20 bytes in total)
18 00 00 00, 1c 00 00 00, 1d 00 00 00, 21 00 00 00, 24 00 00 00
# five items (19 bytes in total)
00 00 00 00
ab
23 01 00 00
45 67 89
03 00 00 00, ab ef
# ↑
# missing a byte in vector
The error message is
Table MixedType
|- _size: 0x2b(2b 00 00 00)
|- _offsets: 0x18, 0x1c, 0x1d, 0x21, 0x24(18 00 00 00, 1c 00 00 00, 1d 00 00 00, 21 00 00 00, 24 00 00 00)
|- f1(vec byte):
|---- _length: 0x0(00 00 00 00)
|- f2(byte): 0xab(ab)
|- f3(Uint32): 0x123(23 01 00 00)
|- f4(Byte3): 0x456789(45 67 89)
|- f5(vec bytes)
|- _length: 0x3(03 00 00 00)
|- [0]: 0xab(ab)
|- [1]: 0xef(ef)
|- [2]: Failed, try parse a byte but got nothing(meet end of input)
Although sometimes the position triggers error is not the root cause, Like case in example 1, the missing bytes is not in the end, but developers can still fix their bug by the error message.
implementation of pack/unpack
in current codec structure, we need a extra context. How to pass it through various type codec method?
It have some approaches.
Pass context as parameter
we should change the pack/unpack function parameters.
pack: (packable: Packable, context?: CodecContext) => Packed;
unpack: (unpackable: Unpackable?: CodecContext) => Unpacked;
And we hide the CodecContext constructor(just don't export it in module). then the developer can only pass the first parameter.
- Pros: Easy to implement
- Cons: take confuse to developers. they will think: Why there is the second parameter that I can't use?
pass the context by this
keywords
pack: (this: CodecContext, packable: Packable, context?: CodecContext) => Packed;
unpack: (this: CodecContext, unpackable: Unpackable?: CodecContext) => Unpacked;
and when we called subtype codec methods, we should do some extra work.
items.map((item, index) => {
return this.context.apply(index, item.codec, buf.slice(0, 114514))
})
- Pros: codec API user friendly. Nothing changes on the API
- Cons: TBD
CodecContext
This class is a rough draft. Maybe we need more discussion.
export class CodecContext<Packed = any, Unpacked = any> {
private subcontextKeyMap: WeakMap<
CodecContext,
string | number
> = new WeakMap();
private subcontexts: CodecContext[] = [];
public type:
| "Uint8"
| "Uint16LE"
| "Uint16BE"
| "table"
| "array"
| "option"
| "struct"
| "vector"
| "union"
| "unknown" = "unknown";
public failed = false;
apply(
path: string,
fn: Codec<Packed, Unpacked>["pack"],
packable: Unpacked
): Packed;
apply(
path: string,
fn: Codec<Packed, Unpacked>["unpack"],
unpackable: Packed
): Unpacked;
apply(path: string, fn: any, input: any): any {
const subcontext = this.createSubcontext(path);
return (fn as Function).call(subcontext, input);
}
createSubcontext(path: string) {
const subcontext = new CodecContext();
this.subcontexts.push(subcontext);
this.subcontextKeyMap.set(subcontext, path);
return subcontext;
}
get lastSubcontext(): CodecContext | undefined {
return this.subcontexts[this.subcontexts.length - 1];
}
get codecPackError(): Error | undefined {
let context: CodecContext | undefined = this;
let errorPaths: (string | number)[] = [];
while (context !== undefined) {
const lastSubcontext: CodecContext | undefined = context.lastSubcontext;
if (lastSubcontext !== undefined) {
errorPaths.push(context.subcontextKeyMap.get(lastSubcontext)!);
} else if (!context.failed) {
return;
}
context = lastSubcontext;
}
const errorPath = errorPaths.reduce(
(acc, cur) => (acc + typeof cur === "number" ? `[${cur}]` : `.${cur}`),
""
);
return new Error(
`${errorPath} is incorrect.
Expected: ${this.type}
`
);
}
}
from lumos.
Idea for Unpack
Thx for such a detailed reply for unpack
enhancement design
I believe the byte layout view in console is helpful for understanding molecule.
But in all of the above scenarios, we presuppose that the content of unpack is PARTIAL incorrect, the molecule header is correct. The Most common error we make is actually exactly "unpacking an unknown binary data source via a Codec
". For example
MyDataCodecForA.unpack(cellA.data) // correct
MyDataCodecForA.unpack(cellB.data) // incorrect
from lumos.
Seems we can not display the actual type in error message.
but tell user the expected type is not difficult.
Like this error message.
Expect type Uint8BE at arrayField[2] but got error: Value must be between 0 and 255, but got 1131796
from lumos.
Expect type Uint8BE at arrayField[2] but got error: Value must be between 0 and 255, but got 1131796
got it, we can display the error message and the path, it is enough for troubleshooting
from lumos.
Seems we can not display the actual type in error message. but tell user the expected type is not difficult. Like this error message.
Expect type Uint8BE at arrayField[2] but got error: Value must be between 0 and 255, but got 1131796
I create a new PR, but it is not include unpack
implementation.
#403
from lumos.
Related Issues (20)
- Add support for CRLF on the molecule parser
- [Question] How to use `pnpm` with `Lumos@22`? HOT 2
- Depreacte the support of Omnilock Bitcoin authentication's P2SH address HOT 11
- Allow customizing the httpClient for the `Indexer`
- New RPC test_tx_pool_accept to check if a transaction is acceptable
- A helper for blake160 since it's common in CKB HOT 1
- Add `next.js` in `Docs/Receipts/CRA, Vite, Webpack or Other`
- JS error on Lumos Address Conversion tool. HOT 1
- Docs for the usage of `TransactionSkeleton`
- Demo for issuing and transferring xUDT
- Support the RPC method `test_tx_pool_accept` in [email protected] HOT 3
- How to calculate the fee for sending all balances HOT 1
- Got error when transfer ckb using omni_lock address HOT 2
- get_raw_tx_pool response TxPoolEntries is changed from 115
- One-step function to dump the tranasction for ckb debugger
- How to parse testnet address to script HOT 3
- Create Type id cells HOT 1
- New RPC metods for 0.117 HOT 1
- [FR] Add XUDT testnet reference by `data1` HOT 4
- Lead users to ckb-ecofund/ccc HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lumos.