Comments (1)
fxamacker
commented on June 12, 2024
CCF codec is almost ready for initial reviews. PR will be opened to add it to onflow/cadence within 1-2 days.
Preliminary Size and Benchmark Comparisons
We are not comparing apples to apples. Prior formats (CBF and JSON-Cadence Data Interchange) didn't specify requirements for validity, sorting, etc.
- CCF encoder sorts events data for deterministic encoding.
- CCF decoder verifies that events data are well-formed and sorted.
At this time, CCF decoder doesn't include the option to check for "Preferred Serialization" (encoding to smallest size).
Size Comparisons
| Encoding | Event Count | Encoded size | Comments |
|---|---|---|---|
| JSON | 48,309 | 13,858,836 | JSON-Cadence Data Interchange Format |
| CCF | 48,309 | 6,159,931 | CCF in fully self-describing and deterministic mode |
CCF's partially self-describing mode would be even smaller (roughly 1/4 the size of JSON) in some use cases.
Speed and Memory Comparisons
│ 48k_events_encode_json.log │ 48k_events_encode_ccf.log │
│ sec/op │ sec/op vs base │
EncodeBatchEvents-20 89.84m ± 17% 69.28m ± 3% -22.88% (p=0.000 n=10)
│ 48k_events_encode_json.log │ 48k_events_encode_ccf.log │
│ B/op │ B/op vs base │
EncodeBatchEvents-20 32.45Mi ± 0% 25.82Mi ± 0% -20.45% (p=0.000 n=10)
│ 48k_events_encode_json.log │ 48k_events_encode_ccf.log │
│ allocs/op │ allocs/op vs base │
EncodeBatchEvents-20 756.6k ± 0% 370.4k ± 0% -51.05% (p=0.000 n=10)
│ 48k_events_decode_json.log │ 48k_events_decode_ccf.log │
│ sec/op │ sec/op vs base │
DecodeBatchEvents-20 646.2m ± 8% 158.3m ± 5% -75.50% (p=0.000 n=10)
│ 48k_events_decode_json.log │ 48k_events_decode_ccf.log │
│ B/op │ B/op vs base │
DecodeBatchEvents-20 234.97Mi ± 0% 56.16Mi ± 0% -76.10% (p=0.000 n=10)
│ 48k_events_decode_json.log │ 48k_events_decode_ccf.log │
│ allocs/op │ allocs/op vs base │
DecodeBatchEvents-20 4.746M ± 0% 1.288M ± 0% -72.86% (p=0.000 n=10)
Event Data Details
The 48,309 events are from a transaction on mainnet with unusually high number of events.
There were 9 event types. These 3 event types had over 15,000 events each: FlowToken.TokensDeposited, FlowToken.TokensWithdrawn, FlowIDTableStaking.DelegatorRewardsPaid
To simplify benchmark code (it's Sunday night), all event values for each event type are the same (i.e. the values are from the first event of that type).
from ccf_draft.
Related Issues (20)
- Add support for `cadence.PathLink` type when the static type is added to Cadence HOT 1
- Explicitly specify handling of invalid UTF-8 in CBOR text strings and Cadence strings HOT 1
- Need to define new top level CCF message type in order to bind typedef with value it describes
- Add constraints in CDDL to uint-value, uint128-value, and uint256-value
- Add section for Restrictions HOT 1
- Shorten and cleanup Why CBOR section of specs HOT 1
- Update Security Considerations
- Remove unnecessary cadence-type-id
- Update Acknowledgements section
- Add support for type parameters in function value
- Update specs for composite-type-value.initializers
- Remove nullable type-value and explicitly specify individual type-value as nullable
- Allow null for capability-type and restricted-type elements HOT 1
- Update CCF examples to match latest changes to CDDL and confirm they match CCF Codec
- Update bullet point about deterministic encoding in Objectives section
- Update paragraph about fxamacker/cbor (CBOR codec in Go) HOT 1
- README has outdated status and introduction can be improved
- Update "Why CBOR" with 2 comparisons published by IETF HOT 1
- Update CCF specs and tag RC2 before it gets hosted at onflow/ccf repo HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ccf_draft.