gannan08 / rdf-canonize-rs Goto Github PK

TLDR; hash_first_degree_quads is a good target for optmizations. I'll be doing another breakdown for that function.

124 micros for single iteration inside the test harness

90 micros

https://github.com/gannan08/rdf-canonize-rust/blob/main/src/urdna2015.rs#L58-L97

3 micros

https://github.com/gannan08/rdf-canonize-rust/blob/main/src/urdna2015.rs#L61-L68

82 micros

https://github.com/gannan08/rdf-canonize-rust/blob/main/src/urdna2015.rs#L83-L89

Multiple passes through hash_first_degree_quads looks like this (don't mind the rounding)

https://github.com/gannan08/rdf-canonize-rust/blob/main/src/urdna2015.rs#L246-L340

FIRST_DEGREE_QUADS_1 39
FIRST_DEGREE_QUADS_1 61
FIRST_DEGREE_QUADS_1 1
FIRST_DEGREE_QUADS_1 6
FIRST_DEGREE_QUADS_1 1
FIRST_DEGREE_QUADS_1 10

4 micros

https://github.com/gannan08/rdf-canonize-rust/blob/main/src/urdna2015.rs#L98-L119

6 micros

https://github.com/gannan08/rdf-canonize-rust/blob/main/src/urdna2015.rs#L174-L236

First, I want to make sure you're aware of this branch and its capabilities:

https://github.com/gannan08/rdf-canonize-rust/blob/1d2b3b250c3efc7a2e4e5043115b107dfb489a35/benches/README.md#L11-L12

Secondly, by adding timers and an accumulator, I have discovered that 2/3 of the time spent doing test044_canonize is in this one function:

https://github.com/gannan08/rdf-canonize-rust/blob/1d2b3b250c3efc7a2e4e5043115b107dfb489a35/src/urdna2015.rs#L302-L303

Presently, total runtime is approx 3ms for that test, and 1.95ms of that time is spent in create_hash_to_related.

The amazing thing is, that 24 hours ago a test iteration was 9-10ms!

I've been journaling progress here: https://app.gitbook.com/@digitalbazaar/s/rust/v/master/rdf-canonize-optimizations

I'm going to get a start on this. I'll report my progress here.

Working here: https://github.com/gannan08/rdf-canonize-rust/tree/nquad-dataset-rio

• Clean up commits
• Change License to BSD-3
  - [ ] Get final regex code into main
• Update README.md

I was curious to know if we were dealing with a pure crypto race so I put this together:

https://github.com/mattcollier/rust-hash-bench

Which goes with this: https://github.com/digitalbazaar/loop-bench/blob/rdf-canonize-local-native/index25.js

which shows this for the default 1M operations.

NODEJS MS 3013.116585
RUST MS 1916.46965

Rust wins for big and small numbers of operations.

This test demonstrates the failure:

https://github.com/digitalbazaar/loop-bench/blob/b066d4782f20fb6ff6fc1d916ffe0d043f33b0aa/index23.js#L67-L68

The assertion will pass if the created date is modified to 2020-12-29T21:12:26
https://github.com/digitalbazaar/loop-bench/blob/b066d4782f20fb6ff6fc1d916ffe0d043f33b0aa/index23.js#L42

The assertion also fails if L42 is removed entirely.

• https://github.com/mattcollier/rust-hyperscan-test
• rust-lang/regex#348
• Cool visualizer
  • https://www.debuggex.com/

Looks like we just need to wire what you have here into this: https://github.com/mattcollier/rust-node-bindgen-canonize/blob/9aecb5646dfcbc2d633495a5b09fdfa712cece15/src/lib.rs#L79-L80

I've tested this and it will accept the inputs from rdf-canonize tests.

You just need nj-cli to build the bindgen-canonize stuff: https://github.com/infinyon/node-bindgen#cli-installation

// within the rust-node-bindgen-canonize folder
nj-cli build

const rustCanonize = require('../../rust-bindgen-canonize/dist/index.node');

// in the rdf-canonize test suite
result = canonize.canonize.apply(null, clone(jsParams));