rlay-project / rlay-ontology Goto Github PK

According to @MichaelHirn, the currently published version of @rlay/ontology throws an error on import:

Error: Cannot find module './rlay_ontology_js_nodejs_bg'
    at Function.Module._resolveFilename (module.js:548:15)
    at Function.Module._load (module.js:475:25)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/Users/mj/rlay-utils/rlay-client-lib/node_modules/@rlay/ontology/rlay_ontology_js_nodejs.js:1185:8)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)

(Late night thoughts)

Currently the Entity fields (fields of the different entity kinds) are not that concretely specified. A lot of them are currently states as being either an "IRI" (taken from OWL spec, but not really applicable) or just plain bytes.

The rough idea is that all fields that represent relationships to other entities should be CIDs, while all fields that represent "data" should be Multicodec-prefixed bytes (which could also be CIDs given that multiformats/multicodec#49 is added).

By making the data fields Multicodecs, we bascially get a very good equivalent to Literals/Datatypes in OWL2 for free.

Data fields: Annotation->value, (Negative)DataPropertyAssertion->target

Parsing a FormatWeb3 often causes an error because it contains an additional "cid" field that is added during serialization.

Should be easy enough to ignore when deserializing.

For some operations (e.g. verifying the existence of all referenced CIDs), it would be nice to have a function that returns an iterator (or similar) over all fields that are supposed to contain CIDs.

Motivation

The Protobuf format is not that great. An overview from the current doc page of the pros and cons (https://rlay-project.github.io/rlay-client/docs/rlay-ontology-serialization-formats#protobuf-based-format):

Pros:

• Protobuf libaries were easily available for prototyping in Rust and Solidity at time of creation
• Via ordered fields in protobuf schemas, it is pretty easy to have a determenistic content-addressable format
• Low size overhead over contents

Cons:

• Protobuf is comparatively complex for the simple features we need of it
• As the protobuf encoding doesn't contain any information about the entity kind, the entity kind has to be known for the encoding to be correctly interpreted
• Per-EntityKind CID multicodecs would require a lot of codecs to be registered/coordinated
• Unwieldy to use in end-user applications

Proposal

As a replacement for the Probuf format, I am proposing a CBOR-based format with included prefix. The major upside is that CBOR is standardized, and has robust implementations in many languages. The other major change would be an included prefix that makes it possible to deserialize the entity without externally provided information what entity kind the bytes represent.

• Prefix as varint (CBOR unsigned vs. multibase varint?) that specifies the entity kind. Could also be a full multicodec per entity kind. Maybe it's even possible to have it be both a multibase varint that together with a multicodec prefix (for a whole block of numbers) can act as a full multicodec.
• Body as CBOR where empty fields are omitted.
• As field keys numbers are used instead of full field names like "annotation", so it is more compact.
• It could be a good idea to use a shared mapping of "field name"->"field key number", so that the only thing that is necessary for properly deserializing is the the entity kind and that mapping. Otherwise we would need to have a per-entity-kind mapping, which would be more complex in its creation. E.g. annotations would be 0 for Class while it is 1 for DataProperty.
• Since only the field key numbers from 0-23 use 1 byte, and the ones after that 2 bytes, the mapping should prioritize the most common field names. Most common might mean either the field names that appear in most entity kinds (e.g. annotations), or field names that appear in entity kinds that are used very often (e.g. all field names in DataProperty).