hyperhyperspace / hyperhyperspace-core Goto Github PK
View Code? Open in Web Editor NEWA library to create p2p applications, using the browser as a full peer.
Home Page: https://www.hyperhyperspace.org
License: MIT License
A library to create p2p applications, using the browser as a full peer.
Home Page: https://www.hyperhyperspace.org
License: MIT License
At P4P Berlin we talked about various protocols that partially overlap in goals with HyperHyperSpace last weekend. Would be nice to have a more detailed comparison of how HHS fits into that and compares in features with other protocols! Does such a comparison exist already?
Also, I remember two years ago at Ink&Switch Unconf in Berlin we had a discussion about how to kick someone out of a group, and then yesterday this exact topic came up at the Local First online meetup:
https://youtu.be/Fm_BlBm1gTw?t=1471
How do those "signature chains" compare to what HHS does? Would love to chat more about this!
Right now it is not possible to indicate that some mutation operations can safely be ignored. I'd like to add another special field to MutationOp
that is similar to prevOps
but, instead of indicating that this op comes after some others, it indicates that such previous ops are obsolete and that they should not be fetched.
The sync agent should take this field into consideration when backtracking from the current state, and don't ask for ops that have been obsoleted.
Since in the most common case this cannot be done without breaking BFT, try to do it in a way that doesn't impose the burden of checking that there is no op-obsolescence on all the types that don't support it (maybe a parameter passed to MutableObject
's constructor enabling obsolescence?).
I ran across a situation where I couldn't delete stuff from a CausalArray
. This was in the wiki for blocks in a block list and pages in a page list.
Choose your own public key crypto framework.
Currently HHS supports two storage backends:
It would be nice to have a backend over a proper storage engine (LevelDB or LevelUp seem like promising candidates) available when running in Node.
Tests for the storage module are here. The tests can be run by doing
yarn test
Right now, Identity
objects are HashedObject
derivatives and are stored as any other object. They contain some information about the holder of the identity -just an info map, I usually use just two fields (info.name
and info.type
) but it's really application defined- and a key pair. Whenever an object is assigned an author (by means of obj.setAuthor(id)
, when it is saved in the store that id
object will be looked up in the store and used to cryptographically sign obj
. Hence the store is functioning as an identity & key store as well. A trick is used to prevent the private part of the key pair being synchronized whenever the identity is sent to other peers: the hash of the private part is replaced by a custom computation that's done on-the-fly, so that the Identity
object has no hash-references pointing to the private key that would make the synchronizer to actually send it.
I'd like to move the identity / signing part to a different module, and allow for the application to impose limitations to what things can be signed with each loaded identity. I'd also like to make this more explicit, and not rely on any exceptional behavior.
Right now, whenever obj.loadAllChanges()
is called, all the mutations are loaded from the store and applied. It'd be very convenient to enable mutable types to save a checkpoint to the store when they so choose, and having loadAllChanges()
automatically use the most recent checkpoint, having to apply only those mutations that happened after that point.
The new mutation events should allow a correct implementation of sync'ing in recursive mode.
Hello, I was wondering if you have any plans to provide an ES module that could be requested via a CDN?
I am having trouble building this on my local machine because it looks like the wrtc
library only runs on Node 14 and under. I am on a machine that is running Node 15 currently.
This library looks fantastic and would love to explore it more!
Even though the real sync state of objects cannot be known (because of network availability, gossip propagation time, etc.), we should make the discovered state observable from outside the sync agent.
(Don't forget to update the react bindings as well)
Stopping & starting the sync of a space (for example by choosing "No" in the Sync column, then "Yes" again) seems to leave the space in a non-sync'ing state (or at least that what seems by looking at the UI)
Adding JSDoc annotations and autogenerated API documentation would be really helpful.
I'm really excited to be learning HHS, as I can see a lot of great, very intelligent, work has gone into it already. From reading the documentation it sounds like it will work for the app I'm building. Now I'm trying to understand how I would use HHS to implement the data model for my app. I've used many different databases but nothing like p2p. I'm very new to this space (no depth of knowledge of Merkle-DAG's, spanning trees, WebRTC etc...but I get graphs). I'm trying to bridge the gap between what I want to do and how HHC works. Here's what I've been able to piece together after a few days with a fresh set of eyes. My apologies in advance if I missed anything obvious in the docs. If anyone is able to confirm or correct my understanding and point me in the right direction I would really appreciate it! Open source examples are always welcome.
I can put my static SPA app on a CDN and peers will sync app data among themselves. A signaling server is needed to aid in this process, but load on the server is not high. HHS is like a toolbox with tools to enable this.
A Store is an abstract for actual data persistence to a peer, for instance IndexedDB in the browser. An app's data model is made up of many interlinked spaces. Sync logic takes data in a 'Space' and updates the store on a peer using the verify function of each class. The full graph of spaces is represented across the network of peers. Individual peers only have the spaces they edited or requested. Access is granted or revoked using the CausalSet. The whitepaper says CapabilitySet but I think the API has changed since it was written.
Spaces can be nested within other spaces by referencing those spaces. Spaces can be discovered independently (of any particular app) via a 3-word code. A space is an abstract for exactly one grouping of data, "a chat room, an article, a blog, an ecommerce store, etc." A class (written by a dev) defines that grouping of data, a potential combination of literals and references to other spaces/class instances, based on business logic. A space is implemented when a class inherits from either an immutable HashedObject or a mutable abstract, ie MutableSet which is a MutableObject.
When to use one or the other, I'm not sure. Seems like a good use-case for HashedObject would be an error log. Or would an error log be represented as a HashedObject nested inside a MutableSet? What are some good use-cases for an immutable space? Will most app data be mutable?
"They can be universally looked up using 3-word codes, like suburb-suburb-awake." If a space really is a class instance, does that mean every single class instance needs a 3-word code? Is this what the Shuffle class is for? If I code a BlogSet class and an Article class, and there are 3 articles in the blog, does that mean there must be 4 spaces total for the blog? Is there ever a situation where the blog would be just one space?
It seems like an application data model would represent a graph, with the application (for a particular user) pointing to some logical root. Various spaces/classes would then be organized under the root in a way that makes sense according to the business logic. Each user would have various access to all or parts of this node. ie.
App Data (for Acme Employee: Bob) =>
Acme Corp {
Meta: {orgSpace: "acme-corp-best"}
EmployeeSet -> Employee: {name, startDate, title, boss...}
CustomerSet -> Customer: {name, address...}
OrderSet -> Order: {date, total, Lines -> LineItem: {product, qty, price, amount, total}, customer...}
ProductSet -> Product: {name, priceSet...}
OutsideMeetingSet -> Meeting: {mtgDateTime, discussion, Invites -> Invite:{orgSpace: "abc-corp-great"}}
}
*Bob may be granted access to all or parts of this graph depending on his role at Acme.
*This is not the app I want to build, just an example.
"Peers in the network form application-defined groups over which mutable objects are synchronized. The method for obtaining peers is also application defined..." I'm not sure what this means. Let's say both Acme Corp ("acme-corp-best") and ABC Corp ("abc-corp-great") are pointing to their respective Org graphs. Does HHS have logic, or does the app need custom logic, to make sure Acme employees are primarily trying to sync with other Acme employees and not ABC employees, other than for shared data like an OutsideMeeting discussion Forum which both companies can access? How does an app define peer groups? Implicitly? Or is there example code for this?
Again, great work on HHC and thank you to anyone who can help confirm/deny assumptions and answer questions. It looks very promising!
Dump and restore Hyper Hyper Space spaces to and from JSON literals.
So starting from the space's entry point, it should be possible to dump all its internal state into a single JSON-encoded string. The JSON should have enough metadata to recover the space without any additional information. This is fairly similar to what Context
and specially LiteralContext
do. It may be easier to go through a (memory-backed?) store to do the saving, and then using its contents to populate a LiteralContext
to be included in the generated JSON. For loading it up back again, maybe it would be easier to load the contents of the LiteralContext
back into a store (may be optional - if omitted we could just create a memory-backed one) than attempting to populate all the objects recursively automatically.
The generated JSON could be a first file format for importing / exporting spaces to and from files.
When hashing a public key in the process of creating an identity, remove all whitespace / newlines before doing the actual hash, in order to make the process deterministic.
Hi, just read the white paper, and this project seems great.
There are a lot of similar p2p projects with similar goals and principles currently going on.
I suggest we list here similar projects and discuss the differences, overlaps and potential for collaboration & integration. After all, interoperability in one ecosystem is worthless if its only another isolated platform that compete without i nteroperability with each others.
Off the top of my head:
If no peer group is specified in the MeshNode
, a default peer group is created per-object. Add support for accessing the default peer group for a given object, and for changing the default peer group per node (e.g. after setting, and if no peer group is supplied to sync, that peer group should be used).
would you please be openminded to put a code block near the top of the readme which explains how one would set this up and use it?
that would make it easier to evaluate and understand the API
Maybe it's possible to use the libp2p pubsub://TOPIC/
interface in agregore as a replacement for WebRTC / centralized peer discovery?
Some resources:
Types:
MutableSet
MutableReference
MutableArray
GrowOnlySet
CausalReference
CausalSet
CausalArray
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.