Comments (16)
@cblgh I think I understand the [understandable] confusion! My understanding of the situation is this:
Right now, all deletes are propagated using the Channel Time Range Request
: chat message deletes, but also deletions of topic edits, joins, and parts. I think @cryptix is asking why Channel State Request
will provide NEW topics but not info on DELETED topics. I agree that it seems strange, since deletions of topics is also conceptually part of channel state.
(I'm going to constrain my responses on this issue to just here, rather than discuss partly here & partly on the pub cabal.)
from cable.
Great questions cryptix!
Why aren't
post/delete
s part of channel state replies?The way i'm seeing it, getting for e.g. two topics without the delete in the middle, where the 2nd/older topic would have non-recent links, would require a certain amount of trust to the peer, no?
Adding the delete in the middle would at least show "well this thing is gone" now and maybe i still have that old topic anyway.
This is a good point. I like the idea of this request returning deletions relevant to the channel state. Curious if @cblgh has any feedback re: the implementation complexity of such an edit.
tangentially related: can all post's be deleted?
Right now one would need to issue a post/delete
with EVERY HASH THEY EVER CREATED. This is unwieldy, and there's been discussion of adding another post type or augmenting this post type with a "time range" option for deletes.
If i create a new channel and then delete the join, what's the most
recent state? empty channel? no channel? if somebody else joins and
i join again, should i reference the delete?
Great Q! The spec definitely doesn't address this case.
If we say that it deletes the channel, then as the post deletion propagated, everyone would eventually delete that channel. It is an extra bit of implementation logic though, to check for this case.
If we say that that the channel remains, I could see that being a bit weird from a user perspective -- having an empty channel with no members.
More broadly then, if all posts in a channel are deleted, does that channel remain or be destroyed? I lean toward considering the channel not existing.
from cable.
Proposals for spec (and thus implementation) changes:
- Clarify that if all posts in a channel are deleted such that the clauses in 5.4 Channels are no longer true, the channel no longer exists either.
- Specify that
Channel Time Range Request
returnspost/delete
s only forpost/text
posts, andChannel State Request
returnspost/delete
s forpost/join
,post/leave
, andpost/topic
.
cc @cblgh @cryptix @mycognosist
from cable.
In response to the first suggested spec change: I don't have a strong preference either way. It sort of feels like an implementation detail to me (whether the channel is considered empty or non-existent) but I can see that there are side-effects - for example, inclusion of empty channels in channel list requests (where it might be preferable not to include empty channels at all).
Regarding the second proposed change: I am still thinking through the ramifications. I think @cblgh raises an important point about the lack of a time-based limiting mechanism for channel state requests. Need to ferment this some more before I can offer further input.
from cable.
My concern:
Channel State Request
lacks a history limiting mechanism. With the proposed change we could have a really old channel whose response to aChannel State Request
blasts a metric ton of hashes for deletes of posts that nobody actually even has anymore; we are stuck returning thosepost/delete
hashes forever for that channel. This kind of everlasting channel state history is a complaint I've seen aired wrt Matrix so I think it is something to be aware about.
Channel Time Range Request
's time range enables graceful loss of such hashes over time.One solution I'm playing with in the request I'm introducing in the moderation work is a field
oldest
, it has this description currently:
since i'm on the soapbox, might as well proselytize the magical power of tangles and predsl.
for example, Staltz's new p2p data structures with a history limiting mechanism,
but also sorry if this isn't helpful, feel free to ignore me! π»
from cable.
@cblgh Yes, ππ» to your rephrase.
from cable.
given that the verification requires to have the original message (to check that the public key matches), i could see that as an motivation not to hold onto them in any case.
from cable.
Why aren't
post/delete
s part of channel state replies?The way i'm seeing it, getting for e.g. two topics without the delete in the middle, where the 2nd/older topic would have non-recent links, would require a certain amount of trust to the peer, no?
Adding the delete in the middle would at least show "well this thing is gone" now and maybe i still have that old topic anyway.This is a good point. I like the idea of this request returning deletions relevant to the channel state. Curious if @cblgh has any feedback re: the implementation complexity of such an edit.
As indicated in the public cabal I don't quite understand the question being posed. Can someone that understands it rephrase it? I really don't grok where delete is getting into the picture in the quoted paragraph of the opening post
My flawed attempt at understanding it, demonstrated with a scenario:
Let's say we have two users that are setting a topic of a channel, User A and user B. User A posts post/topic
(call it topic 1
) to channel C. User B then posts another post/topic
(call it topic 2
) to channel C.
My read of what @cryptix was saying was that someone requesting channel state for channel C would benefit from peers verifiably deleting the obsolete topic, topic 1
, with a post/delete
. (How off the mark is this @cryptix? ^^')
But since the only posts a user may delete are those posts they have authored themselves, this only makes sense for the case where a single user has posted two topics in a row for the same channel (and as such may delete the first post/topic
of the two because they authored it). In my posited scenario, the only user who may delete topic 1
is user A (the author of the post setting topic 1
), which is why I think I am misunderstanding something and requested a more complete example over the public cabal X)
cc @hackergrrl
from cable.
ππ» on the proposed changed. Seems much more consistent.
why
Channel State Request
will provide NEW topics but not info on DELETED topics
yup, that part. Holding on to the deletes should keep the links tree intact, i think and make clear that a peer isn't lying or just sending out-of-date info..?
At first i thought about not keeping track of deletes at all but now I'm not 100% sure that would be correct.. in a sense those are only important for people that saw the message referenced in a delete?
can all post's be deleted?
why are words so hard..? π that should have been: can all types of post's be deleted? but I think you also covered that in your change proposal now, @hackergrrl.
from cable.
Proposals for spec (and thus implementation) changes:
1. Clarify that if all posts in a channel are deleted such that the clauses in _5.4 Channels_ are no longer true, the channel no longer exists either.
Seems reasonable!
2. Specify that `Channel Time Range Request` returns `post/delete`s only for `post/text` posts, and `Channel State Request` returns `post/delete`s for `post/join`, `post/leave`, and `post/topic`.
I have qualms!
My concern: Channel State Request
lacks a history limiting mechanism. With the proposed change we could have a really old channel whose response to a Channel State Request
blasts a metric ton of hashes for deletes of posts that nobody actually even has anymore; we are stuck returning those post/delete
hashes forever for that channel. This kind of everlasting channel state history is a complaint I've seen aired wrt Matrix so I think it is something to be aware about.
Channel Time Range Request
's time range enables graceful loss of such hashes over time.
One solution I'm playing with in the request I'm introducing in the moderation work is a field oldest
, it has this description currently:
Field oldest is a time, expressed in milliseconds since UNIX Epoch, limiting the amount of history sent if set. If oldest > 0 then the posts whose hashes are being returned should fulfill post.timestamp >= oldest. Implementations are recommended to set a large value for oldest, for example on the order of 1 year from the time of requesting: oldest = - 31536000000.
Though this mechanism may not play well with channel membership, specifically in the case of post/leave
, as well as retrieving post/info
for old-ass nicknames.
from cable.
Agreed that we shouldn't let the state grow uncontrollably! The more I think about it the more I'm leaning towards not storing post/delete
s at all. Use and forget, done.
Forwarding them to "live" (future=1
) requests seems sensible to me to, still. I'd like to ensure a peer is not being withheld information or being gamed by a malicious peer, trying to suppress things.
How do other's feel about the holes in the link dag? It could be that my fear there is purely academic. I guess they would heal? A few examples would help me. I will try to see I've I can make one or two.
from cable.
Forwarding them to "live" (future=1) requests seems sensible to me to, still.
Agreed. Ensuring the post/delete
has been passed on to at least one peer before forgetting sounds desirable to me. Deletions (the right to be forgotten) seem to be a very important part of the protocol and I wouldn't want to take actions which degraded their efficacy in some way. Keen to hear what @hackergrrl and @cblgh think about it.
How do other's feel about the holes in the link dag?
Since links are primarily intended to provide a causal proof for post ordering - and assuming that non-deleted post/text
s will greatly outnumber deleted post/text
s - I don't see it as a major problem. I also think it would be useful to think through with some examples.
I haven't implemented links fully in cable.rs
yet; @cblgh will likely have a more informed opinion.
from cable.
i was just thinking about the Cable spec and realized the implications of how deletes are implemented (if i understand correctly): a delete removes the entire message (including link metadata), not just the content. versus say bamboo or ppppp.
How do other's feel about the holes in the link dag?
personally, i'd prefer deletes removed the content and not the link metadata, so a delete couldn't affect causality. i love tangles and i too probably have a kink for technical purity. π
from cable.
@cryptix wrote:
The more I think about it the more I'm leaning towards not storing post/deletes at all. Use and forget, done.
@mycognosist wrote:
Ensuring the post/delete has been passed on to at least one peer before forgetting sounds desirable to me.
π
It's important to me that deletions are very durable, and are a part of the eventually consistent state. Simultaneously, I definitely agree with y'all that we should be very judicious about not having any responses be grow-only sets! Thanks for pointing that out @cblgh.
Having thought about it again, I like the idea of ALL post/delete
s coming in through Channel Time Range Request
. The way I'm thinking about it in my head is
Channel State Request
: "please give me the latest state info so I don't need to dig deep into historic posts"Channel Time Range Request
: "what new stuff has happened in this channel since$TIME
?"
Fortunately this is already how Channel Time Range Request
works! Although it uses some vague-as-heck language, saying " SHOULD include the hashes of all known post/text and post/delete posts made to a channel" which is silly because post/delete
has no channel
field. :)
So, a new edit proposal:
- Revise
Channel Time Range Request
to be very explicit about exactly what ought to be returned. Here is, I think, everything it ought to return, within the time range given:- All
post/text
- All
post/delete
that pertain to{post/text, post/topic}
for this channel - All
post/delete
that pertain to a member or ex-member of this channel - If the client doesn't know what kind of
post/*
type a delete pertains to, return it here too.
- All
(2) means that deletes will generally not propagate for deletions of post/{join,leave,delete}
.
(3) means that duplicate hashes can be returned if Alice has other channels in common with Bob if he deletes one of his post/info
posts. I think this is ok, since clients are expected to deduplicate & not re-request known posts. Fancy client implementations could deduplicate at the socket level and know not to send that hash down all live Channel Time Range Request
s, too.
(4) This is for the case when a client sees a post/delete
for a post they don't have: they can't tell if it's a join or a topic or a chat msg or something else. This will actually also happen if a client fully deletes the post in question locally without a tombstone: they won't know what ANY deletions are about when someone makes a Channel Time Range Request
. :)
from cable.
@cblgh @mycognosist @cryptix I'm going to make these edits pretty soon if there are no objections or calls for further discussion.
from cable.
So, a new edit proposal:
* Revise `Channel Time Range Request` to be very explicit about exactly what ought to be returned. Here is, I think, everything it ought to return, within the time range given: 1. All `post/text` 2. All `post/delete` that pertain to `{post/text, post/topic}` for this channel 3. All `post/delete` that pertain to a member or ex-member of this channel 4. If the client doesn't know what kind of `post/*` type a delete pertains to, return it here too.
(2) means that deletes will generally not propagate for deletions of
post/{join,leave,delete}
.(3) means that duplicate hashes can be returned if Alice has other channels in common with Bob if he deletes one of his
post/info
posts. I think this is ok, since clients are expected to deduplicate & not re-request known posts. Fancy client implementations could deduplicate at the socket level and know not to send that hash down all liveChannel Time Range Request
s, too.(4) This is for the case when a client sees a
post/delete
for a post they don't have: they can't tell if it's a join or a topic or a chat msg or something else. This will actually also happen if a client fully deletes the post in question locally without a tombstone: they won't know what ANY deletions are about when someone makes aChannel Time Range Request
. :)
@hackergrrl A big ol π from meβthese revisions will help implementors a lot! I'll have to revisit my logic and the indexing to be certain, but I think these are roughly the interpretations glyph & I landed in when doing our respective implementations :) Once this lands I'll make time to patch the places in cable-core.js that don't conform if any.
To clarify my own understanding by rephrasing - does the revision essentially say:
"A channel time range request, for a given time range, should return all hashes for:"
post/text
posted in the channelpost/delete
targeting apost/text
orpost/topic
posted to the channelpost/delete
targeting any post type authored by a member/ex-member of the channel (i.e. includes delete ofpost/join
,post/info
,post/leave
)post/delete
for any unknown post types / posts we don't have knowledge about
from cable.
Related Issues (13)
- Some Questions HOT 1
- Questions & Comments HOT 1
- post header: move timestamp to be header field 5 HOT 1
- after v1 review: create test vectors to facilitate interoperating implementations
- SHA-256 or SHA-512 when signing? HOT 3
- Handshake: Post-Handshake Operation missing end-of-stream marker to protect against stream truncation HOT 6
- Eventual consistency issue + proposed solution
- Time? Whose? HOT 4
- what kind of varint HOT 1
- reusable circuits
- sub-protocols
- add version number query HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cable.