y-crdt / yrb-actioncable Goto Github PK
View Code? Open in Web Editor NEWAn ActionCable companion for Y.js clients.
Home Page: https://y-crdt.github.io/yrb-actioncable/
License: MIT License
An ActionCable companion for Y.js clients.
Home Page: https://y-crdt.github.io/yrb-actioncable/
License: MIT License
Hi - thank you for sharing this code.
I have multiple editors on one page.
If _proto.connectBc = function connectBc()
runs, then they both end up broadcasting to each other, so they end up identical. But I want two separate ydocs with different contents.
I think that because it's only using the channel name inside that function, so it can't differentiate between different paths in the publish call.
Not sure - I don't really understand any of this. However, changing this fixed my issue.
I'm looking into using yrb-actioncable with my rails app, but am blocked on the requirement to use yarn… Is there a way to use this project with import maps, which is the standard way to load JavaScript in a rails app since 7.0?
The only adapter supported is Redis
, but we should have a configurable test
adapter like Rails ActionCable itself, to make it easier to test sync (and reliable sync), and also reduce a test dependency on a running Redis instance.
Hi, thanks for making this gem!
I'm working on a collaborative IDE of sorts and I'm running into a problem loading initial content from the server.
I'm proactively checking in pages#index
if content exists in redis (in the example, it just returns if full_state nil), and if it's not present, I set it, retrieve it and return it – but that hasn't worked. I noticed two issues.
sessions:id
where id is the path, and the id of the session for the sync channel is something like sync:channel-syncchannel:id-hello2
so the index controller will always have full_state
as nil and @editor_content
will always be ""
– is this supposed to be the case? If so why?Thanks so much for your help! 🙏
We are currently storing document data with Redis as the destination, using the EXAMPLE example as a guide.
Is there any way to extract only the input string information from the binary data exchanged by y.js?
I am planning to store the data in mysql as well as Redis, using batches and such. However, the data used in this library is binary information that follows the y.js protocol, so I would like to handle it in a simple markdown-like format.
Also, I'm assuming that when I return values from mysql to editor, I need to re-convert them to a form that follows the y.js protocol. If you know how to do that, please let me know.
I encountered an issue while using Y::Lib0::Decoding.read_var_uint8_array
, where pos does not change, preventing successful sequential reading of strings.
I suspect the solution might involve adapting and fixing the part decoder.pos += len
from https://github.com/dmonad/lib0/blob/616c7cc61c0bc03d1419bb347977259673ca8aa3/decoding.js#L104.
js:
https://github.com/dmonad/lib0/blob/616c7cc61c0bc03d1419bb347977259673ca8aa3/decoding.js#L104
export const readUint8Array = (decoder, len) => {
const view = new Uint8Array(decoder.arr.buffer, decoder.pos + decoder.arr.byteOffset, len)
decoder.pos += len
return view
}
ruby:
https://github.com/y-crdt/yrb-actioncable/blob/358efdce316042d4261ced4caaed29e8a4b85252/gems/yrb-actioncable/lib/y/lib0/decoding.rb#L29-L31
def self.read_uint8_array(decoder, size)
view = Buffer.create_uint8_array_view_from_buffer(decoder.arr, decoder.pos + 0, size)
end
doc = Y::Doc.new
text = doc.get_text("text")
text << "content"
p doc.diff
# => [1, 1, 219, 249, 254, 144, 12, 0, 4, 1, 4, 116, 101, 120, 116, 7, 99, 111, 110, 116, 101, 110, 116, 0]
decoder = Y::Lib0::Decoding::Decoder.new(doc.diff)
Y::Lib0::Decoding.read_var_uint(decoder) #=> 1 (numOfStateUpdates)
Y::Lib0::Decoding.read_var_uint(decoder) #=> 1 (numberOfStructs)
Y::Lib0::Decoding.read_var_uint(decoder) #=> 3256859867 (client)
Y::Lib0::Decoding.read_var_uint(decoder) #=> 0 (clock)
Y::Lib0::Decoding.read_var_uint(decoder) #=> 4 (info(String))
Y::Lib0::Decoding.read_var_uint(decoder) #=> 1 (parent info)
# first, read "text"
array = Y::Lib0::Decoding.read_var_uint8_array(decoder)
p array
#=> [116, 101, 120, 116]
p array.pack("C*")
#=> "text"
# **Problem**
# **Now, we should increment `pos` manually**
decoder.pos += array.size
# Now, we can read next content "content"
array = Y::Lib0::Decoding.read_var_uint8_array(decoder)
p array
#=> [99, 111, 110, 116, 101, 110, 116]
p array.pack("C*")
#=> "content"
A reliable channel ensures at-least-once
guarantees via acknolwedge
messages sent by both, client and server. Therefore, the client, and the server must reliably store messages that weren't acknowledged by all recipients yet. In case of the client, this means the server must respond with a last-ack-id
response when messages are sent, and the client must always send the full message queue to workaround missing “in-between” updates.
The default WebsocketProvider
does not support message buffers and acknowledgment of messages. The ReliableWebsocketProvider
is therefore a drop-in replacement that takes on the additional responsibilities of transparently managing a send buffer
, and acknowledging
message retrieval.
Due to Y.js supporting staging of messages (messages are not integrated until all necessary messages are available), and applying updates being idempotent, this isn't an issue.
sequenceDiagram
Client->>+Server: Send message buffer to server
alt no response
loop Retry (w/ exponential backoff)
Client-->>Server: Send message buffer
end
else
Server-->>-Client: Acknowledge message retrieval
end
We aim for eventual consistency (at-least-once
delivery), and similar transport latency as with the regular WebsocketProvider
on the happy path (exactly-once
delivery). There is a chance of transmitting messages twice, when an “older” message wasn't transmitted, but the client is producing “new” messages.
When Node.js cannot be detected, but the app is server-side rendered, the following import statement fails due to initializing an instance of LocalStorage
. We should introduce an SSR flag (mode) and re-implement the broadcastchannel
with some lazy initialization for the LocalStorage
, so that it will only be created when disableBc = false
.
import { publish, subscribe, unsubscribe } from 'lib0/broadcastchannel';
First of all, thank you for the great gem.
We are trying to use this gem in a closed beta product. We will actually offer this functionality to our users on a trial basis, although the number of users using it is small.
So I have a request. I would like you to release a new version (0.1.6 or patch version) including the following commits Without this change, we will get errors in name resolution on CI.
#41
The awareness development is only left as a TODO, so it is impossible to track when the client side disconnects from the socket. I was wondering if you are aware of this issue and if so, are you planning to develop it, and if not, can I raise a pull request for it?
Thanks for developing a great gem.
And thanks for answering some of my previous questions.
We have been using this gem for the past few months as a beta version and have found cases where messages are lost under certain conditions.
First, some information about our environment:
Ruby on Rails: 7.0.4
Ruby: 3.2.1
Backend Data Source: Redis
Frontend Frame Work: Next.js (We use y-actioncable npm package in app.)
Our scripts. For debug, I put some loggers.
class DocumentChannel < ApplicationCable::Channel
include Y::Actioncable::Sync
def subscribed
reject unless document
sync_for(document) do |id, update|
redis.save(id, update)
end
end
def unsubscribed
stop_stream_from canonical_channel_key
end
def receive(message)
sync_to(document, message)
end
def doc
@doc ||= load do |id|
redis.load(id)
end
end
def document
@document ||= organization.documents.find(params[:document_id])
end
def redis
@redis ||= Yrb::Editor::Redis.new
end
end
module Yrb
module Editor
class Redis
attr_reader :client, :logger
def initialize
@client ||= ::Redis.new(url: Rails.application.config.x.redis_url)
@logger = SemanticLogger[self.class.to_s]
end
def save(id, update)
begin
data = encode(update)
doc = client.set(id, data)
logger.info("Yrb::Editor::Redis Finish Save", name: self.class.to_s, id:)
doc
rescue ::Redis::BaseError => e
logger.error("Yrb::Editor::Redis Failed to Save", name: self.class.to_s, inspect: e.inspect, message: e.message)
end
end
def load(id)
begin
data = client.get(id)
if data.present?
logger.info("Yrb::Editor::Redis Finish Load", name: self.class.to_s, id:)
decode(data) unless data.nil?
else
logger.error("Yrb::Editor::Redis Not Found", name: self.class.to_s, id:)
nil
end
rescue ::Redis::BaseError => e
logger.error("Yrb::Editor::Redis Failed to Load", name: self.class.to_s, inspect: e.inspect, message: e.message)
end
end
def encode(update)
update.pack("C*")
end
def decode(data)
data.unpack("C*") unless data.nil?
end
end
end
end
In the video above, the message is saved correctly for normal input, but after a large amount of input, it is lost after the page is reloaded. (Lines of text beginning with "e" are missing.)
Could #25 be related to this issue?
If you want any information to understand the issue, please let us know.
Thank for reading.
To allow the extension of the Sync
module with e.g., Reliable
messaging, there need to be defined hooks/callbacks. We can use them to transform behavior (e.g., store sent messages in buffer, move client tracker, send acknowledge messages to client, etc.).
ActionCable
like callbacks to Sync (like subscribed
, unsubscribed
).Reliable
for Sync
)A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.