JSON event length is limited about plezi HOT 10 CLOSED

Hi @uidzip ,

Thank you very much for opening this issue.

This might be a facil.io issue effecting iodine (the websocket parser). I will have to run some tests to check this out... however, it seems like a very odd limit (156 bytes), considering the three parsing modes (up to 127 bytes, between 128B and 64KiB and over 64KiB).

The data also seems (I assume) garbled, as if it wasn't decoded properly.

I'll try to push a fix as soon as I can, but it's a busy couple of weeks (midterms).

It might be related to the new(er) Websocket parser, in which case it's possible that a rollback to iodine version 0.4.8 will fix the issue (though it this will re-expose the child-reaping concern, as listed in the changelog).

Thanks again for opening the issue!
Bo.

from plezi.

Update:

I tracked down the issue (a typo in the Websocket parser, which resulted in the parser being offset by 2 bytes).

I just finished testing a fix and releasing a patched iodine version.

As an added bonus (though you probably don't need it), the 0.4.10 version includes a High Sierra fix for fork in the Ruby runtime (due to be released in the next version of Ruby), as explained here and as patched here (you can also see this blog entry).

Please upgrade and let me know it you have any other issues.

Thanks again for opening this issue!!!
👍🏻🙏🏻

from plezi.

Excellent, thanks! If it had been a ruby issue I would have done a pull request, but since it seems to have been in the C code, it probably would have taken me weeks to figure it out. :) I can confirm that iodine 0.4.10 fixes the issue.

from plezi.

@uidzip - thanks for keeping me posted and confirming the fix :-)

from plezi.

Still having a problem with this. If I have an event as such: client.emit({ event: 'send_file', message; 'x'.repeat(86) }) iodine is losing / not sending the data to plezi. It loops infinitely spitting out "AutoDispatch warning: Received non-JSON message. Closing connection." If you change the 86 to an 85, it works. In fact, only the range 85-91 does not work, above 91 works fine!

I changed plezi's controller.rb as follows to try to see what it is getting as follows:

   rescue => e
        puts 'AutoDispatch Warnnig: Received non-JSON message. Closing Connection.'
        puts "#{e.class}: #{e.to_s}"
        puts "data: #{data}"
        puts "length: #{data.length}"
        close
        Kernel.exit! # prevent endless loop
        return
     end

The output with the above event with message a string between 86 and 91 characters inclusive is:

AutoDispatch Warnnig: Received non-JSON message. Closing Connection.
JSON::ParserError: A JSON text must at least contain two octets!
data:
length: 0

from plezi.

I feel pretty confident this is still in websocket_parser.h or websockets.c in iodine. Socket message length 119 works, 126 works, but 120-125 does not. The length of that range being 6, I think it has to do with the memcpy() or websocket_xmask(), but its not obvious to me exactly how to fix it, I'll keep working on it unless I hear from you :) My C foo is pretty rusty.

from plezi.

@uidzip,

Thank you again for your update. I'll reopen the issue and try to see what's going on. I'm pretty sure you're right about it originating from the websocket_parser.h. I think there might be "magic numbers" that cause the parser to fire "on_data" events to no end. I'm not sure, but I'll work on it tonight and see what can be done.

B.

from plezi.

I've released an update after testing it on my machine (I ran messages of length 0...399 and a message of 74K, all okay).

The issue was caused by my use of len (buffer length) instead of len_indicator (message length indication) in one of the if statements... bad naming leads to @#!... but I fixed some of the logic while I was at it, as I was able to get rid of an if statement and optimize some other minor concerns.

Thank you so much for posting the issue!

Please let me know if this fixed it for you.

from plezi.

Looks like that did it, I can't make it fail now. I knew it had something to do with how websocket_consume() was being called, but I wasn't seeing it. The loop was websocket_consume() not consuming anything, I think. Thanks!

By the way, do you have any ideas on a concurrent-ruby actor-model integration with plezi that doesn't require separate processes and shunting all the messaging through redis? That is the architecture I have come up with, but it doesn't feel right to have two separate programs and processes. Maybe its OK though, it does work quite well.

from plezi.

Looks like that did it, I can't make it fail now.

🎊🎉👏🏻👍🏻

any ideas on a concurrent-ruby actor-model integration with plezi that doesn't require separate processes...

This is a bit broad for me to work out a simple answer. I guess it depends on your implementation requirements.

From what I understand, the "actor-model" basically means the use of "messages" rather than directly effecting mutable objects. Even within the actor model, locks are "hidden away" within an actor's internal state or within the message system's design - although this might be considered to break the strict actor model, once an actor has a mutable internal state, a lock (or verification system) is often required to either prevent internal data corruption or to prevent message handling concurrency.

Celluloid has (had?) an actor based framework... I don't know if it will work with Plezi, though.

I assume you could easily implement an Actor based design using Iodine.defer to send asynchronous messages to thread-safe singleton classes.

If you're using Redis messages, you could use Iodine.subscribe to create a global subscription to a channel... note that messages might be performed concurrently, requiring an internal lock. This is different than connection subscriptions and a connection's #defer method, since these methods lock the connection and require a connection to handle messages in a serial (instead of concurrent) manner.

Note that iodine's internal Pub/Sub engine only requires Redis once you scale beyond the scope process cluster (if you start iodine with 8 processes, pub/sub will work between processes, but if you add an extra machine, than you need Redis, since there's no internal Gossip protocol at the moment).

As you can tell from my roundabout answer, I have no idea what your needs are, so I find it difficult to offer a hand.

from plezi.