yrashk / socket.io-erlang Goto Github PK

The disconnect event for everything except WebSockets is never sent.

I'm trying to run make, but it seems that git update submodule fails:

$ git submodule update
Cloning into priv/Socket.IO...
fatal: https://github.com/LearnBoost/Socket.IO.git/info/refs not found: did you run git update-server-info on the server?
Clone of 'https://github.com/LearnBoost/Socket.IO.git' into submodule path 'priv/Socket.IO' failed

also likely remove those socketio_request calls

I am not sure whether it is socket.io-erlang problem, though — but we can play with this to try to figure this out

=CRASH REPORT==== 24-Jun-2011::14:06:26 ===
crasher:
initial call: gen_event:init_it/6
pid: <0.27812.9>
registered_name: []
exception exit: {badarg,[{erlang,link,[{sslsocket,new_ssl,<0.27808.9>}]},{socketio_transport_polling,handle_cast,2},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]}
in function gen_event:terminate_server/4
in call from proc_lib:init_p_do_apply/3
ancestors: [<0.27811.9>,socketio_client_sup,socketio_listener_sup,socketio_listener_sup_sup,socketio_sup,<0.27571.9>]
messages: []
links: []
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 850
neighbours:

=SUPERVISOR REPORT==== 24-Jun-2011::14:06:26 ===
Supervisor: {local,socketio_client_sup}
Context: child_terminated
Reason: {badarg,[{erlang,link,[{sslsocket,new_ssl,<0.27808.9>}]},{socketio_transport_polling,handle_cast,2},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]}
Offender: [{pid,<0.27811.9>},{name,socketio_client},{mfargs,{socketio_client,start_link,[<0.27614.9>,socketio_transport_polling,"797f75c6-66b4-4329-a063-243f697cf946",socketio_http_misultin,{xhr-polling,{misultin_req,{req,{sslsocket,new_ssl,<0.27808.9>},ssl,{192,168,1,252},55977,undefined,keep_alive,undefined,{1,1},GET,{abs_path,"/1/xhr-polling//1308924387179"},[],[{Host,"marketdata.kodiak.is:1024"},{Accept-Encoding,"gzip"},{Referer,"https://marketdata.kodiak.is:1024/tester"},{Accept-Language,"en-GB, en-US"},{"X-Wap-Profile","http://gsm.lge.com/html/gsm/P500-M6-D1.xml"},{User-Agent,"Mozilla/5.0 (Linux; U; Android 2.2; en-gb; LG-P500 Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1 MMS/LG-Android-MMS-V1.0/1.2"},{Cookie,"socketio=xhr-polling"},{Accept,"text/xml, text/html, application/xhtml+xml, image/png, text/plain, /;q=0.8"},{Accept-Charset,"utf-8, iso-8859-1, utf-16, *;q=0.7"}],undefined,<<>>},<0.27809.9>}}]}},{restart_type,transient},{shutdown,5000},{child_type,worker}]

https://github.com/LearnBoost/Socket.IO-node/blob/master/lib/socket.io/transports/xhr-multipart.js#L63 (blindly copied over to our implementation)

I need to close a (possibly malicious) client's connection cleanly - I can't figure out a way to accomplish that without causing crashes.

Is that right? Shouldn't it :reply instead?

TL;DR: When we're using 'handle_request/5' to deal with a request, the control of the socketio_http process is given up to the user and queries can get lost when doing a non-selective receive during quick successions of client requests

I used the handle_request/5 call and had it communicating with non-socket.io processes. As part of its task, the callback had to do a non-selective receive and digest everything it received in a while to handle them later (much like a gen_server would do it).

However, socketio_http runs the 'handle_request' function calls within its own process, so whenever socketio_http_misultin or some other instance calls the socketio_http server for the current connection more than once (can be simulated by reloading many tabs of the same browser really fast), the 'handle_request' currently running is the one getting the {request, 'GET', ...} message bypassing the socketio_http process altogether. The process model used in this case should be modified to avoid such issues.

In the meantime (and for external readers), I solved the issue by basically spawning a new process in handle_request, then monitoring it and waiting only for the monitor of that process to be done. During that time the other process can send the data to the socket as much as it wants, but we're forcing keep-alive/pipelined calls to be serialized when they're in the same section. This exact model should probably be moved from my own application to socket.io-erlang itself (to be discussed).

It seemingly executes connection callback

Implement xhr polling transport

There doesn't seem to be a way to catch the event that misultin receives when a client disconnects from the server.
This means that the xhr loop has to finish before a the reconnect timeout is set, causing the client's process to hang for the loop time (or what's left of it) + reconnect timeout before appearing to have left.

https://github.com/LearnBoost/Socket.IO-node/blob/master/lib/socket.io/transports/xhr-multipart.js#L18

When sending buffered messages, only some of them are received by the server.

Socket.io buffers up messages as such:
mlengthmMessage0mlengthmMessage1

It seems like only Message0 would be received here. I'm trying to confirm this atm.

We we're try to send unicode data to websocket, but had a crash with this error

** Generic server <0.2107.0> terminating
** Last message in was {websocket,
[126,109,126,50,49,126,109,126,126,106,126,123,34,
116,101,120,116,34,58,34,209,139,209,132,208,178,
208,176,208,178,209,139,208,176,34,125],
{misultin_ws,
{ws,#Port<0.5164>,http,false,
{10,1,5,51},
61776,undefined,
{'draft-hixie',76},
"http://localhost:7879","10.1.5.51:7879",
"/socket.io/websocket",
[{'Upgrade',"WebSocket"},
{'Connection',"Upgrade"},
{'Host',"10.1.5.51:7879"},
{"Origin","http://localhost:7879"},
{"Sec-Websocket-Key1",
"3U4 06< <86 0O0 ) 48"},
{"Sec-Websocket-Key2",
"3 6 8g. )r2 1 4 W_2,370"}],
false},
<0.2105.0>}}
** When Server state == {state,"e95bc077-5f92-4bf4-9bbc-ba98e4636d5a",
undefined,socketio_http_misultin,
{websocket,
{misultin_ws,
{ws,#Port<0.5164>,http,false,
{10,1,5,51},
61776,undefined,
{'draft-hixie',76},
"http://localhost:7879","10.1.5.51:7879",
"/socket.io/websocket",
[{'Upgrade',"WebSocket"},
{'Connection',"Upgrade"},
{'Host',"10.1.5.51:7879"},
{"Origin","http://localhost:7879"},
{"Sec-Websocket-Key1",
"3U4 06< <86 0O0 ) 48"},
{"Sec-Websocket-Key2","3 6 8g. )r2 1 4 W_2,370"}],
false},
<0.2105.0>}},
1,
{#Ref<0.0.0.4154>,10000},
<0.2108.0>,<0.82.0>}
** Reason for termination ==
** {badarg,[{jsx_eep0018,collect,3},
{socketio_data,json,2},
{socketio_transport_websocket,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]}

After insert this stupid code, every thing goes fine:

json(_Length1, Body) ->
Length = erlang:length(Body),
io:format("~n++++ Length ~p Body pn", [Length, Body]),
{Object, Rest} = lists:split(Length, Body),
io:format("~n++++ Object ~p Rest pn", [Object, Rest]),
[#msg{content=jsx:json_to_term(list_to_binary(Object), [{strict,false}]), json=true} |
handle_rest(Rest)].

socketio_data:json/2

I get this after a refresh:

=ERROR REPORT==== 8-Apr-2011::17:52:35 ===
** Generic server <0.73.0> terminating 
** Last message in was timeout
** When Server state == {state,"ff03a3ac-947b-4c56-a3ea-03150ec65133",
                         {misultin_req,
                          {req,#Port<0.4255>,http,
                           {127,0,0,1},
                           53645,undefined,keep_alive,undefined,
                           {1,1},
                           'GET',
                           {abs_path,"/socket.io/xhr-multipart/"},
                           [],
                           [{'Host',"localhost:7878"},
                            {'User-Agent',
                             "Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.2.16) Gecko/20110323 Ubuntu/10.10 (maverick) Firefox/3.6.16"},
                            {'Accept',
                             "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
                            {'Accept-Language',
                             "fr,fr-fr;q=0.8,en;q=0.6,en-us;q=0.4,it;q=0.2"},
                            {'Accept-Encoding',"gzip,deflate"},
                            {'Accept-Charset',
                             "ISO-8859-1,utf-8;q=0.7,*;q=0.7"},
                            {'Keep-Alive',"115"},
                            {'Connection',"keep-alive"},
                            {'Referer',"http://localhost:7878/"},
                            {'Cookie',"socketio=xhr-multipart"},
                            {'Pragma',"no-cache"},
                            {'Cache-Control',"no-cache"}],
                           <<>>},
                          <0.70.0>},
                         {<0.71.0>,#Ref<0.0.0.238>},
                         {'xhr-multipart',none},
                         9,10000,8000,<0.74.0>,<0.56.0>}
** Reason for termination == 
** {{function_clause,[{gen,call,
                           [{<0.71.0>,#Ref<0.0.0.238>},
                            '$gen_call',connection_gone,5000]},
                      {gen_server,call,2},
                      {socketio_transport_xhr_multipart,handle_info,2},
                      {gen_server,handle_msg,5},
                      {proc_lib,init_p_do_apply,3}]},
    {gen_server,call,[{<0.71.0>,#Ref<0.0.0.238>},connection_gone]}}

=CRASH REPORT==== 8-Apr-2011::17:52:35 ===
  crasher:
    initial call: socketio_transport_xhr_multipart:init/1
    pid: <0.73.0>
    registered_name: []
    exception exit: {{function_clause,
                         [{gen,call,
                              [{<0.71.0>,#Ref<0.0.0.238>},
                               '$gen_call',connection_gone,5000]},
                          {gen_server,call,2},
                          {socketio_transport_xhr_multipart,handle_info,2},
                          {gen_server,handle_msg,5},
                          {proc_lib,init_p_do_apply,3}]},
                     {gen_server,call,
                         [{<0.71.0>,#Ref<0.0.0.238>},connection_gone]}}
      in function  gen_server:terminate/6
    ancestors: [socketio_client_sup,socketio_listener_sup,
                  socketio_listener_sup_sup,socketio_sup,<0.53.0>]
    messages: []
    links: [<0.62.0>,<0.74.0>,<0.59.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 4181
    stack_size: 24
    reductions: 3990
  neighbours:

=CRASH REPORT==== 8-Apr-2011::17:52:35 ===
  crasher:
    initial call: gen_event:init_it/6
    pid: <0.74.0>
    registered_name: []
    exception exit: {{function_clause,
                         [{gen,call,
                              [{<0.71.0>,#Ref<0.0.0.238>},
                               '$gen_call',connection_gone,5000]},
                          {gen_server,call,2},
                          {socketio_transport_xhr_multipart,handle_info,2},
                          {gen_server,handle_msg,5},
                          {proc_lib,init_p_do_apply,3}]},
                     {gen_server,call,
                         [{<0.71.0>,#Ref<0.0.0.238>},connection_gone]}}
      in function  gen_event:terminate_server/4
    ancestors: [<0.73.0>,socketio_client_sup,socketio_listener_sup,
                  socketio_listener_sup_sup,socketio_sup,<0.53.0>]
    messages: []
    links: []
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 377
    stack_size: 24
    reductions: 137
  neighbours:

=SUPERVISOR REPORT==== 8-Apr-2011::17:52:35 ===
     Supervisor: {local,socketio_client_sup}
     Context:    child_terminated
     Reason:     {{function_clause,
                      [{gen,call,
                           [{<0.71.0>,#Ref<0.0.0.238>},
                            '$gen_call',connection_gone,5000]},
                       {gen_server,call,2},
                       {socketio_transport_xhr_multipart,handle_info,2},
                       {gen_server,handle_msg,5},
                       {proc_lib,init_p_do_apply,3}]},
                  {gen_server,call,
                      [{<0.71.0>,#Ref<0.0.0.238>},connection_gone]}}
     Offender:   [{pid,<0.73.0>},
                  {name,socketio_client},
                  {mfargs,
                   {socketio_client,start_link,
                    [<0.56.0>,socketio_transport_xhr_multipart,
                     "ff03a3ac-947b-4c56-a3ea-03150ec65133",
                     {'xhr-multipart',
                      {{misultin_req,
                        {req,#Port<0.4255>,http,
                         {127,0,0,1},
                         53645,undefined,keep_alive,undefined,
                         {1,1},
                         'GET',
                         {abs_path,"/socket.io/xhr-multipart/"},
                         [],
                         [{'Host',"localhost:7878"},
                          {'User-Agent',
                           "Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.2.16) Gecko/20110323 Ubuntu/10.10 (maverick) Firefox/3.6.16"},
                          {'Accept',
                           "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
                          {'Accept-Language',
                           "fr,fr-fr;q=0.8,en;q=0.6,en-us;q=0.4,it;q=0.2"},
                          {'Accept-Encoding',"gzip,deflate"},
                          {'Accept-Charset',"ISO-8859-1,utf-8;q=0.7,*;q=0.7"},
                          {'Keep-Alive',"115"},
                          {'Connection',"keep-alive"},
                          {'Referer',"http://localhost:7878/"},
                          {'Cookie',"socketio=xhr-multipart"},
                          {'Pragma',"no-cache"},
                          {'Cache-Control',"no-cache"}],
                         <<>>},
                        <0.70.0>},
                       {<0.71.0>,#Ref<0.0.0.238>}}}]}},
                  {restart_type,transient},
                  {shutdown,5000},
                  {child_type,worker}]

Hey guys,

woudn't it be cool to prepare a test suite that will benchmark original socket.io-node versus socket.io-erlang, ideally collecting cpu utilization?

I need to be able to either shut the listener server down or detect and handle the case where the server is already started so that my handler modules (gen_server, not gen_event like in the demo) can crash and restart cleanly; perhaps socketio_listener:start/1 can return something like {error, {already_started, Pid}} or is there a way for me to detect the running server and get its Pid before attempting to call socketio_listener:start/1?

Since socket.io-erlang has quite a few external dependencies, and because of problems we've had with changes to external APIs breaking our tests (like the jsx issue no. 38) I think using specific versions of our dependencies would be a good idea.

Any reason not to do that?

https://github.com/yrashk/socket.io-erlang/blob/master/test/prop_transport.erl#L94

socketio_data_tests: prop_test (module 'socketio_data_tests')...*failed*
::error:undef
  in function jsx:eventify/1
    called as eventify([start_array,start_array,start_array,start_array,start_array,start_array,
 start_array,start_array,start_object,
 {key,[]},
 {string,[]},
 end_object,end_array,end_array|...])
  in call from prop_transport:'-json/0-fun-1-'/1
  in call from proper_gen:generate/3
  in call from proper_gen:generate/1
  in call from proper_gen:safe_generate/1
  in call from proper:run/2
  in call from proper:perform/7
  in call from proper:inner_test/2
  output:<<"Testing prop_transport:prop_sanity_msg/0

I'm working on this

Error in process <0.61.0> with exit value:
{badarg,[{lists,keyfind,['Content-Length',1,#Port<0.4257>]},{misultin_utility,get_key_value,2},{misultin_http,add_output_header,2},{misultin_http,socket_loop,3},{misultin_http,handle_get,2},{misultin_http,body,2}]}

Cause is yet unknown

It will require updates to tests and demo

I noticed when sending JSON that data is truncated when received in firefox.

I tracked this down to an issue in socketio_data:encode, after encoding the term via jsx:encode, the length of the returned Json is determined using:

Length = integer_to_list(length(JSON) + ?JSON_FRAME_LENGTH),

However length(JSON) does not count escape sequences, only escaped chars, so the following JSON string:

"[{"name":"andrew","address":"5 nowhere st"}]"

returns 44, where it should return 52.

This results in payloads sent to the browser being truncated to that of the value return from length(JSON) above....though it is only truncated on specific transport mechanisms. I have seen it using firefox which was using jsonp-polling.

Guillermo Rauch [email protected] Jun 24 03:19PM -0300 ^

As of Monday the 27th 10AM PST, the Socket.IO repositories will change names
in GitHub.

learnboost/socket.io-node will become learnboost/socket.io
learnboost/socket.io *will become learnboost/socket.io-client*
*
*
The names in NPM will remain the same (socket.io and socket.io-client *
respectively*).
*
*
This is only important for those using git submodules or those who have
clones of the repositories that they actively pull code from.
In these cases you should simply edit the files .gitmodules and
.git/config and make the appropriate changes.

Remember that the recommended way of getting socket.io is through NPM in
which case there are no changes.

Guillermo Rauch
LearnBoost CTO
http://devthought.com

The current tests use "open -a " to start up a browser and get things going. This command works on OSX only. We should try to find [at least] a Linux equivalent in order to make it possible for more people to develop on socket.io-erlang, including myself on my personal laptop.

An idea to help with this that was mentioned on IRC would be to use Meck to build mocks representing the default behaviour of a client+server discussion. We test these mock objects on OSX to make sure they work, and then make them canonical for our suite so we can run them on Linux, but also make them public so we can allow other developers building their own products on top of socket.io-erlang use them to test their products with more ease. This could then become a full-blown feature and likely a competitive advantage over any other socket.io implementation out there.

Hi,

following situation: there are two (ore more) clients connected via xhr-multipart (Firefox 4 here).
The first client disconnects and after the timeout the first disconnect message is sent.
Then the second client disconnects and the second disconnect message is sent.

But if both disconnect together at nearly the same time (closing both tabs) only one event is triggered.
At least in 5 of 6 tries.

I hope this is reproducible.