Giter Club home page Giter Club logo

socket.io-stream's Introduction

Socket.IO stream

Build Status NPM version

This is the module for bidirectional binary data transfer with Stream API through Socket.IO.

Installation

npm install socket.io-stream

Usage

If you are not familiar with Stream API, be sure to check out the docs. I also recommend checking out the awesome Stream Handbook.

For streaming between server and client, you will send stream instances first. To receive streams, you just wrap socket with socket.io-stream, then listen any events as usual.

Server:

var io = require('socket.io').listen(80);
var ss = require('socket.io-stream');
var path = require('path');

io.of('/user').on('connection', function(socket) {
  ss(socket).on('profile-image', function(stream, data) {
    var filename = path.basename(data.name);
    stream.pipe(fs.createWriteStream(filename));
  });
});

createStream() returns a new stream which can be sent by emit().

Client:

var io = require('socket.io-client');
var ss = require('socket.io-stream');

var socket = io.connect('http://example.com/user');
var stream = ss.createStream();
var filename = 'profile.jpg';

ss(socket).emit('profile-image', stream, {name: filename});
fs.createReadStream(filename).pipe(stream);

You can stream data from a client to server, and vice versa.

// send data
ss(socket).on('file', function(stream) {
  fs.createReadStream('/path/to/file').pipe(stream);
});

// receive data
ss(socket).emit('file', stream);
stream.pipe(fs.createWriteStream('file.txt'));

Browser

This module can be used on the browser. To do so, just copy a file to a public directory.

$ cp node_modules/socket.io-stream/socket.io-stream.js somewhere/public/

You can also use browserify to create your own bundle.

$ npm install browserify -g
$ cd node_modules/socket.io-stream
$ browserify index.js -s ss > socket.io-stream.js
<input id="file" type="file" />

<script src="/socket.io/socket.io.js"></script>
<script src="/js/socket.io-stream.js"></script>
<script src="/js/jquery.js"></script>
<script>
$(function() {
  var socket = io.connect('/foo');

  $('#file').change(function(e) {
    var file = e.target.files[0];
    var stream = ss.createStream();

    // upload a file to the server.
    ss(socket).emit('file', stream, {size: file.size});
    ss.createBlobReadStream(file).pipe(stream);
  });
});
</script>

Upload progress

You can track upload progress like the following:

var blobStream = ss.createBlobReadStream(file);
var size = 0;

blobStream.on('data', function(chunk) {
  size += chunk.length;
  console.log(Math.floor(size / file.size * 100) + '%');
  // -> e.g. '42%'
});

blobStream.pipe(stream);

Socket.IO v0.9 support

You have to set forceBase64 option true when using the library with socket.io v0.9.x.

ss.forceBase64 = true;

Documentation

ss(sio)

  • sio socket.io Socket A socket of Socket.IO, both for client and server
  • return Socket

Look up an existing Socket instance based on sio (a socket of Socket.IO), or create one if it doesn't exist.

socket.emit(event, [arg1], [arg2], [...])

  • event String The event name

Emit an event with variable number of arguments including at least a stream.

ss(socket).emit('myevent', stream, {name: 'thefilename'}, function() { ... });

// send some streams at a time.
ss(socket).emit('multiple-streams', stream1, stream2);

// as members of array or object.
ss(socket).emit('flexible', [stream1, { foo: stream2 }]);

// get streams through the ack callback
ss(socket).emit('ack', function(stream1, stream2) { ... });

socket.on(event, listener)

  • event String The event name
  • listener Function The event handler function

Add a listener for event. listener will take stream(s) with any data as arguments.

ss(socket).on('myevent', function(stream, data, callback) { ... });

// access stream options
ss(socket).on('foo', function(stream) {
  if (stream.options && stream.options.highWaterMark > 1024) {
    console.error('Too big highWaterMark.');
    return;
  }
});

ss.createStream([options])

  • options Object
    • highWaterMark Number
    • encoding String
    • decodeStrings Boolean
    • objectMode Boolean
    • allowHalfOpen Boolean if true, then the stream won't automatically close when the other endpoint ends. Default to false.
  • return Duplex Stream

Create a new duplex stream. See the docs for the details of stream and options.

var stream = ss.createStream();

// with options
var stream = ss.createStream({
  highWaterMark: 1024,
  objectMode: true,
  allowHalfOpen: true
});

ss.createBlobReadStream(blob, [options])

  • options Object
    • highWaterMark Number
    • encoding String
    • objectMode Boolean
  • return Readable Stream

Create a new readable stream for Blob and File on browser. See the docs for the details of stream and options.

var stream = ss.createBlobReadStream(new Blob([1, 2, 3]));

ss.Buffer

Node Buffer class to use on browser, which is exposed for convenience. On Node environment, you should just use normal Buffer.

var stream = ss.createStream();
stream.write(new ss.Buffer([0, 1, 2]));

License

MIT

socket.io-stream's People

Contributors

nkzawa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

socket.io-stream's Issues

Comprehensive documentation

Hello, this module is very good, however we need more documentation, specially about:

  1. API reference, what can we do and how can we do it. (right now your socket.on doc doesn't match with your examples).
    -Object methods, return type, argument types
    -Module depdendancy
  2. Many examples code, on:
    -how to emit file from server to client.
    -how to emit file from client to server.
    -how to add listeners of progress in client.
    -how to add listeners of progress in server.

Please, I'm looking forward to this. Let me know if I can help on this task.

Regards.

Receiving invalid stream id

I am trying to implement socket.io-stream into my project, and I have noticed that when I stream objects to my client, I keep getting an invalid stream id error. Now it appears that all my data has been streamed successfully and the stream has ended successfully too, but it comes after the stream end event is sent and acknowledged.

Here is an image to show what I mean
screen shot 2014-12-05 at 4 16 28 am

Any suggestions or ideas would be appreciated.
Thanks!

I cannot use it with browserify

If I use the module via

require('socket.io-stream'); 

and then I compile my app.js with browserify, I receive the following error in web console:

Uncaught Error: Cannot find module './_stream_duplex'

Web Workers in Firefox

socket.io-stream uses FileReader which is not supported in Firefox workers. Instead, FileReaderSync can be used. Is there any way to work around this?

Deactivate debug (optionally)

It would be nice to be able to toggle debug, because it's quite verbose.

Which makes it hard to see debugging coming from other parts of an application.

Browser socket.io-stream

Hello, I'm trying to use socket.io-stream in a browser to upload a file. However, I keep getting this error, in the file change event.

"Uncaught ReferenceError: ss is not defined "

Am I missing something obvious? Please help.

Thanks in advance.

Here is the html source:

<script type="text/javascript" src="/socket.io/socket.io.js"></script> <script type="text/javascript" src="/js/socket.io-stream.js"></script> <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script> <script> $(function() { $('#file').change(function(e) { var file = e.target.files[0]; var stream = ss.createStream();
            // upload a file to the server.
            ss(socket).emit('file', stream, {size: file.size});
            ss.createBlobReadStream(file).pipe(stream);
        });
    });
    var socket = io.connect('http://lb1:8080');

</script>

Unexpected behavior for BlobStream pause and resume?

Hello,
I'm having issues pausing and resuming the stream upload using BlobStream.pause() and and BlobStream.resume(). I read BlobStream returned a Readable Stream and that these methods were implemented. Pause does nothing, and resume emits 1 data event and pauses.

Have I missed something ? What would be the best way to pause and resume a stream upload?

Thanks a lot! :)

Socket.io-stream vs Express Post

I'm developing a server that needs to forward relatively big files (average of around 10MB). Streaming them from client to client using Socket.io-stream sounds ideal, because the server wouldn't need to buffer/store them.
I created a simple test where my server receives file through a socket.io-stream, creates a new stream and streams it to another client. But the problem was that the cpu was skyrocketing, with 10 clients, sending 10MB files every 10 seconds. So I simplified the server to just receiving socket.io-streams and writing them to disk (on RAM) (using pipe), but the problem persists. It's a problems because new clients can't connect anymore, probably because it's to busy.

When post files to a node.js server using Express, handling posts and piping incoming req to the same RAM disk it stays between 0% and 50%, when I have 100 clients, sending 10MB files every 10 seconds. (Using 50 to 190 MB memory)

Is there anyway I can tweak socket.io-stream. I tried increasing the highWaterMark but that seemed to create connection problems even earlier (new clients not being able to connect).

Not handled stream can block stream for other clients?

I'm trying to forward a file over socket.io-stream to multiple clients. But when the same stream is piped to two socket.io-stream clients, I notices that if one client doesn't handle the event (or use the stream) the other client can't fully retrieve the stream either.
I've tried to create an as simple as possible example.
How should I handle this? Am I trying to do something impossible?
I can't just add another read stream because in my actual use case that stream is another socket.io-stream that is send to the server.
To clarify, when both clients listen (handle the event and the stream) it works fine.

var socketClient  = require('socket.io-client');
var debug         = require('debug')('streams');
var ss            = require("socket.io-stream");
var fs            = require('fs');
var PORT          = 5000;
var URL           = "http://localhost:"+PORT;

server();
consumer("a",true);
consumer("b",false);

// SERVER
function server() {
  var http          = require('http').Server();
  var io            = require('socket.io')(http);
  http.listen(PORT, function(){
    debug('server listening on *:' + PORT);
  });
  var nsp = io.of('/');
  var consumers = [];
  nsp.on('connection', function(socket) {
    var query = socket.handshake.query;
    debug("server: new connection: ",query.name);
    consumers.push(ss(socket));
  });
  setTimeout(function() {
    var filename = "image.jpg";
    var readStream = fs.createReadStream("assets/"+filename);
    for(var i in consumers) {
      var outgoingStream = ss.createStream();
      consumers[i].emit('file',outgoingStream,{name:filename});
      readStream.pipe(outgoingStream);
    } 
  },500); 
}

// CLIENTS
function consumer(name,listen) {
  var serverSocket = socketClient(URL+"?name="+name,{forceNew:true});
  var serverStreamSocket = ss(serverSocket);
  serverSocket.once('connect', function(){
    debug("consumer "+name+" connected");
    if(listen) {
      serverStreamSocket.on("file",function(stream,data) {
        debug("consumer on file");
        stream.pipe(fs.createWriteStream(name+"-"+data.name));
      });
    }
  });
}

No support for Windows?

npm install socket.io-stream

npm WARN package.json [email protected] No description
npm WARN package.json [email protected] No repository field.
npm WARN package.json [email protected] No README data
npm WARN package.json [email protected] No description
npm http GET https://registry.npmjs.org/socket.io-stream
npm http 304 https://registry.npmjs.org/socket.io-stream
npm http GET https://registry.npmjs.org/socket.io-stream/-/socket.io-stream-0.5.2.tgz
npm http 200 https://registry.npmjs.org/socket.io-stream/-/socket.io-stream-0.5.2.tgz

> [email protected] prepublish z:\temp\Users\Farzher\AppData\Local\Temp\npm-501412-vqhKrfgb\socket.io-stream-0.5.21393148684527-0.14867000188678503\package
> make test-all build

make: *** No rule to make target `test-all'.  Stop.
npm ERR! addLocalDirectory Could not pack "z:\\temp\\Users\\Farzher\\AppData\\Local\\Temp\\npm-501412-vqhKrfgb\\socket.io-stream-0.5.21393148684527-0.14867000188678503\\package" to "C:\\Users\\Farzher
\\AppData\\Roaming\\npm-cache\\socket.io-stream\\0.5.2\\package.tgz"
npm ERR! [email protected] prepublish: `make test-all build`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the [email protected] prepublish script.

Enable writing to a stream?

I'm trying to write data to the stream as it comes in, but nothing seems to be getting sent from the client to the server. When I attach an 'on data' event listener on the client everything breaks. When I attach an 'on data' event listener on the server it is never triggered.

1. @bStream = ss.createStream()
2.
3. #ย if i add this i see an error
4. @bStream.on 'data', (data) ->
5.   console.log data.length
6.
7. ss(@socket).emit 'talk', @bStream, meta
8. @bStream.write myInt16ArrayBuffer

Error triggered on line 4 above.

Uncaught TypeError: Cannot read property '_read' of null
IOStream._read @ socket.io-stream.js:223
Readable.read @ socket.io-stream.js:3193
Readable.resume @ socket.io-stream.js:3562
Readable.on @ socket.io-stream.js:3528

I had everything working with Binaryjs, I wonder am I allowed to write to the stream like that with socket.io-stream?

using callbacks with socket.io-stream possible?

Hi nkzawa,

Thanks for creating socket.io-stream!
I'm wondering if it's possible to use callbacks with socket.io-stream. This would be very convenient.

Best wishes,
Rick

server.js

var fs = require('fs');
var app = require('express')();
var http = require('http').Server(app);
var io = require('socket.io')(http);
var ss = require('socket.io-stream');

io.of('/').on('connection', function(socket) {

  ss(socket).on("getImage", function(cb) {
    var outstream = ss.createStream();
    cb(outstream); //   'announcing' a stream in a callback. Is this possible?
    fs.createReadStream("image.jpg").pipe(outstream);
  });
});

http.listen(3000, function(){
  console.log('listening on *:3000');
});

client.js

var io = require('socket.io-client'); 
var ss = require('socket.io-stream');
var socket = io.connect('http://localhost:3000');

socket.on("connect", function() {

  ss(socket).emit("getImage",function(stream) {

    //how to decorate a stream received as argument from the server
    //would something like socketIOStream(stream) be possible?

    socketIOStream(stream).on("data", function(data) {   
      console.log("data",data);
    });
  });
});

Writing custom data on a stream

Hi!

I want to know how can you send custom data from a browser through one of your streams, how can you "write" on this stream anything that's not a string?
I want to pass an array (or bufferArray, or one of those) do I need to use a Buffer object?

Thanks!

"Unknown encoding: buffer" error when using Safari

Hi,

I have set up an upload system which reports progress etc, all taken from the main github page. Code:

Client

var stream = ss.createStream();
ss(Socket).emit('beginUpload', stream, $scope.video);

var blobStream = ss.createBlobReadStream($scope.videoFile);
var size = 0;

$scope.uploading = true;

blobStream.on('data', function(chunk) {
    size += chunk.length;
    var progress = (size / $scope.videoFile.size * 100).toFixed(2);
    //console.log('progress', progress);
    if (progress != 100) {
        updateProgress(progress);
    }
});

blobStream.on('error', function(err) {
    console.log('ERROR', err);
});

blobStream.on('end', function() {
    Socket.emit('uploadComplete', $scope.video);
});

blobStream.pipe(stream);

Server

ss(socket).on('beginUpload', function(stream, data) {

    // Save stream to file
    var filename = config.upload.tmpFolder+'/'+data.filename;

    stream.pipe(fs.createWriteStream(filename)).on('error', function(err) {
        console.log('crashed at begin upload', err);
    });
});

This works perfectly on Chrome/Firefox & Internet Explorer from my tests, but for reason using Safari causes the following error on the server:

buffer.js:377
      throw new TypeError('Unknown encoding: ' + encoding);
            ^
TypeError: Unknown encoding: buffer
    at Buffer.write (buffer.js:377:13)
    at new Buffer (buffer.js:215:28)
    at IOStream.Readable.push (_stream_readable.js:122:15)
    at push (/home/user/www/node_modules/socket.io-stream/lib/iostream.js:147:20)
    at IOStream._onwrite (/home/user/www/node_modules/socket.io-stream/lib/iostream.js:153:5)
    at Socket._onwrite (/home/user/www/node_modules/socket.io-stream/lib/socket.js:231:10)
    at Socket.<anonymous> (/home/user/www/node_modules/socket.io-stream/node_modules/component-bind/index.js:21:15)
    at Socket.EventEmitter.emit (events.js:106:17)
    at Socket.onevent (/home/user/www/node_modules/socket.io/lib/socket.js:327:8)
    at Socket.onpacket (/home/user/www/node_modules/socket.io/lib/socket.js:287:12)

Am I doing something wrong or is this a problem?

reduce file size of client js library

You have an awesome implementation here, but the frontend (browser) library is a massive 147.975 kb minified, which for me makes it a too heavy implementation. I have no experience with browserify, can it compile a smaller lib?

Stream managment

I've build a service that would receive a socket.io-stream from one socket, and then tried to forward it to other socket's. This has lot's of challenges like, sharing a stream over multiple node instances and piping a stream to multiple sockets (in the same namespace). It was build as API, which meant that I couldn't control how it was used. I couldn't rely on client's listening to new stream events or then piping the stream.

I had to write some code to manage this, that I would like to share. Maybe something can be integrated into socket.io-stream or it can help others. Some of this might constitute a nice separate module, but since the project was aborted, I can't.

I used two basic solutions:
1. Limit the number of streams to one per namespace. Override an idle stream when a new stream is send, but block a new stream when a stream is already streaming.
I've written a StreamManager that stores streams and allows sharing streams over multiple node.js instances.
When forwarding a stream withing a namespace I would check whether a stream was already stored under an id (in my case the namespace name), and check using a isStreaming method whether it was already streaming. When it wasn't streaming I could remove the old stream and add a new one. When it was streaming I would respond with an error.
To check whether a stream was streaming I had to override the stream's push method, which is a ugly hack. I would love something like a streaming boolean in the native Stream API.
2. Add timeouts.
Because streams weren't always handled I had to build in timeouts. When adding a stream to this StreamManager you can also provide a timeout. This would auto destroy a stream automatically.

Broadcast stream?

Is there anyway to broadcast the stream to all clients in a namespace or room?
I'd like to do something like this:

var nsp = io.of('/user');
nsp.on('connection', function(socket) {
  // receive a image by a broadcasting client
  ss(socket).on('image', function(incomingStream, data) {
    debug("on image ",data);

    // send the image
    var outgoingStream = ss.createStream();
    // emit stream to only one socket
    //ss(consumerSocket).emit('image', outgoingStream, {name: data.name});
    // broadcast stream to all other clients in namespace
    ss(socket).broadcast('image', outgoingStream, {name: data.name});
    incomingStream.pipe(outgoingStream);
  });
});

XHR polling error

Hi

I try use socket.io-stream for streaming socket.io requests!
I have two nodejs servers connected by socket.io (v 1.0) and socket.io-client.
All requests from first server by Express redirect by Socket.io chennel and anotwher side by Request (module) transit to first server by socket.io-stream. (request().pull(soi-stream))

f i use get or post queryes all ok
but if i use xhr i have timeout on request().pull(soi-stream)

I can`t correct you code (all packed)

Thank you !

socket.removeAllListeners() breaks ss(socket)

Currently when you call removeAllListeners on the socket connection that's used by socket.io-stream you break the socket.io-stream insance.
Should we overrule the removeAllListeners method and make sure we re-add the socket.io-stream listeners?
Something like:

var originalRemoveAllListeners = sio.removeAllListeners;
  sio.removeAllListeners = function() {
  originalRemoveAllListeners.apply(sio, arguments);
  // re-add socket.io listeners
}

How to build a stream with JavaScript Array

Hello everybody,

I have to build a web app in order to send the voice of my boss to all users who are connected to the society's website.

I am using NodeJS for my server, and Node-Core-Audio in order to get the sound of my soundcard (a microphone is connected to the computer where Node-Core-Audio is installed. So i'm able to send a JavaScript Array with 1024 audio samples to my NodeJS server

Node-Core-Audio (with NodeJs) --------------> NodeJS Server |---------------------> Users (AngularJS)

I'm using socket.io to transport the sound, but it's not perfect, i've got some latency periods.

I'm reading about socket.io-stream but it seems that it is only used to send files ?

How can I build a socket.io-stream which send all my arrays filled with 1024 audio samples ?

Support duplex streams

Supporting Duplex streams that are both readable and writable would be really nice. Nothing should prevent it technically.

I'm currently going through your code to see how we could implement it.

Let me know what your status is on this and if you have any advice.

Thanks

how to send to room?

Client/Browser upload file with file event.
ss(socket).emit('file', stream, {size: file.size, name: file.name})

Then I can pipe back to the client from server.
ss(socket).on('file', function (stream, data) {

var outStream = ss.createStream()
ss(socket).emit('file', outStream, data)

//outStream.pipe(stream) // this is wrong
stream.pipe(outStream)
})

But I want to sent it to the room. For example, send msg to the room with socketio / sio
sio.sockets.in('roomName').emit('msg', JSON.stringify(msgObj))

How can I sent stream to the room?

Regards,

Mike

Ref: http://stackoverflow.com/questions/10058226/send-response-to-all-clients-except-sender-socket-io

'disconnect' causes errors on old streams, memory leak?

I'm noticing that when a client disconnects, an error event gets emitted on my server's socket.io-stream even though the server stream has already received an 'end' event. I would have expected that the 'end' event would exclude any further events from happening on my server read stream (except maybe 'close'), and that the stream would be cleaned up. Specifically, I would expect Socket.prototype._onend to do something like stream.destroy() or socket.cleanup(id) to the readable end of my stream.

I might be overlooking something, but isn't this a memory leak? How should I close down my streams, if receiving the 'end' event on them doesn't do the job? Should I manually be calling destroy() on an 'end' handler?

The symptom of this problem is that my server-side readable socket.io-stream receives the commands in Socket.prototype._ondisconnect even after it has "ended", particularly the stream.emit('error', new Error('Connection aborted'));. I wouldn't expect stream that I've already consumed to give me some error event a some future time, just because the client eventually disconnects.

Am I missing something? Thanks.

Multiple streams over a same socket

I don't really have a snippet to reproduce this yet, that'll have to wait a few days since I'm very busy on other projects (for my Startup).

In one of the applications that streams terminals over socket.io-stream, I seem to have an issue when trying to open multiple streams over the same socket.

Do you have an idea where this is coming from ?

Data passed to server always undefined

Hi

I'm trying to use the latest socket.io-stream with socket.io v.1.3.5. I've set up the most basic example (as per the github page). Even though the event is handled correctly, I always get the error that stream has no method 'pipe' and that data is undefined. Socket.io itself runs without any issues. Tested on Chrome/Safari/Firefox.

ss(socket).on('file', function(stream, data) {
    var filename = path.basename(data.name);
    stream.pipe(fs.createWriteStream(filename));
});

Could there be any issues with the latest version of socket.io? (I'm also using the express framework on my nodejs server). I 'd love to use the library in my project, any help is apreciated.

Support more than one write/read at a time (improve performance)

Right now since we toggle readable/writable everytime to synchronize reads/writes, I suspect that's damaging the performance of socket-io.stream.

Would be great if we could do something smarter rather than simply toggling. I think it would dramatically improve performance.

Image broadcast streaming from browser

Is there a way to broadcast an image or another type of file from the browser to all nodes in browser as well?. All is good until the moment when I have to receive the image on all the sockets connected. Is there a seamless way to accomplish this?

FileReader

May be to indicate in docs the browsers which support FileReader?

socket.io-stream and promises

Hello
I have following questions

Is it correct that following code on client side is blocking

var socket = io.connect('http://localhost');
var stream = ss.createStream();

ss(socket).emit('foo', stream);
fs.createReadStream('foo.txt').pipe(stream);

If yes , do you see some problem to wrap the sending part above in promise based code :

Var d=Q.defer();
ss(socket).emit('foo', stream);
fs.createReadStream('foo.txt').pipe(stream);
....
return d.promise;

Additional question : what is recommended way to handle sending errors?
Thanks in advance

Adding more stream status info to IOStream?

Currently it's hard to know whether a incoming stream is being read. One of my workarounds has been to listen to the ss.Socket.event+"-read" event on the socket instance.
I'd like some more properties and maybe events on the IOStream instance so it's easier to know the status of the stream.
Maybe we can add a status propertie to the IOStream instance, that is one of the following:

  • idle (The stream might have been emitted, but it's not being consumed yet)
  • streaming (The stream is being read)
  • finished (The stream has finished streaming)
  • failed (Some error has occored and the stream is aborted)
    There could also be a state event that is emitted on change.

Thoughts on this? I'm open to contribute on this, but I'd like some input first.

Support allowHalfOpen properly

We will follow the implementation of net.Socket, since IOStream has similar nature.

  • set allowHalfOpen to false by default.
  • implement the destroy method.
  • clean up everything on destroy.
  • call the destroy method on finish or end when allowHalfOpen was false.
  • call the destroy method after both finish and end were emitted when allowHalfOpen was true.

How to browserify socket.io-stream.js?

I do it as your introduce on the README to browserify. But When I reference it with src = '/js/socket.io-stream.js' or 'socket.io-stream/socket.io-stream.js', My browser complain that can't get the resource.

note: I start my server with node app.js.

var app = require("express")();
var http = require("http").Server(app);
var io = require("socket.io")(http);
var fs = require("fs");
//var ss = require("socket.io-steam");
var sockets = {};

app.get("/", function (req, resp) {
resp.sendFile(__dirname + "/view/message.html");
});

http.listen(3000, function () {
console.log("listening on 3000");
});

Any event

I need to forward socket.io-streams over a server (from client to another client), so I needed a way to handle any event. I already found tricks to do this with socket.io, see:
socketio/socket.io#1715

But since socket.io-stream works differently, it for example relies on overriding the event handler the users supplies (to replace the StreamID with a actual IOStream), I had to find a different solution.

The basics of my solution:

function createAnyEvents(socket) {
  if(socket.sio && socket.$emit) {
    // listen for stream events on original socket.io socket
    socket.sio.on("stream",function() {
      var args = Array.prototype.slice.call(arguments);
      // Chanding original event to any event, 
      // adding original event type as argument
      // from: eventType, pos, streamID, data, callback
      // to: any, pos, eventType, streamID, data, callback
      // Adding original eventType after pos:
      args.splice(2,0,args[0]); 
      // Changing event type to any:
      args[0] = 'any'; 
      // Increment pos (streamID positions)
      // (because we added eventType in front of them)
      for(var i in args[1]) args[1][i]++;
      socket.$emit.apply(socket,args);
    });
  } else {
    debug("Error: Can't create 'any' event");
  }
}
createAnyEvents(ss(socket));
ss(socket).on('any', function (eventType,incomingStream,data,callback) {
    var outgoingStream = ss.createStream();
    ss(consumerSocket).emit(eventType,outgoingStream,data,callback);
    incomingStream.pipe(outgoingStream);
  });

I'm posting this here so others can use it, maybe we can find a better solution in the future.

createDataURLReadStream in browser client

It would be nice to have the option to create the read stream from a dataURL instead of a file. Something like this:

ss.createDataURLReadStream(dataURL)

Let me explain the scenario. I'm doing some image manipulation on a canvas in the browser (crop, resize, etc...) and I want to upload the processed image, not the image from disk.

Performance of ss.createBlobReadStream(blob).pipe(stream)

Hello there

I experience performance issues with the following piece of code:

            var stream      = ss.createStream(),
                self        = this,
                framesCount = self.framesCount;

            this.canvas.toBlob(function(blob) {

                // this is too slow!

                ss(self.socket).emit(
                    'newFrame',
                    stream,
                    {
                        framesCount: framesCount
                    }
                );

                ss.createBlobReadStream(blob).pipe(stream);

            }, 'image/jpeg', this.settings.video.quality);

My goal is to send images to the server in binary. I made measurements and found out that ss.createBlobReadStream(blob).pipe(stream); is the slowest part.

Firefox alone needs about 50ms - 60ms to pipe that blob to the server. In other words the main thread is blocked by 50ms - 60ms!

Any clues?

Memory leak prevention / warnings

When I used socket.io-stream for a bigger project where I did things like pipe multiple streams and sometimes streams where not handled on the end I ran into lots of little memory leak issues. I really had to dive into how socket.io-stream works to figure out what was going wrong. So I'd like to come up with ways to make this easier for other people.
One of the "invisible" problem areas is that each socket.io-stream socket stores references to socket.io-stream streams it uses. When streams are not handled these streams keep piling up. This is especially problematic when you put streams in flowing mode and they already have a chunk of data in their buffer.

  1. One first thing I would do is log (using debug) the amount of stored streams, this way people see this growing number. Preferably when the streams are added.
  2. Another idea might be to give a warning when a certain amount is reached, like Node.js's max listeners warning.
  3. Could it be that streams don't get removed when an error occurs? I wrote error handlers on streams that would manually end() and destroy() that stream.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.