High CPU usage about eventsource HOT 6 CLOSED

I'm not the right person to ask about bugs in http.Agent I'm afraid...

from eventsource.

This doesn't sound like a bug in eventsource to me, so I'm closing this.

from eventsource.

I did finally find a bug, and I intended to post a test and fix for you, but I haven't had a chance yet so I'll just briefly summarize:

The CPU hogging bug occurs at the buf.match() statement (line 112) when the client sits idle for ~10 hours receiving comment messages from the server. In my scenario the server was sending ": staynAlive" every minute or so while idle. The match() statement does not match this comment so buf was not getting reset and was growing arbitrarily large. This caused match() to be called with a slightly larger buffer for every message, eventually consuming all available CPU cycles.

Note this keepalive comment message does not cause a problem on the browser, so that's why I would consider this a bug in eventsource. My workaround for now is to send a real message instead of a comment, which clears the buffer. But that's not a real fix, so I would definitely recommend addressing this. Otherwise eventsource-node has been working great for me, so many thanks for your code.

from eventsource.

Thanks @pwellner that makes sense. I'm not happy with the state of the parser - it should probably be rewritten to deal with this issue as well as #18.

Maybe there are some nuggets in this presentation: http://2012.jsconf.eu/speaker/2012/09/05/faster-than-c-parsing-node-js-streams-.html - I think we need a stream parser.

This is not a high priority for me personally, so I'm unlikely to spend time on it. However, I'd be more than happy to take a pull request.

from eventsource.

I'm also getting high CPU usage when trying to parse very long messages.

Simple test on an i7 haswell:

0 characters took 6ms
1000000 characters took 3417ms
2000000 characters took 6887ms
3000000 characters took 10698ms
4000000 characters took 13796ms
...

Using the code:

eventstream = require \./node_modules/eventsource/lib/eventstream.js

for i til 8_000_000 by 1000000
  input = "event: something\ndata: #{'a' * i}\n\n"
  start = Date.now!
  eventstream.parse input
  console.log "#i characters took #{Date.now! - start}ms"

Firefox is able to decode the same event stream almost instantly (minus IO time).

I'll take a crack at a custom parser. It's a shame jison isn't able to scale that well.

from eventsource.

A custom parser should be fairly easy to write. Take a look at EventSource.cpp from WebKit - it's probably a good starting point.

from eventsource.