Memory leak and process.exit issue about winston-logsene HOT 9 CLOSED

I believe I found the issue. The arrays created on lines 19 and 20 have the result of each log pushed on here. When the log is processed they are never removed so this.errorOutput and this.writeOutput grow indefinitely.

from winston-logsene.

Thx. Will have look into it. I think your app does not call process.exit() thats why it not terminates. I will look into exit handlers or timers keeping the process alive.

from winston-logsene.

I wanted to see if the issue was being caused by logsene-js so I added it to chrisblossom/memory-leak-example

I could only get it to complete the test once or twice, which it didn't look like the memory leak was caused there, but it does error out the majority of the time with the following message:

Error: Uncaught, unspecified "error" event. ([object Object])
    at Logsene.emit (events.js:144:17)
    at Request._callback (/Users/chris/github/memory-leak-example/node_modules/logsene-js/index.js:235:12)
    at Request.self.callback (/Users/chris/github/memory-leak-example/node_modules/request/request.js:187:22)
    at emitTwo (events.js:87:13)
    at Request.emit (events.js:172:7)
    at Request.<anonymous> (/Users/chris/github/memory-leak-example/node_modules/request/request.js:1044:10)
    at emitOne (events.js:77:13)
    at Request.emit (events.js:169:7)
    at IncomingMessage.<anonymous> (/Users/chris/github/memory-leak-example/node_modules/request/request.js:965:12)
    at emitNone (events.js:72:20)

Here is how I start and restart my development environment:
https://gist.github.com/chrisblossom/c67268ba865d7dd00b0a8b38dc557a43

This would be similar to using something like nodemon.

Let me know if you need any other information.

from winston-logsene.

@chrisblossom great find! It looks Winston Memory transport was the "blueprint" https://github.com/winstonjs/winston/search?utf8=%E2%9C%93&q=errorOutput for the initial winston-logsene implementation and it is a left over and both arrays are not used by the winston framework (searched in winston repository ...).

from winston-logsene.

176cf50 fixed the memory leak but the exit issue is still present for me. I will try and do more research what is causing it soon.

Any thoughts on the logsene-js error above? Were you able to reproduce?

from winston-logsene.

Please note exit can be delayed until logs are flushed to Logsene, but max. 10 seconds (after that a process manager like Linux upstart would kill the process with kill -9). The error was that process.exit() was not called when no logs got flushed (e.g. empty buffer). Fixed in e4be2e8 and released as [email protected].

from winston-logsene.

@chrisblossom RE. Error message

Error: Uncaught, unspecified "error" event. ([object Object])
    at Logsene.emit (events.js:144:17)
    at Request._callback (/Users/chris/github/memory-leak-example/node_modules/logsene-js/index.js:235:12)
    at Request.self.callback (/Users/chris/github/memory-leak-example/node_modules/request/request.js:187:22)
    at emitTwo (events.js:87:13)
    at Request.emit (events.js:172:7)
    at Request.<anonymous> (/Users/chris/github/memory-leak-example/node_modules/request/request.js:1044:10)
    at emitOne (events.js:77:13)
    at Request.emit (events.js:169:7)
    at IncomingMessage.<anonymous> (/Users/chris/github/memory-leak-example/node_modules/request/request.js:965:12)
    at emitNone (events.js:72:20)

Logsene-js emits an error event when http requests to Logsene fail, or when you use wrong token... But winston-logsene registers for this error event and emits the error to winston-logsene users https://github.com/sematext/winston-logsene/blob/master/lib/index.js#L44
Do you have an error handler for winston-logsene in your code? If not its clear that it leads to uncaught errors.

So please add something like this to you code:

  winstonLogseneLogger.on('error', function () { 
   console.error('error in winston-logsene', arguments) 
  })

from winston-logsene.

@chrisblossom I assume we can close this issue, right?

from winston-logsene.

@megastef yep. Memory leak issue is fixed. Thanks for the help!

from winston-logsene.