Suspected memory leak about java-gearman-service HOT 8 CLOSED

Very strange. In most operating systems the direct buffers are allocated 
outsize the java heap and should not throw this specific exception.

I'll look into this, however.

How hard are you stressing the system? The buffers grow as needed. When a 
connection is closed any allocated resources should be freed (if not i'll find 
it). I could see this error occurring if 1) the Direct Buffers are using local 
resources and 2) The packets are very large and/or there are too many workers.

If the case is that you're stressing the server too hard, you could make the 
packets smaller and distribute the workers over a larger set of servers.

I did find a memory leak within the NioReactor package. I've patched it. Give 
version 0.6.3 a shot and let me know if you're still receiving this exception.

Thanks
isaiah.v 

Thanks Isaiah I will deploy and let you know ASAP if this helped. 

Just to be complete, yesterday (v0.6.2) I did a heap dump and used Eclipse 
Memory Analyzer. The following was the output :-
One instance of "sun.nio.ch.UnixAsynchronousServerSocketChannelImpl" loaded by 
"<system class loader>" occupies 140,231,368 (80.19%) bytes. 

Keywords
sun.nio.ch.UnixAsynchronousServerSocketChannelImpl

 Shortest Paths To the Accumulation Point
Class Name Shallow Heap Retained Heap 
sun.nio.ch.UnixAsynchronousServerSocketChannelImpl @ 0x74006e1e8 72 140,231,368 
value java.util.HashMap$Entry @ 0x74011de50 32 32 
[19] java.util.HashMap$Entry[64] @ 0x74011dbf0 272 1,104 
table java.util.HashMap @ 0x74011dbc0 48 1,152 
fdToChannel sun.nio.ch.EPollPort @ 0x74006c010 88 3,752 
this$0 sun.nio.ch.EPollPort$EventHandlerTask @ 0x74011d588 16 16 
<Java Local>, target java.lang.Thread @ 0x74011d500 Thread-0 Thread 104 536 
group sun.nio.ch.Invoker$GroupAndInvokeCount @ 0x7c19ec5b0 » 24 24 
port sun.nio.ch.UnixAsynchronousSocketChannelImpl @ 0x7c1933c10 » 152 408 

Hi  Isaiah,
  After deploying the fix I did a heap dump and ran it through Eclipse Memory Analyzer. Below is the results :-
The class "java.lang.ref.Finalizer", loaded by "<system class loader>", 
occupies 9,682,432 (70.56%) bytes. 

Keywords
java.lang.ref.Finalizer

Shortest Paths To the Accumulation Point
Class Name Shallow Heap Retained Heap 
class java.lang.ref.Finalizer @ 0x7cf818c70 System Class 16 9,682,432 

Accumulated Objects
Class Name Shallow Heap Retained Heap Percentage 
class java.lang.ref.Finalizer @ 0x7cf818c70 16 9,682,432 70.56% 
java.lang.ref.Finalizer @ 0x7c8362e20 40 9,682,400 70.55% 
java.lang.Object @ 0x7cf818cf0 16 16 0.00% 
Total: 2 entries 56 9,682,416 0.706 

I did see the use of finalize on some of the classes. here is a link which may 
help fix this (?)
http://www.fasterj.com/articles/finalizer1.shtml

I can't reproduce this finalizer memory leak. It could be, however, that my 
system does not have the resources yours does, and you are able to submit and 
execute jobs faster than the finalizer thread can clean some of these sort 
lived objects.

I've attached a build that has removed the finalize method on some of the sort 
lived objects. Could you verify that it's giving you the desired result?

If this turns out to work for you, I'll push this to v0.6.4.

Thanks,
isaiah.v

Thanks Isaiah!
I ran JConsole overnight and did a GC. The heap looks in good condition. So i 
did not need to use the 0.6.3a fix. Will keep a watch though and let you know 
if I need to use the fix.
Thanks once again for the prompt reply.

Keep me updated :)

If I need to, I can re-open the issue.

Thanks,
isaiah.v

Hi Isaiah

I'm using version 0.6.6 and it seems that I've got the same issue. I have to 
load a lot of objects to gearman in a short period of time and the application 
runs out of memory after about 1-2 million objects submitted.