Could it be a potential issue of AsyncSocket? about asyncio HOT 11 OPEN

It is making sense.
Any idea regarding a solution?
Can you try comment out the cancelioex call and check if the problem still exist?
I'm actually not sure we need the cancellation, however we will need a small state machine for dispose and actually free the memory correctly.

from asyncio.

Thank you for replying.

At this moment, I think immersing the PinnedBuffer into Overlapped and disposing the buffer along with the Overlapped may be the way to go, since the buffer is used by the Overlapped only and it can not be unpinned before the overlapped operation is finished.

from asyncio.

Can you make a PR?

from asyncio.

Sure.

from asyncio.

I'd previously posted a PR with some minor changes. Is it appropriate to merge it?

from asyncio.

The leakage problem is not yet fully solved. Sometimes Overlap instances can not be freed.
I studied the source code closer and found that the sequence of statements in Dispose of AsyncSocket were so critical.

Currently there were four critical statements in the above method:

m_inOverlapped.Dispose();
m_outOverlapped.Dispose();
UnsafeMethods.CancelIoEx(Handle, IntPtr.Zero);
UnsafeMethods.closesocket(Handle);

When the Overlapped instances are disposed, it is very probable that overlap operations such as receiving or sending are outstanding. Thus the Free method inside Overlapped will not be called.
When CancelIoEx is called, the overlapped operations will be canceled and the Overlapped.CompleteOperation MAY be executed, if the cancellation is in time.
When closesocket is called, according to MSDN:

The closesocket function will initiate cancellation on the outstanding I/O operations, but that does not mean that an application will receive I/O completion for these I/O operations by the time the closesocket function returns. Thus, an application should not cleanup any resources (WSAOVERLAPPED structures, for example) referenced by the outstanding I/O requests until the I/O requests are indeed completed.

Since we rely on the completion of Overlapped.CompleteOperation to cleanup the resource consumed by Overlapped, but if the overlapped operation is canceled by closesocket, the CompleteOperation might NOT be called and thus the resource can't be reclaimed.

from asyncio.

If I placed a break point onto the CancelIoEx, and stepped over it, the CompleteOperation was called. But when no break point was there, and the completion port was still alive, sometimes the CompleteOperation was not called. Some overlapped operations ended without any notification.

from asyncio.

from asyncio.

I added code to check the error code of CancelIoEx. It could return true but the CompleteOperation was still not called later.
I also tried to move the Dispose of Overlapped instances from before CancelIoEx to behind. It seemed that the CompleteOperation was missed less, but sometimes it just missed.

I will try more and return to you later.

from asyncio.

I could not yet find the reason why CompleteOperation could not be fired. I eventually created a class which ran thread and threw Overlapped instances which was not handled by CompleteOperation to that thread to destroy those instances after some time.

I recently read the documentation of CancelIoEx. It said that A Winsock client must never issue closesocket on s concurrently with another Winsock function call. Maybe it was potentially leading to another problem that I disposed an AsyncSocket when it was sending or connecting.

from asyncio.

Today I spent a whole day diagnosing this issue. I found a weird thing, which could even happen in the AsyncIO.Tests project. I ran the SendReceive test method.

And placed a breakpoint at the call to CancelIoEx in Socket.cs.

I found that when listener, serverSocket and clientSocket were disposing, the return values of CancelIoEx were all false, and I ran subsequent Marshal.GetLastWin32Error() via the immediate window of the debugger, after each call to CancelIoEx, the return values were all 1168, which indicated the corresponding asynchronous IO operations were not found.

Is it the behavior of CancelIoEx or are we missing something?

from asyncio.