Describe the bug We are using the aws v2 sdk on of our processes w

Similar to: <a class="issue-link js-issue-link" data-error-text="Failed to load title"

S3 Clients are leaving behind open PIPE file descriptors about aws-sdk-java-v2 HOT 3 OPEN

dyutishb commented on July 19, 2024

S3 Clients are leaving behind open PIPE file descriptors

from aws-sdk-java-v2.

Comments (3)

dyutishb commented on July 19, 2024

We were able to narrow this down to the S3AsyncClient with CRT and S3TransferManager, here's an explanation of the scenario:

Step1. Check count of open file descriptors of java process using: sudo ls -l /proc/<pid>/fd | wc -l
Our output:

sudo ls -l /proc/733969/fd | wc -l
123

Step2. Execute this in the process:

S3AsyncClient s3AsyncClient = S3AsyncClient.crtBuilder()
                                           .credentialsProvider(credentialsProvider)
                                           .region(Region.US_EAST_1)
                                           .build();
S3TransferManager s3TransferManager = S3TransferManager.builder()
                                                       .s3Client(s3AsyncClient)
                                                       .build();

Step 3. Run same command as step1.
Our output:

sudo ls -l /proc/733969/fd | wc -l
205

Example of FDs opened by the processes: (run sudo ls -l /proc/<pid>/fd)

lrwx------ 1 dyutishb dyutishb 64 Jun 19 15:41 183 -> 'anon_inode:[eventpoll]'
lr-x------ 1 dyutishb dyutishb 64 Jun 19 15:41 184 -> 'pipe:[16427762]'
l-wx------ 1 dyutishb dyutishb 64 Jun 19 15:41 185 -> 'pipe:[16427762]'
lrwx------ 1 dyutishb dyutishb 64 Jun 19 15:41 186 -> 'anon_inode:[eventpoll]'
lr-x------ 1 dyutishb dyutishb 64 Jun 19 15:41 187 -> 'pipe:[16427763]'
l-wx------ 1 dyutishb dyutishb 64 Jun 19 15:41 188 -> 'pipe:[16427763]'

Step4. Close the clients using:

s3AsyncClient.close();
s3TransferManager.close();

Step 5. Repeat commands from step1 and 3:
Our output:

sudo ls -l /proc/733969/fd | wc -l
169

We still see similar open FDs:

lrwx------ 1 dyutishb dyutishb 64 Jun 19 15:41 161 -> 'anon_inode:[eventpoll]'
lr-x------ 1 dyutishb dyutishb 64 Jun 19 15:41 162 -> 'pipe:[16427755]'
l-wx------ 1 dyutishb dyutishb 64 Jun 19 15:41 163 -> 'pipe:[16427755]'
lrwx------ 1 dyutishb dyutishb 64 Jun 19 15:41 164 -> 'anon_inode:[eventpoll]'
lr-x------ 1 dyutishb dyutishb 64 Jun 19 15:41 165 -> 'pipe:[16427756]'
l-wx------ 1 dyutishb dyutishb 64 Jun 19 15:41 166 -> 'pipe:[16427756]'

The close() calls clears out some of the open pipe handles but not all are cleared and these are not released even on destruction of the object which is leading to our process having too many open handles.

from aws-sdk-java-v2.