File descriptor leakage with multi handle and c-ares about curl HOT 14 CLOSED

all leaked file descriptor number can be find from the strace call, all open by openat(AT_FDCWD, "/etc/resolv.conf|/etc/nsswitch.conf". (after one week , there is around 36000 leaked fd, reach the ulimit then app no response)

from curl.

Is this the same as #13341 ?

You mention c-ares, which version?

from curl.

I am not sure they are related to #13341

Test confirm with c-ares 1.28.1 and 1.28.

beside the file descriptor leak problem, I want to skip the c-ares openat(AT_FDCWD, "/etc/resolv.conf|/etc/nsswitch.conf") step, to verify if them still leak. (not sure how to do this)

from curl.

Please explain how to reproduce this problem. I cannot make it happen and this is not a known issue.

I've reproduced my tests from #13341 with c-ares 1.28.1 and I see no leaks whatsoever.

from curl.

I am working on a minimal test case. before that, I try with http://ip without CURLOPT_RESOLVE, but use CURLOPT_HTTPHEADER to set the hostname.

It is http with ip ( no ssl).

Why I still get this syscall:

open("/etc/resolv.conf", O_RDONLY|O_LARGEFILE) = 29
lseek(29, 0, SEEK_END)                  = 21
lseek(29, 0, SEEK_CUR)                  = 21
lseek(29, 0, SEEK_SET)                  = 0
readv(29, [{iov_base="nameserver 8.8.8.8", iov_len=20}, {iov_base="\n", iov_len=1024}], 2) = 21
close(29)                               = 0
open("/etc/nsswitch.conf", O_RDONLY|O_LARGEFILE) = 29
lseek(29, 0, SEEK_END)                  = 494
lseek(29, 0, SEEK_CUR)                  = 494
lseek(29, 0, SEEK_SET)                  = 0
readv(29, [{iov_base="# /etc/nsswitch.conf\n#\n# Example"..., iov_len=493}, {iov_base="\n", iov_len=1024}], 2) = 494
close(29)                               = 0
open("/etc/netsvc.conf", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("/etc/svc.conf", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
getrandom("\xa0\x1e\xf2\x03\xe2\x51\x13\x5b\x95\x2d\xf5\xd6\x18\xbf\x7b\x84\x5d\x64\x2f\xe1\x83\xba\x61\x8d\x7a\x2c\x15\x40\x17\x60\x3b\x41"..., 256, 0) = 256
uname({sysname="Linux", nodename="test1", ...}) = 0
rt_sigaction(SIGPIPE, NULL, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x350f27}, 8) = 0
rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x350f27}, NULL, 8) = 0
poll([{fd=33, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)

from curl.

Why I still get this syscall:

Your list shows several syscalls. Which one are you asking about?

from curl.

all of t syscalls before rt_sigaction.

Because when I connect a IP without ssl, they are not needed.

from curl.

Those are name server config files so I would presume they are loaded by c-ares. Probably done unconditionally of the name resolve. Hardly relevant for this issue though...

from curl.

1. the app send 39692 request every minute. for each request there is 16 syscall unnecessary.

2. all leaked fd is related to those fd open by "/etc/resolv.conf|/etc/nsswitch.conf", I want to test if there still leaked fd without them

tail -n 1000000 app_log.json | grep '2024-04-11T19:10'|wc
39692  158768 27688591

from curl.

tail -n 100000 app_log.json | grep '2024-04-11T19:15'|wc
38398  153592 26739507

for every sec, 10239 syscall unnecessary. (I start 2 stand alone app, app use single thread)

from curl.

Those are name server config files so I would presume they are loaded by c-ares. Probably done unconditionally of the name resolve. Hardly relevant for this issue though...

In my case, no DNS is needed since I use IP as url address. so I want to know if we can skip the c-ares related call from curl side.

from curl.

tail -n 100000 app_log.json | grep '2024-04-11T19:15'|wc
38398  153592 26739507

for every sec, 10239 syscall unnecessary. (I start 2 stand alone app, app use single thread)

This is wrong number. after I count the syscall call from strace, I think they are called for every new connection. (not every request)

from curl.

We will welcome your optimizations, thank you.

In the mean time, this issue is about a file descriptor leak I have not managed to reproduce.

from curl.

after more test I confirm the fd leaked problem is not a curl issue. sorry for the miss report.

from curl.