wptagent, docker and defunct process about webpagetest.agent HOT 9 OPEN

Not 100% sure it will help but I just added logic to run the browser in a separate session and kill all processes in the session when cleaning up.

from webpagetest.agent.

Thank you, I test it when a new release will be pushed on dockerhub : https://hub.docker.com/r/webpagetest/agent/

from webpagetest.agent.

That's bizarre. dockerhub is supposed to build on every push but it looks like it got wedged a month ago. Poking at it now to see what is going on.

from webpagetest.agent.

ok, new build looks like it went through

from webpagetest.agent.

I try with latest version and I still have the problem.

root 9660 9596 11 09:55 ? 00:00:46 python /wptagent/wptagent.py --server http://webpagetest/work/ --location agent1 --name Docker Test --key key1 --xvfb --dockerized
root 9977 9660 2 09:55 ? 00:00:08 Xvfb :1597771276 -screen 0 1920x1200x24
root 10886 9660 0 09:55 ? 00:00:00 [chrome] <defunct>
root 10887 9660 0 09:55 ? 00:00:00 [nacl_helper] <defunct>
root 11184 9660 0 09:55 ? 00:00:02 [chrome] <defunct>
root 11489 9660 0 09:55 ? 00:00:00 [chrome] <defunct>
root 11490 9660 0 09:55 ? 00:00:00 [nacl_helper] <defunct>
root 11730 9660 0 09:55 ? 00:00:01 [chrome] <defunct>
root 11999 9660 0 09:55 ? 00:00:00 [chrome] <defunct>
root 12000 9660 0 09:55 ? 00:00:00 [nacl_helper] <defunct>

from webpagetest.agent.

Same here. Running four docker wptagent instances this sums up to about 5000 processes in 2 days (alternating firefox and dbus processes). While these don't consume significant ressources, this can lead to several other problems, apart from breaking the monitoring of the server (normally I monitor the processes on the server to be alarmed im something spawns endlessly). Running on the newest bundled WPT release (17.12).

Any clues?

Snippet from ps ax:

26450 ? Z 0:00 [gconfd-2] <defunct>
26455 ? Z 0:00 [Web Content]
26527 ? Z 0:00 [dbus-launch] <defunct>
26528 ? Zs 0:00 [dbus-daemon]
26531 ? Z 0:00 [dbus-launch] <defunct>
26532 ? Zs 0:00 [dbus-daemon]
26534 ? Z 0:00 [gconfd-2] <defunct>
26543 ? Z 0:00 [dbus-launch]
26544 ? Zs 0:00 [dbus-daemon] <defunct>
26547 ? Z 0:00 [gconfd-2]
26565 ? Z 0:00 [Web Content] <defunct>
26578 ? Z 0:00 [gconfd-2]
26627 ? Z 0:00 [firefox] <defunct>

from webpagetest.agent.

Actually, the docker conecpt just seems to miss some proper cleanup, especially when looking at the potential number (and frequency) of spwaned processes from within. This was a topic at the docker community some time ago (long story short: transferring PID 1 if there is no init system built into the docker instance itself).

Adding --init to the docker run command fixes this problem. In it's simplest form:

docker run -d --rm --init -e SERVER_URL="https://example/work/" -e LOCATION="Coolest_Location" -e NAME="Coolest_Agent" wptagent

Edit: I should note that this requires at least docker version >=1.13, if you're on a lower version you should look into some external solutions like https://github.com/Yelp/dumb-init

from webpagetest.agent.

--init option is not available to containers running in kubernetes. is there another way to avoid defunct chrome processes?

from webpagetest.agent.

after a bit googling and reading, this https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace/#understanding-process-namespace-sharing seems to resolve the issue in kubernetes containers. The container process no longer has PID 1 bit is the key as the termination of chrome processes failed, if agent startup script was assigned pid 1. hope this helps others. :)

from webpagetest.agent.