Comments (12)
Friendly paste :)
Phusion - Docker and the PID 1 zombie reaping problem
from docker.
I'm seeing some zombie ssh processes as well.
from docker.
zombies accumulate indeed
$ ps fauwwwx | grep defunct | wc -l 68
from docker.
Hi there,
I ran into this exact issue and created https://github.com/krallin/tini as a solution. The idea was to have a drop-in "zombie reaper" that doesn't do anything else (in comparison phusion's my_init
does quite a lot).
My Dockerfile
looks like this:
FROM jenkins
USER root
ENV TINI_VERSION v0.3.4
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN chmod +x /tini
USER jenkins
ENTRYPOINT ["/tini", "--", "/usr/local/bin/jenkins.sh"]
@ndeloof - Would it make sense to make a PR to add tini
to the image?
Obviously, pulling the binary directly (which is what I pasted above) may not be desirable, but I have code here that can be used to build tini
in the Dockerfile
and cleanup the environment. I'm also planning on building debs for it.
https://github.com/krallin/tini-images/tree/master/autogen
from docker.
I had the same issue and it was making some Jenkins jobs failing because the job was involving some tests that were checking running processes. I applied the solution proposed by krallin and it is now working. It would be great to include a solution in the base image.
from docker.
I wonder why this appear as a docker image issue, not a generic jenkins issue. Jenkins is (supposed to) kill all processes after build completion, so should not generate such zombie processes (until there's something docker-specific I'm missing)
proposed tini
workaround just seems to hide the root issue
from docker.
@ndeloof This is not correct. Lets consider following process tree: jenkins -- bash -- python. If bash dies unexpectedly (ie. kill -9) python process will have no parent. In such case parent will be changed automatically by kernel to PID 1 (init process in normal Linux based system). In such case Jenkins can and should clean up after bash. But python is not Jenkins business anymore, because it is not in his process tree anymore. So python won't be cleaned by Jenkins, but it will be detected by init process as new child and cleaned up by init.
But in your container PID 1 is Jenkins. So this python process will be stuck as zombie process forever.
from docker.
Jenkins do rely on inherited env to detect child processes created by a job and kill them on build completion. Need to experiment with this and try to find a jenkins-centric fix for it, better than hacking the docker image. Thanks for the detailed description
from docker.
@ndeloof ,
Note that zombie reaping wouldn't necessarily be that useful in Jenkins outside of a Docker environment, considering zombies are usually reaped by init
(though on newer kernels you could register Jenkins as a subreaper).
Cheers,
from docker.
tini indeed seems to be the sole option we have. Sorry I didn't understood the issue first time I commented here
@krallin how can a process be registered as a subreaper ? IIUC this require a system call from the process to become subreaper, there's no CLI tool for this purpose, right ?
from docker.
Yes, it requires a system call. It's pretty easy to write a CLI tool to do it (it persists across a fork). You'd also need to actually reap zombies though.
Cheers,
from docker.
@krallin
I use tini start jenkins, but it still became zombie process after docker stop jenkins container:
[root@docker32-111 ~]# docker ps -a
231890d0b055 jenkins:2.7.1 "/bin/tini -- /usr/lo" 4 weeks ago Exited (4294967295) 1 hour ago big_rosalind
[root@docker32-111 ~]# ps -elf | grep -v grep | grep defunct
0 Z root 15524 15507 6 80 0 - 0 exit 10月13 ? 1-04:40:10 [java]
pid 15507 is tini process:
[root@docker32-111 ~]# ps -elf | grep -v grep | grep 15507
4 S root 15507 1 0 80 0 - 0 wait 9月29 ? 00:01:38 [tini]
0 Z root 15524 15507 6 80 0 - 0 exit 10月13 ? 1-04:40:10 [java]
and the tini cmd name displays wrong, the normal is "/bin/tini -- /usr/local/bin/jenkins.sh" but not "[tini]"
there are lots of child processes of 15524, and the status all is D
[root@docker32-111 ~]# ps -elf | grep 15524
1 D root 522 15524 0 80 0 - 11527716 sleep_ 10月31 ? 00:00:00 java -jar jenkins.war
1 D root 839 15524 0 80 0 - 11527716 sleep_ 10月31 ? 00:00:00 java -jar jenkins.war
......
here is Dockerfile entrypoint:
ENTRYPOINT ["/bin/tini", "--", "/usr/local/bin/jenkins.sh"]
Is there any good suggestions ?
from docker.
Related Issues (20)
- windowsservercore-2019 image not updated HOT 5
- Ship production build of Java 21 from Eclipse Temurin HOT 3
- Upgrade curl to >=8.4.0 to fix CVE-2023-38545 HOT 3
- CVE-2023-38039, CVE-2023-38408, CVE-2023-38039, CVE-2023-38039, CVE-2023-44487 HOT 4
- Unexpected loss of more precise labels in favor of less precise labels HOT 13
- The directory separator used in the Windows build script is dependent on the OS
- latest with jdk17 & almalinux HOT 2
- Update Docker Hub description with JDK17 images instead of JDK11 HOT 2
- Set JDK17 as the default JDK for Windows image
- Publish a Windows image for each supported Java version HOT 2
- `windowsservercore-2019` images use `windowsservercore-1809` as base image HOT 2
- Windows - There should be a set of weekly and LTS short tags not including Jenkins version in their names HOT 4
- After Jenkins docks with ldap, the first login the next day will display a password error. The second login will be successful, and submitting the pipeline using Jenkins' API will fail. Jenkins logs indicate that the connection to ldap has timed out
- JenkinsProject
- `jenkins-plugin-cli` reports `War not found, installing all plugins: /usr/share/java/jenkins.war` HOT 5
- CVE-2024-23897 still exists in latest LTS version jenkins/jenkins:2.440.1-lts-jdk11 HOT 2
- The repository 'https://packagecloud.io/github/git-lfs/debian bookworm InRelease' is not signed. HOT 1
- CVE-2024-22201 still appears in jenkins/jenkins:2.444-jdk11 image HOT 2
- pipeline-input-step:2.8 Test cases run failed (Server Issue) HOT 10
- Add ARM64 Support for Alpine JDK 11 and 17 - Inbound Agent HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docker.