container create failed under podman 4.6.1 and conmon 2.1.8 about podman HOT 3 CLOSED

The message in the debugging log does not make sense to me. The file oom_score.adj is owned by the host user who launched Podman. INFO[0000] Running conmon under slice user.slice and unitName libpod-conmon-8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933.scope [conmon:d]: failed to write to /proc/self/oom_score_adj: Permission denied ls -al /proc/self/oom_score_adj -rw-r--r-- 1 s-dmadminua documentum-ua 0 May 6 09:23 /proc/self/oom_score_adj I set up a Podman-Desktop with Podman 5.02 on Windows. It can build the existing dockerfiles and run containers successfully. Unfortunately, the Podman 5.0.2 is not available for RHEL 8.x. It is scheduled to be released for RHEL 9.5 podman run --log-level debug --hostname dasvr --name dasvr --rm -d -p 8180:8080 --memory 8g dasvr INFO[0000] podman filtering at log level debug DEBU[0000] Called run.PersistentPreRunE(podman run --log-level debug --hostname dasvr --name dasvr --rm -d -p 8180:8080 --memory 8g dasvr) DEBU[0000] Using conmon: "/usr/bin/conmon" DEBU[0000] Initializing boltdb state at /export/home/s-dmadminua/.local/share/containers/storage/libpod/bolt_state.db DEBU[0000] Overriding graph root "/export/home/s-dmadminua/.local/share/containers/storage" with "/home/s-dmadminua/.local/share/containers/storage" from database DEBU[0000] Overriding static dir "/export/home/s-dmadminua/.local/share/containers/storage/libpod" with "/home/s-dmadminua/.local/share/containers/storage/libpod" from database DEBU[0000] Overriding volume path "/export/home/s-dmadminua/.local/share/containers/storage/volumes" with "/home/s-dmadminua/.local/share/containers/storage/volumes" from database DEBU[0000] Using graph driver overlay DEBU[0000] Using graph root /home/s-dmadminua/.local/share/containers/storage DEBU[0000] Using run root /tmp/containers-user-1293208/containers DEBU[0000] Using static dir /home/s-dmadminua/.local/share/containers/storage/libpod DEBU[0000] Using tmp dir /run/user/1293208/libpod/tmp DEBU[0000] Using volume path /home/s-dmadminua/.local/share/containers/storage/volumes DEBU[0000] Using transient store: false DEBU[0000] Not configuring container store DEBU[0000] Initializing event backend file DEBU[0000] Configured OCI runtime runj initialization failed: no valid executable found for OCI runtime runj: invalid argument DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument DEBU[0000] Configured OCI runtime youki initialization failed: no valid executable found for OCI runtime youki: invalid argument DEBU[0000] Configured OCI runtime crun-wasm initialization failed: no valid executable found for OCI runtime crun-wasm: invalid argument DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument DEBU[0000] Configured OCI runtime ocijail initialization failed: no valid executable found for OCI runtime ocijail: invalid argument DEBU[0000] Using OCI runtime "/usr/bin/runc" INFO[0000] Setting parallel job count to 25 INFO[0000] podman filtering at log level debug DEBU[0000] Called run.PersistentPreRunE(podman run --log-level debug --hostname dasvr --name dasvr --rm -d -p 8180:8080 --memory 8g dasvr) DEBU[0000] Using conmon: "/usr/bin/conmon" DEBU[0000] Initializing boltdb state at /export/home/s-dmadminua/.local/share/containers/storage/libpod/bolt_state.db DEBU[0000] Overriding run root "/run/user/1293208/containers" with "/tmp/containers-user-1293208/containers" from database DEBU[0000] Overriding graph root "/export/home/s-dmadminua/.local/share/containers/storage" with "/home/s-dmadminua/.local/share/containers/storage" from database DEBU[0000] Overriding static dir "/export/home/s-dmadminua/.local/share/containers/storage/libpod" with "/home/s-dmadminua/.local/share/containers/storage/libpod" from database DEBU[0000] Overriding volume path "/export/home/s-dmadminua/.local/share/containers/storage/volumes" with "/home/s-dmadminua/.local/share/containers/storage/volumes" from database DEBU[0000] Using graph driver overlay DEBU[0000] Using graph root /home/s-dmadminua/.local/share/containers/storage DEBU[0000] Using run root /tmp/containers-user-1293208/containers DEBU[0000] Using static dir /home/s-dmadminua/.local/share/containers/storage/libpod DEBU[0000] Using tmp dir /run/user/1293208/libpod/tmp DEBU[0000] Using volume path /home/s-dmadminua/.local/share/containers/storage/volumes DEBU[0000] Using transient store: false DEBU[0000] [graphdriver] trying provided driver "overlay" DEBU[0000] overlay: storage already configured with a mount-program DEBU[0000] backingFs=xfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=false DEBU[0000] Initializing event backend file DEBU[0000] Configured OCI runtime youki initialization failed: no valid executable found for OCI runtime youki: invalid argument DEBU[0000] Configured OCI runtime ocijail initialization failed: no valid executable found for OCI runtime ocijail: invalid argument DEBU[0000] Configured OCI runtime runj initialization failed: no valid executable found for OCI runtime runj: invalid argument DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument DEBU[0000] Configured OCI runtime crun-wasm initialization failed: no valid executable found for OCI runtime crun-wasm: invalid argument DEBU[0000] Using OCI runtime "/usr/bin/runc" INFO[0000] Setting parallel job count to 25 DEBU[0000] Successfully loaded 1 networks DEBU[0000] Adding port mapping from 8180 to 8080 length 1 protocol "" DEBU[0000] Pulling image dasvr (policy: missing) DEBU[0000] Looking up image "dasvr" in local containers storage DEBU[0000] Normalized platform linux/amd64 to {amd64 linux [] } DEBU[0000] Loading registries configuration "/etc/containers/registries.conf" DEBU[0000] Failed to decode keys ["registries.search.unqualified-search-registries"] from "/etc/containers/registries.conf" DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/000-shortnames.conf" DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/001-rhel-shortnames.conf" DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/002-rhel-shortnames-overrides.conf" DEBU[0000] Trying "localhost/dasvr:latest" ... DEBU[0000] parsed reference into "[overlay@***@***.***" DEBU[0000] Found image "dasvr" as "localhost/dasvr:latest" in local containers storage DEBU[0000] Found image "dasvr" as "localhost/dasvr:latest" in local containers storage ([overlay@***@***.***) DEBU[0000] exporting opaque data as blob "sha256:f10ec6c412dec67dddb6c80584a9325eee599dc60972a5bfdbb3451ed4c1edc0" DEBU[0000] Looking up image "localhost/dasvr:latest" in local containers storage DEBU[0000] Normalized platform linux/amd64 to {amd64 linux [] } DEBU[0000] Trying "localhost/dasvr:latest" ... DEBU[0000] parsed reference into "[overlay@***@***.***" DEBU[0000] Found image "localhost/dasvr:latest" as "localhost/dasvr:latest" in local containers storage DEBU[0000] Found image "localhost/dasvr:latest" as "localhost/dasvr:latest" in local containers storage ([overlay@***@***.***) DEBU[0000] exporting opaque data as blob "sha256:f10ec6c412dec67dddb6c80584a9325eee599dc60972a5bfdbb3451ed4c1edc0" DEBU[0000] Looking up image "dasvr" in local containers storage DEBU[0000] Normalized platform linux/amd64 to {amd64 linux [] } DEBU[0000] Trying "localhost/dasvr:latest" ... DEBU[0000] parsed reference into "[overlay@***@***.***" DEBU[0000] Found image "dasvr" as "localhost/dasvr:latest" in local containers storage DEBU[0000] Found image "dasvr" as "localhost/dasvr:latest" in local containers storage ([overlay@***@***.***) DEBU[0000] exporting opaque data as blob "sha256:f10ec6c412dec67dddb6c80584a9325eee599dc60972a5bfdbb3451ed4c1edc0" DEBU[0000] Inspecting image f10ec6c412dec67dddb6c80584a9325eee599dc60972a5bfdbb3451ed4c1edc0 DEBU[0000] exporting opaque data as blob "sha256:f10ec6c412dec67dddb6c80584a9325eee599dc60972a5bfdbb3451ed4c1edc0" DEBU[0000] Inspecting image f10ec6c412dec67dddb6c80584a9325eee599dc60972a5bfdbb3451ed4c1edc0 DEBU[0000] Inspecting image f10ec6c412dec67dddb6c80584a9325eee599dc60972a5bfdbb3451ed4c1edc0 DEBU[0000] Inspecting image f10ec6c412dec67dddb6c80584a9325eee599dc60972a5bfdbb3451ed4c1edc0 DEBU[0000] Inspecting image f10ec6c412dec67dddb6c80584a9325eee599dc60972a5bfdbb3451ed4c1edc0 DEBU[0000] using systemd mode: false DEBU[0000] setting container name dasvr DEBU[0000] Loading seccomp profile from "/usr/share/containers/seccomp.json" DEBU[0000] Allocated lock 1 for container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 DEBU[0000] parsed reference into "[overlay@***@***.***" DEBU[0000] exporting opaque data as blob "sha256:f10ec6c412dec67dddb6c80584a9325eee599dc60972a5bfdbb3451ed4c1edc0" DEBU[0000] Created container "8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933" DEBU[0000] Container "8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933" has work directory "/home/s-dmadminua/.local/share/containers/storage/overlay-containers/8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933/userdata" DEBU[0000] Container "8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933" has run directory "/tmp/containers-user-1293208/containers/overlay-containers/8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933/userdata" DEBU[0000] overlay: mount_data=lowerdir=/home/s-dmadminua/.local/share/containers/storage/overlay/l/XCX2VOTPIT5DKCKOBDAGKUJBKH:/home/s-dmadminua/.local/share/containers/storage/overlay/l/ICMDUICANYFC4HP5FX2RS2WJDL:/home/s-dmadminua/.local/share/containers/storage/overlay/l/EDYT3EKFVS33PIMRGVKRAGE3X5:/home/s-dmadminua/.local/share/containers/storage/overlay/l/EGHG57JCPSJK7R7RDJ6O4EK42U,upperdir=/home/s-dmadminua/.local/share/containers/storage/overlay/6a35548919ebc78cd665bb719c94542c7aeed998a173548d938e688a29d5ac4a/diff,workdir=/home/s-dmadminua/.local/share/containers/storage/overlay/6a35548919ebc78cd665bb719c94542c7aeed998a173548d938e688a29d5ac4a/work,,volatile DEBU[0000] Made network namespace at /run/user/1293208/netns/netns-fca018dd-c71f-c544-1878-00589c1519cd for container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 DEBU[0000] slirp4netns command: /usr/bin/slirp4netns --disable-host-loopback --mtu=65520 --enable-sandbox --enable-seccomp --enable-ipv6 -c -r 3 -e 4 --netns-type=path /run/user/1293208/netns/netns-fca018dd-c71f-c544-1878-00589c1519cd tap0 DEBU[0000] Mounted container "8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933" at "/home/s-dmadminua/.local/share/containers/storage/overlay/6a35548919ebc78cd665bb719c94542c7aeed998a173548d938e688a29d5ac4a/merged" DEBU[0000] Created root filesystem for container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 at /export/home/s-dmadminua/.local/share/containers/storage/overlay/6a35548919ebc78cd665bb719c94542c7aeed998a173548d938e688a29d5ac4a/merged DEBU[0000] rootlessport: time="2024-05-06T09:17:45-04:00" level=info msg="Starting parent driver" DEBU[0000] rootlessport: time="2024-05-06T09:17:45-04:00" level=info msg="opaque=map[builtin.readypipepath:/run/user/1293208/libpod/tmp/rootlessport1221606974/.bp-ready.pipe builtin.socketpath:/run/user/1293208/libpod/tmp/rootlessport1221606974/.bp.sock]" DEBU[0000] rootlessport: time="2024-05-06T09:17:45-04:00" level=info msg="Starting child driver in child netns (\"/proc/self/exe\" [rootlessport-child])" DEBU[0000] rootlessport: time="2024-05-06T09:17:45-04:00" level=info msg="Waiting for initComplete" DEBU[0000] rootlessport: time="2024-05-06T09:17:45-04:00" level=info msg="initComplete is closed; parent and child established the communication channel" time="2024-05-06T09:17:45-04:00" level=info msg="Exposing ports [{ 8080 8180 1 tcp}]" DEBU[0000] rootlessport: time="2024-05-06T09:17:45-04:00" level=info msg=Ready DEBU[0000] rootlessport is ready DEBU[0000] Not modifying container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 /etc/passwd DEBU[0000] Not modifying container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 /etc/group DEBU[0000] /etc/system-fips does not exist on host, not mounting FIPS mode subscription WARN[0000] Additional gid=10 is not present in the user namespace, skip setting it DEBU[0000] Setting Cgroups for container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 to user.slice:libpod:8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 DEBU[0000] reading hooks from /usr/share/containers/oci/hooks.d DEBU[0000] Workdir "/opt/dctm" resolved to host path "/export/home/s-dmadminua/.local/share/containers/storage/overlay/6a35548919ebc78cd665bb719c94542c7aeed998a173548d938e688a29d5ac4a/merged/opt/dctm" DEBU[0000] Created OCI spec for container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 at /home/s-dmadminua/.local/share/containers/storage/overlay-containers/8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933/userdata/config.json DEBU[0000] /usr/bin/conmon messages will be logged to syslog DEBU[0000] running conmon: /usr/bin/conmon args="[--api-version 1 -c 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 -u 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 -r /usr/bin/runc -b /home/s-dmadminua/.local/share/containers/storage/overlay-containers/8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933/userdata -p /tmp/containers-user-1293208/containers/overlay-containers/8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933/userdata/pidfile -n dasvr --exit-dir /run/user/1293208/libpod/tmp/exits --full-attach -s -l k8s-file:/home/s-dmadminua/.local/share/containers/storage/overlay-containers/8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933/userdata/ctr.log --log-level debug --syslog --conmon-pidfile /tmp/containers-user-1293208/containers/overlay-containers/8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /home/s-dmadminua/.local/share/containers/storage --exit-command-arg --runroot --exit-command-arg /tmp/containers-user-1293208/containers --exit-command-arg --log-level --exit-command-arg debug --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/user/1293208/libpod/tmp --exit-command-arg --network-config-dir --exit-command-arg --exit-command-arg --network-backend --exit-command-arg cni --exit-command-arg --volumepath --exit-command-arg /home/s-dmadminua/.local/share/containers/storage/volumes --exit-command-arg --db-backend --exit-command-arg boltdb --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg runc --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --events-backend --exit-command-arg file --exit-command-arg --syslog --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933]" INFO[0000] Running conmon under slice user.slice and unitName libpod-conmon-8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933.scope [conmon:d]: failed to write to /proc/self/oom_score_adj: Permission denied DEBU[0000] Received: -1 DEBU[0000] Cleaning up container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 DEBU[0000] Tearing down network namespace at /run/user/1293208/netns/netns-fca018dd-c71f-c544-1878-00589c1519cd for container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 DEBU[0000] Unmounted container "8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933" DEBU[0000] ExitCode msg: "runc: time=\"2024-05-06t09:17:46-04:00\" level=error msg=\"runc create failed: unable to start container process: unable to setup user: invalid argument\": oci runtime error" DEBU[0000] Removing container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 DEBU[0000] Cleaning up container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 DEBU[0000] Network is already cleaned up, skipping... DEBU[0000] Container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 storage is already unmounted, skipping... DEBU[0000] Removing all exec sessions for container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 DEBU[0000] Container 8997af1f0922652e3b46045e5a749ffd4c98541f872d5f111835e9ec77de9933 storage is already unmounted, skipping... Error: OCI runtime error: runc: time="2024-05-06T09:17:46-04:00" level=error msg="runc create failed: unable to start container process: unable to setup user: invalid argument" DEBU[0000] Shutting down engines From: dezza ***@***.***> Sent: Saturday, May 4, 2024 8:18 AM To: containers/podman ***@***.***> Cc: Yue, Brian ***@***.***>; Author ***@***.***> Subject: Re: [containers/podman] container create failed under podman 4.6.1 and conmon 2.1.8 (Issue #22597) Its a generic error you can narrow it down a little bit more by using --log-level=debug and strace I'm stuck in a similar situation (#22274 [github. com]), all forward upgrades will break it and there is no temporary file I know of that could Its a generic error you can narrow it down a little bit more by using --log-level=debug and strace I'm stuck in a similar situation (#22274 [github.com]<https://urldefense.com/v3/__https:/github.com/containers/podman/issues/22274__;!!MLsdJ25-fIk!qQtnSvgUdupEZpd1rSQ1XTrCBF00A4cJRuyPshr3iBgx3oAdPrFNRDTOiP_-SWP-zs33aEXV5NKM5EEPDXBrfnyeFp8$>), all forward upgrades will break it and there is no temporary file I know of that could hinder it. Mine started happening at 4.7.2 - yours could be a different issue if you look into it with above suggestions. — Reply to this email directly, view it on GitHub [github.com]<https://urldefense.com/v3/__https:/github.com/containers/podman/issues/22597*issuecomment-2094180722__;Iw!!MLsdJ25-fIk!qQtnSvgUdupEZpd1rSQ1XTrCBF00A4cJRuyPshr3iBgx3oAdPrFNRDTOiP_-SWP-zs33aEXV5NKM5EEPDXBrEOnnUi0$>, or unsubscribe [github.com]<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BAI5L4HWXATOOZBZ3FDSYJLZATNXFAVCNFSM6AAAAABHF6KTS2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJUGE4DANZSGI__;!!MLsdJ25-fIk!qQtnSvgUdupEZpd1rSQ1XTrCBF00A4cJRuyPshr3iBgx3oAdPrFNRDTOiP_-SWP-zs33aEXV5NKM5EEPDXBr9fn_nbA$>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>