Comments (2)
clemenskol
commented on May 27, 2024
FYI, I looked at other tickets with a similar problem (e.g., #2336), but either the root-cause described in those tickets is different or the proposed work-around did not work for me.
I have tried many different work-arounds, and none worked for me (aside from touching every file in the file-system, but this is not an option for me)
from kaniko.
clemenskol
commented on May 27, 2024
another observation: if I change this to a multi-stage build AND I do more than just use RUN commands, then it sometimes works. Yes, I execute the same build twice in a row, and it seems I have about a 50% chance to get a working container image
Dockerfile:
# syntax=docker/dockerfile:1
FROM amd64/ubuntu as stage
RUN \
apt update && \
apt install \
--no-install-recommends \
--assume-yes \
python3.11
FROM stage as final
ADD .ignore /.ignore
Build command (note that I'm not pushing remotely just to safe roundtrip time, pushing to a remote registry has the same outcome):
/kaniko/executor \
--context /workspace \
--dockerfile Dockerfile \
--destination kamiko-test:3 \
--no-push \
--tar-path output.tar \
--target final \
-v debug
Test command:
docker image rm kamiko-test:3
docker load -i output.tar
docker run --rm -ti kamiko-test:3 bash -c '/usr/bin/python3.11 --version'
In some cases we get this result (installation worked):
Untagged: kamiko-test:3
Deleted: sha256:c42801f9c6b74e0dd7002f9439d0e2675fddc2070665f5646b0303e5e9277a01
Deleted: sha256:58ee2628caa0ebb2dd0b9ee2893bb7f6a3996ed8b41177a209154b270e2952f5
Deleted: sha256:c6a78351595ae2bb76e7284ec47f720e5b7d7e9f66ffab997d24436d143c491d
e2b5084e6f6a: Loading layer [==================================================>] 49.89MB/49.89MB
e74d10928493: Loading layer [==================================================>] 259B/259B
Loaded image: kamiko-test:3
Python 3.11.0rc1
In other cases we get this result (installated files were not committed to the snapshot/image):
Untagged: kamiko-test:3
Deleted: sha256:ac778b382fa91f37cfb3d35e2d56d0a52531fb42082b7e2226e44858b0167f29
Deleted: sha256:a1a681b7fa20e5528304dfe34897ebac67a8f4ff3ecceaf6774445c6fd37fe18
Deleted: sha256:6262b815a55b0dc3bb6679ac18aa94d9aa3fa1074357640627318925a53d05af
e9f9bcb2687e: Loading layer [==================================================>] 6.344MB/6.344MB
f81778963cd0: Loading layer [==================================================>] 252B/252B
Loaded image: kamiko-test:3
bash: line 1: /usr/bin/python3.11: No such file or directory
As said, it's random and about 50% to have the one or other outcome. And even more weirdly, it seems to alternate if it works or if it fails. As if a cache would corrupt and then uncorrupt itself (note that in these experiments the cache is off).
I have captured the stdout (build command outputs) + stderr (kamiko debug-verbosity logs) from a successful and failing build.
The stdout build command output is essentially identical (aside from the download/timing info from apt)
The stderr kamiko debug output is very different however and once contains the expected binary in one of the logs but not the other
from kaniko.
Related Issues (20)
- Recipe for adding bash to Kaniko HOT 1
- Required Socket Files Ignored
- mktemp present but non-functional HOT 3
- error building image: stream error: stream ID 13; NO_ERROR; received from peer
- During image push and pull, error building image: stream error: stream ID 13; NO_ERROR; received from peer
- Even when `KANIKO_NO_PUSH="true"` there is a info message with "Pushed"
- Creating a custom kaniko image using a multi-stage Dockerfile fails
- [TEST FLAKE] Intermittently seeing - `No such image: localhost:5000/<test-image>` HOT 3
- [Bug] --registry-mirror does not work with warmer since v1.21.0 HOT 1
- error removing bin to make way for new symlink: unlinkat //bin/sh: permission denied
- Question regarding a vulnerability (CVE-2023-45288), vuln scans & results publishing HOT 1
- Setting `--kaniko-dir` (or `KANIKO_DIR`) clobbers `DOCKER_CONFIG` if it is explicitly set. HOT 2
- Execution takes too long for "Unpacking rootfs as cmd COPY" HOT 1
- "GET https://example.com/users/sign_in: unexpected status code 200 OK" when pushing to self-hosted GitLab with Kaniko HOT 5
- compression=zstd not working starting from v1.16
- After run kaniko in container, image is changed and part of local context stop to work HOT 2
- using Kaniko with a registry-mirror or registry-map which includes a path does not work for pulling images from DockerHub HOT 1
- Cannot pass empty value of --build-arg
- error pushing image: MANIFEST_BLOB_UNKNOWN: blob unknown to registry;
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kaniko.