Comments (5)
I remember briefly looking at this a while ago and ending up somewhere in the shlex
package. The \
character is defined as the escape character. But I really didn't dig too much into this.
While almost all Dockerfile stanzas will be fine with using /
instead of \
one notable exception is the RUN
stanza. Some Windows commands are really adamant about using actual \
as a path separator. In most cases, if we use:
# Actuall SHELL stanza parameters may differ
SHELL ["powershell.exe", "-Command"]
and use PowerShell as a shell instead of cmd.exe
, we can get away with using /
as a path delimiter for most commands. In any case, I may have some time next week to have a look.
from buildkit.
I had started looking into this in #1621, but didn't get much further than some tests and a draft resolution approach that was determined to be the wrong approach. The branch is still around if anyone wants to look at it, as I suspected the issue was still around. See also #616 (comment)
That said, this one in particular might be not a problem with COPY per-se, but with the end-of-line continuation character. Does
FROM mcr.microsoft.com/windows/nanoserver:ltsc2022
COPY hello.txt C:\\
CMD ["cmd", "/C", "type C:\\hello.txt"]
work?
from buildkit.
That error suggests we're failing at https://github.com/moby/buildkit/blob/master/util/system/path.go#L190-L193.
I suspect, based on rough recollection of #1621 investigations, that the parse correctly produced C:\
, but a use of path.clean
or locally-coded equivalent on that string later produced C:\/
, and that was normalised to \/
, and that was given to CheckSystemDriveAndRemoveDriveLetter
, which in that linked code turns it into //
and hence looks like a (null) UNC path. We don't see this in Docker's built-in builder because it doesn't go through so many layers (no dockerfile2llb) so it doesn't hit this issue. (I suspect in Docker we are simply using filepath.clean
because we don't actually support cross-OS image builds, so the daemon can assume hostOS == targetOS
and use filepath
directly.)
I don't think we saw this when building Windows containers on Linux though, but I don't remember why.
The discussion in #1621 for this sort of issue was that dockerfile2llb should take care of this sort of thing, and LLB should only see unix-style paths. During container build steps that aren't RUN
, we can assume only C: is present, so the "strip drive letter" logic could probably also be moved into the dockerfile2llb layer (and barf there if someone tries to COPY to a different drive letter, or UNC path, etc.)
(Edit: Just noticed that this failure is probably happening during dockerfile2llb processing (dispatchCopy
calling system.NormalizePath
I think) so structurally it's already trying to fix the paths before they're seen by LLB. The "Fix should be in dockerfile2llb" comment was about my attempted fix in #1621, which was in the instruction parser, i.e. earlier in the stack. Also, a bunch of code has moved since then, so my fix might not even be applicable. My test-case was in dockerfile2llb at least.)
Also, I'll note that my repro in #1621 was with WORKDIR
, so that one's also worth including in tests. I imagine things like the --mount
argument's target path in RUN
have similar risks, but they might just be flowing through in to execution arguments verbatim.
from buildkit.
@TBBle -- thanks for taking a look. You are right, the issue is with C:\
, the backslash not being escaped, so C:\\
it is. So it's not a COPY stanza thing.
However, this dockerfile works with docker build
but fails with buildctl build
:
FROM mcr.microsoft.com/windows/nanoserver:ltsc2022
COPY hello.txt C:\\
CMD ["cmd", "/C", "type C:\\hello.txt"]
I get this:
buildctl build `
>> --output type=image,name=docker.nandaa.dev/test,push=false `
>> --progress plain "-frontend=dockerfile.v0" `
>> --local context=C:/dev/play/dockerfiles/repro-4696 `
>> --local dockerfile=C:\dev\play\dockerfiles\repro-4696
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 151B done
#1 DONE 0.0s
#2 [internal] load metadata for mcr.microsoft.com/windows/nanoserver:ltsc2022
#2 DONE 1.8s
#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s
#4 [internal] load build context
#4 transferring context: 43B done
#4 DONE 0.0s
#5 [1/2] FROM mcr.microsoft.com/windows/nanoserver:ltsc2022@sha256:6e6053f0358f9522d2d14693f9bc152f47fe04c82c53dc8c6d127a5a823c8720
#5 resolve mcr.microsoft.com/windows/nanoserver:ltsc2022@sha256:6e6053f0358f9522d2d14693f9bc152f47fe04c82c53dc8c6d127a5a823c8720 0.1s done
#5 CACHED
#6 [2/2] COPY hello.txt C:\
#6 ERROR: cleaning path: removing drive letter: UNC paths are not supported
------
> [2/2] COPY hello.txt C:\:
------
Dockerfile:2
--------------------
1 | FROM mcr.microsoft.com/windows/nanoserver:ltsc2022
2 | >>> COPY hello.txt C:\\
3 | CMD ["cmd", "/C", "type C:\\hello.txt"]
4 |
--------------------
error: failed to solve: cleaning path: removing drive letter: UNC paths are not supported
from buildkit.
@TBBle -- yes, looks close. Found this on my debugging, Dest: "/\\"
:
> github.com/moby/buildkit/solver/llbsolver/ops.(*FileOpSolver).getInput() C:/dev/container-core/buildkit/solver/llbsolver/ops/file.go:412 (hits goroutine(764):1 total:1) (PC: 0x229c996)
411:
=> 412: func (s *FileOpSolver) getInput(ctx context.Context, idx int, inputs []fileoptypes.Ref, actions []*pb.FileAction, g session.Group) (input, error) {
...
(dlv) p actions[0].Action
github.com/moby/buildkit/solver/pb.isFileAction_Action(*github.com/moby/buildkit/solver/pb.FileAction_Copy) *{
Copy: *github.com/moby/buildkit/solver/pb.FileActionCopy {
Src: "/hello.txt",
Dest: "/\\",
Owner: *github.com/moby/buildkit/solver/pb.ChownOpt nil,
Mode: -1,
FollowSymlink: true,
DirCopyContents: true,
AttemptUnpackDockerCompatibility: false,
CreateDestPath: true,
AllowWildcard: true,
AllowEmptyWildcard: true,
Timestamp: -1,
IncludePatterns: []string len: 0, cap: 0, nil,
ExcludePatterns: []string len: 0, cap: 0, nil,
AlwaysReplaceExistingDestPaths: false,},}
(dlv)
Further up:
> github.com/moby/buildkit/client/llb.(*fileActionCopy).toProtoAction() C:/dev/container-core/buildkit/client/llb/fileop.go:526 (PC: 0x17adb8a)
521: if err != nil {
522: return nil, err
523: }
524: c := &pb.FileActionCopy{
525: Src: src,
=> 526: Dest: normalizePath(parent, a.dest, true),
527: Owner: a.info.ChownOpt.marshal(base),
528: IncludePatterns: a.info.IncludePatterns,
529: ExcludePatterns: a.info.ExcludePatterns,
530: AllowWildcard: a.info.AllowWildcard,
531: AllowEmptyWildcard: a.info.AllowEmptyWildcard,
(dlv) p a.dest
"/\\"
Looks like normalizePath()
isn't doing a good job here; I'm sending a fix.
from buildkit.
Related Issues (20)
- wishlist: annotations on docker/dockerfile images HOT 2
- WCOW fails to load custom frontend HOT 3
- Proposal: Use referrers api to import/export inline type cache
- bake does not merge compose services/<service>/build/x-bake setting with serivice extensions
- copy --parents silently ignores nonexistent paths HOT 2
- buildkit WCOW cannot seemingly run `RUN powershell ...`, while vanilla dockerd can HOT 4
- Proposal: csv syntax for git repos HOT 3
- Support extracting `ADD --checksum=.. https://.. ..` HOT 6
- 0.13.2 corrupts cache and cannot start (`go.etcd.io/bbolt.(*freelist).read`, `panic: invalid freelist page: 0, page type is unknown<0`) HOT 4
- Local and Registry cache not used or *invalidating* cache unnecessarily HOT 3
- can't add CA cert for gha remotecache?
- Inconsistencies in RUN mount options parsing/handling HOT 1
- dockerfile: Add integration test for ARG with empty value and default
- [v0.13] It seems that "registry.insecure=true" doesn't work. HOT 7
- buildkit remote cache fails if manifest is larger than 1MB
- Windows buildkit: copy context error HOT 6
- Proposal: lint disable controls HOT 2
- windows differ is not implemented HOT 1
- `FROM --platform=$BUILDPLATFORM a as b` not overriding `TARGETPLATFORM` for multi-stage builds HOT 7
- `checksum`ing "local files" for remote builder contexts HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from buildkit.