Builder - Add zip support to ADD about buildkit HOT 9 OPEN

Automatic extraction (with ADD) only works for local archives; if you only need the files, not the archive, it's better to just add the extracted files, instead of a zip, and later extracting during docker build.

If you're adding a remote archive; use a multi stage build, and decompress in an intermediate build-stage

from buildkit.

I think this is a dupe of moby/moby#15036

As mentioned in that issue, ADD id not recommend best practice so I'm not sure it's worth extending it.

from buildkit.

I'm not married to ADD as the keyword but the scenario is pretty valuable.

from buildkit.

@taylorb-microsoft the problem is that you typically don't want exactly what's in a .zip. Instead, you want the zip, do some transformation on the contents (eg. compile source code), and then get rid of anything from the zip you don't need. To keep image-size minimal, you want to do that in a single layer.

Here's an example: https://github.com/docker-library/golang/blob/master/1.8/windows/windowsservercore/Dockerfile#L15

Details here: https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#add-or-copy

Do you have a specific use-case in mind?

from buildkit.

ADD has been frozen a few years ago. It's been discussed a bunch of times already on this issue tracker. The issue with ADD is that it has too much magic and it's pretty hard to keep track of all the edge cases when using the instruction in a Dockerfile.

ADD shouldn't be extended at all. Instead, new keywords which state the action to be taken explicitly should be introduced. COPY was the first instruction which was meant to help deprecate ADD in favor of specialized instructions. This means we should split up ADD into separate instructions. COPY already copies files. UNPACK or EXTRACT should handle various archives (.tar.gz, .tar.bz2, .tar.xz, .zip). Another instruction should exist for the retrieval of files from remote sources. This fetch instruction could also have a parameter which allow the archive to be extracted or just retrieved for storage as it is.

from buildkit.

Thanks, we are looking at bringing better compression tools support into Windows but it does seem like being able to accomplish these operations without starting a container might be beneficial.

from buildkit.

@taylorb-microsoft Perhaps this is something to discuss to come up with a new instruction for the Dockerfile. It really is a bad idea to overload ADD any further with any new functionality.

There's no reason why such functionality couldn't be added using a different new instruction (UNPACK, EXTRACT and so on). It's just that it needs a bit more effort to make sure we take into account all the various scenarios.

from buildkit.

But how do you integrate ZIP files then? AFAIK, the COPY instruction will create an image layer you do not need afterwards in most cases for this scenario.

This is a waste of space which will become noticable for large archives.

https://devops.stackexchange.com/questions/3172/copy-and-zip-files-in-dockerfile

from buildkit.

Automatic extraction (with ADD) only works for local archives; if you only need the files, not the archive, it's better to just add the extracted files, instead of a zip, and later extracting during docker build.

I need to be able to build under Windows and the NTFS filesystem mapping may not preserve the original metadata exactly, hence I would like to unpack an archive instead.

from buildkit.