Giter Club home page Giter Club logo

docker-multi-arch-example's Introduction

Docker multi-arch example

This repository demonstrates how to create a multi-arch Docker image supporting multiple platforms, e.g. both x86_64 and ARM, but building all images on an x86_64 platform. It is primarily for personal use for reference on how to build manifests and how they work.

This example also only relies on Docker Hub to build all images, including the ARM variants, and does not rely on separate build servers or environments to build non-amd64 images.

It's a TL;DR version of the references linked in the README.

Steps

To create a multi-arch image, follow the next 4 steps

  1. Build and push the image for the amd64 platform
  2. Build and push the image for the arm* platforms
  3. Create the manifest for the multi-arch image
  4. Push the maniest for the multi-arch image

Automated builds via Docker Hub

The Docker Hub Custom build phase hooks allow in combination with QEMU an entirely automated build of the Docker images via Docker Hub for all major platforms - as it is used in this repository.

To see how it's done in this repository, see

Use the following build rules in Docker Hub for each multi-arch image:

  • One build rule per Dockerfile/platform, e.g. amd64.dockerfile
  • The Docker Tag indicates the platform, e.g. amd64
  • No build rule for the (multi-arch) image tag in the manifest, e.g. latest

Docker Hub Build rules

Note that the builds may fail at the beginning until each build rule ran once. This can be because the manifest references images that don't exist yet. Hence, after running all of them, each referenced image exists and the manifest can be pushed succesfully.

Push multi-arch manifest automatically (Docker Hub)

Once Docker Hub has published the amd64 and arm* images, Docker Hub executes the hooks/post_push script.

The script downloads the manifest-tool and pushes the multi-arch manifest multi-arch-manifest.yaml, which - simply put - makes ckulka/multi-arch-example:latest a list containing references to the various platform variants.

To see how it's done in this repository, see

Push the multi-arch manifest manually (manifest-tool)

If you want to push the multi-arch manifest yourself using the manifest-tool, here are the steps:

  1. Create the mulit-arch manifest file, e.g. multi-arch-manifest.yaml
  2. Download the manifest-tool
  3. Push the manifest using manifest-tool
# Download the manifest-tool
curl -Lo manifest-tool https://github.com/estesp/manifest-tool/releases/download/v0.9.0/manifest-tool-linux-amd64
chmod +x manifest-tool

# Push the multi-arch manifest
./manifest-tool push from-spec multi-arch-manifest.yaml

# On Docker for Mac, see https://github.com/estesp/manifest-tool#sample-usage
./manifest-tool --username ada --password lovelace push from-spec multi-arch-manifest.yaml

For more details, see manifest-tool: Create/Push.

Push the multi-arch manifest manually (Docker CLI)

If you want to push the multi-arch manifest yourself using the Docker CLI, here's how.

As long as the docker manifest commands are experimental, enable the experimental mode for the Docker CLI in ~/.docker/config.json first:

{
  "experimental": "enabled"
}

Next up, create and publish multi-arch the manifest.

# Create and push the manifest
docker manifest create ckulka/multi-arch-example:latest ckulka/multi-arch-example:amd64 ckulka/multi-arch-example:arm32v7
docker manifest push --purge ckulka/multi-arch-example:latest

The created manifest acts as a reference for the linked images. The Docker client, when pulling ckulka/multi-arch-example:latest, looks up a "fitting" image and then uses that one.

The --amend parameters allows adding additional platforms:

docker manifest create --amend ckulka/multi-arch-example:latest ckulka/multi-arch-example:arm64v7

The --purge parameter deletes the local manifest, which allows recreating and subsequently replacing the list:

# Release version 1.0 as latest image variant
docker manifest create ckulka/multi-arch-example:latest ckulka/multi-arch-example:1.0-amd64 ckulka/multi-arch-example:1.0-arm32v7
docker manifest push --purge ckulka/multi-arch-example:latest

# Release version 2.0 as latest image variant
docker manifest create ckulka/multi-arch-example:latest ckulka/multi-arch-example:2.0-amd64 ckulka/multi-arch-example:2.0-arm32v7
docker manifest push --purge ckulka/multi-arch-example:latest

Inspecting the result

The docker manifest inspect command shows the image manifest details - for the multi-arch image, it's the list of images it references and their respective platforms:

docker manifest inspect ckulka/multi-arch-example:latest
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
  "manifests": [
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "size": 1728,
      "digest": "sha256:3a859e0c2bdfb60f52b2c805e2cb55260998b3c343d9e2ea04a742d946be1b1e",
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "size": 1728,
      "digest": "sha256:3ef9264d6e5ad96ad7bac675d40edf265ae838ae6ca60865abed159c8c5124c8",
      "platform": {
        "architecture": "arm",
        "os": "linux"
      }
    }
  ]
}

Known limitations

Building non-x86_64 images on Docker Hub results in images with incorrectly labelled architectures.

For example, the arm32v7 image runs on arm32v7, but will labelled as amd64, because Docker Hub builds the images on x86_64. There is a workaround for it, but I've not added it here to keep things simple.

This issue is currently tracked in moby/moby#36552.

References

docker-multi-arch-example's People

Contributors

ckulka avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

docker-multi-arch-example's Issues

How do you configure the Docker Hub automated build?

I've tried to replicate your approach in this repo. It was configured to build ./Dockerfile which now doesn't exist anymore, but the configuration insists on knowing about one dockerfile, so I just chose one:
image

This seems wrong to me but at least the build starts. It fails on the post_push hook which might be a different issue, I am not sure. Can you tell me if I am on the right track?

Handeling tags

Hey,
First thanks for this repo and explanation, I have implemented it for https://github.com/guysoft/CustomPiOS .

The thing is - I am preparing for a release. Which means that once I push a tag dockerhub would build and publish a new image tag. However the manifest setup is built only to handle a hard-coded tag name.

Is there a way to handle post push tagging? Which does exist on dockerhub?

I would think there is an env variable to find out what tag is made and then string-replace with something like sed a palceholder in a committed yaml file.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.