patrickhoefler / dockerfilegraph Goto Github PK

Similarly to #19, it would be useful to also support bind mounts.

Currently, only the build stages of the Dockerfile are included in the graph output. It would be super awesome if the external images that the stages are based on are also shown in the graph output. They should look slightly different so that they can easily be distinguished from the internal build stages.

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: renovate.json
Error type: Invalid JSON (parsing failed)
Message: Syntax error: expecting end of expression or separator near "] "majo

I have a use case for which I need to add several groups of
artifacts to the final image, each coming from a given stage,
like in the test case where we have download-node-setup and
download-get-pip.
However, I'd like each stage to be self-contained within
a given portion of the Dockerfile. In other words, I don't want all

COPY --from=stageX ...

statements to be at the end of the Dockerfile, but I'd rather
keep them at the end of the stage that generates them.

For this I'm essentially using the (perhaps unorthodox) pattern:

FROM baseimage AS final
RUN somestuff

FROM ext1 AS build-foo
RUN whatever1
FROM final AS final
COPY --from=build-foo ...

FROM extN AS build-bar
RUN whateverN
FROM final AS final
COPY --from=build-bar ...

In this cause though, the current implementation will
create N copies of the final stage.

PR on its way with my suggested changes.

Considering the following reproduction:

FROM docker.io/library/alpine:latest AS base
RUN touch base

FROM base AS base2
RUN touch base2

FROM base AS base3
RUN touch base3

FROM base AS base4
RUN touch base4

FROM base
RUN --mount=from=base2,target=/base2 \
    --mount=from=base3,target=/base3 \
    --mount=from=base4,target=/base4 \
    cp base*/* .
RUN ls base*

the final dag rendered only shows one of the --mount-from connections.

When creating the graph with --layers, the edges currently point to invisible nodes that represent the build stage. Instead, the edges should point from the external base image or the dependent build stage directly to the corresponding layer, as visualized in #82.

Current Situation

As reported in #272

At the moment, the waitFor dependencies — the edges in the graph — are only possible between stages, but for #82 they need to be drawn between stages and layers.

The output file could use a legend for the arrow types! I know they're documented in the readme, but having them embedded / on hand would be great!

Just came across this library 5 minutes ago - the docker run example worked perfectly for me, thank you!

This issue provides visibility into Renovate updates and their statuses. Learn more

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Pin dependencies (actions/checkout, actions/setup-go, docker/login-action, github/codeql-action, golang, golangci/golangci-lint-action, goreleaser/goreleaser-action, ubuntu)

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

dockerfilegraph should have a --version flag.

Introduction

Hi! Big kudos for creating this tool. I would like to suggest a feature. It is popular in my area that we specify versions of images as arguments. So they can be easily changeable. What is seen is that dockerfilegraph does not resolve them and for graphs, I would expect them to be resolved. I am not aware of how the tool works itself, but maybe we could analyze the Dockerfile after the first compilation? After the arguments are resolved. Please let me know what you all think about it.

Chart

Dockerfile

ARG PHP_VERSION=8.0
ARG ALPINE_VERSION=3.15
ARG COMPOSER_VERSION=2
ARG PHP_EXTENSION_INSTALLER_VERSION=latest

FROM composer:${COMPOSER_VERSION} AS composer
FROM mlocati/php-extension-installer:${PHP_EXTENSION_INSTALLER_VERSION} AS php_extension_installer

FROM php:${PHP_VERSION}-fpm-alpine${ALPINE_VERSION} AS base

FROM scratch AS foo

FROM scratch AS bar
RUN --mount=type=cache,from=foo,target=/mnt

renders as

But

FROM scratch AS foo

FROM scratch AS bar
RUN --mount=type=cache,target=/mnt,from=foo

does not recognize the link between the stages and renders as

This Dockerfile,

ARG $BASE
FROM $BASE

produces this graph:

This very similar file,

ARG ${BASE}
FROM ${BASE}

instead produces the following error:

Oh no, something went wrong while generating the graph!

                                        This is the Graphviz file that was generated:

                                        digraph G {
        compound=true;
        nodesep=1;
        rankdir=LR;
        external_image_0->stage_0;
        external_image_0 [ color=grey20, fontcolor=grey20, label="${BASE}", shape=Mrecord, style=dashed, width=2 ];
        stage_0 [ fillcolor=grey90, label="0", shape=Mrecord, style=filled, width=2 ];

}

                                        The following error was reported by Graphviz:

                                        Error: bad label format ${BASE}

This turns out to be because the Mrecord node shape causes the label to be parsed as a record-based node, and curly braces are special syntax for that sort of node.

I don't see that the fancy record-based node features are being used anywhere, so I am guessing Mrecord is being used to get a rounded rectangle shape. That can be accomplished with shape=box, style=rounded instead, without limiting the label format.

Hi there,

I was wondering if you will build an Alpine docker image since will be smaller in size.

The current image with ubuntu focal is:

ghcr.io/patrickhoefler/dockerfilegraph   latest            0455eb60381c   2 weeks ago         111MB

I think with Linux Alpine could be even smaller.

Anyway, thanks for your project. It is awesome.

This is a follow up of #258.

To have a better benchmark I created a skeleton Dockerfile from our real-world 300 lines Dockerfile:

FROM --platform=linux/amd64 ubuntu:22.04 AS ubuntu_amd64

FROM --platform=linux/arm64 ubuntu:22.04 AS ubuntu_arm64

FROM ubuntu:22.04 AS ubuntu_with_amd64_emulation
COPY --from=ubuntu_amd64 /etc/apt/sources.list /etc/apt/sources.list.amd64
COPY --from=ubuntu_arm64 /etc/apt/sources.list /etc/apt/sources.list.arm64
COPY --from=ubuntu_amd64 /usr/bin/cat ./cat
COPY --from=ubuntu_arm64 /usr/bin/cat ./cat

FROM alpine:3.12.0 AS a_dependency

FROM ubuntu_with_amd64_emulation AS base
COPY --from=a_dependency /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.14 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.14.1 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.14.2 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.14.3 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.14.4 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.14.5 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.14.6 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.14.7 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.14.8 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.15 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.15.1 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.15.2 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.15.3 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.15.4 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.15.5 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.15.6 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.16.1 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.16.2 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.16.3 /bin/cat /tmp/cat
COPY --from=docker.io/alpine:3.16 /bin/cat /tmp/cat

FROM --platform=$BUILDPLATFORM ubuntu:22.04 AS build_buildplatform
RUN --mount=type=cache,target=/root/.m2/repository echo mvn dependencies ...
RUN --mount=type=cache,target=/root/.m2/repository echo mvn package ...

FROM base as test
COPY --from=build_buildplatform /bin/cat /tmp/cat
COPY --from=build_buildplatform /bin/cat /tmp/cat
COPY --from=build_buildplatform /bin/cat /tmp/cat
RUN --mount=type=cache,target=/root/.m2/repository echo mvn verify ...

FROM base
COPY --from=build_buildplatform /bin/cat /tmp/cat
COPY --from=build_buildplatform /bin/cat /tmp/cat

This file will render the same nodes and edges as the original and it also includes a couple of aspects of the original.
Feel free to include this file as a benchmark in your repo.

The rendering currently looks like this (dockerfilegraph -c):

(Note that I updated the Dockerfile above with real alpine versions so it would build. The images here are generated with fake tags.)

If I play with the flatten option, it doesn't really get more compact/readble (dockerfilegraph -c -u 4):

The output I prefer looks like this and it is achieved by setting nodesep=0.25,ranksep=2 in the canon file generated by dockerfilegraph -c -o canon:

So I would love to see an option to override nodesep and ranksep directly from the dockerfilegraph CLI. Maybe even allow overwriting any graph-wide settings.

Another thing to notice is that the nodes for external images don't grow with the image name, but the image name is elided aggressively, so it is hard (and sometimes impossible) to discern the images. This is a problem in the real-world file as well and there it is even more pronounced, since all images use 8-character git hashes as tags. Here, I would like to see an option to set the maximum text length of a node.

Finally, this Dockerfile uses --platform=.... Visually indicating that --platform is used for a node, would help to understand the multi-arch behaviour of this Dockerfile better. I could imagine changing the background color and/or allowing renderings for each target platform individually.

Would be awesome if we could see buildkit's mount=type=cache as edges. So take the running example in the README.md; imagine current using a cache with the following Dockfile directive:

RUN --mount=type=cache,from=builderbase,source=/go/pkg/mod/cache/,target=/go/pkg/mod/cache/

and this could be represented with a relation

builderbase -> current [label="cache, from ...."]

Or the (stage,cache directory) tuple could be its own node in the graph.

When I use multiple copies of an external image, multiple nodes pop up for them, even though edges only come from one node.

For example,

# syntax=docker/dockerfile:1.4

FROM scratch AS download-node-setup
ADD https://deb.nodesource.com/setup_16.x ./

FROM scratch AS download-get-pip
ADD https://bootstrap.pypa.io/get-pip.py ./

FROM alpine AS final
COPY --from=download-node-setup setup_16.x ./
COPY --from=download-get-pip get-pip.py ./

makes two nodes for "scratch":

Hi, thank you for this excellent tool. I used it to visualize a huge Dockerfile with many stages. While doing so, I wished for a couple of extra options.
But here is the result out of the box:

I would like to see a way of grouping multiple COPY --from=.. arrows into one arrow only to improve readability. I manually edited the canon file to add concentrate=true, but this overlays FROM ... and COPY ..., so they aren't traceable anymore.

If grouping would be more intelligent, I think dashing / coloring the line for e.g. COPY --from=... would make it even more readable as I don't have to look to closely at the arrow heads.

Finally, I would like to see svg export. I am exporting to canon currently and execute dot afterwards.

My workaround is this dockerfile, which when build with docker build -o . . exports the result into the current directory:

FROM ghcr.io/patrickhoefler/dockerfilegraph:alpine AS build
WORKDIR /workspace
COPY the-dockerfile .
RUN /dockerfilegraph --filename the-dockerfile --output canon
RUN <Dockerfile.canon sed 's/rankdir=LR/rankdir=LR,concentrate=true/g' >Dockerfile.canon.patch
RUN dot -Tsvg Dockerfile.canon.patch -o Dockerfile.svg

FROM scratch
COPY --from=build /workspace/Dockerfile.svg /Dockerfile.svg

I have committed and pushed the Dockerfile.pdf to my git repo.

Now when I run the same command on the same Dockerfile, there are changes in git.

docker run \
  --rm \
  --user "$(id -u):$(id -g)" \
  --workdir /workspace \
  --volume "$(pwd)":/workspace \
  ghcr.io/patrickhoefler/dockerfilegraph:alpine

It's also worth noting that I move the generated .PDF to another directory after the command is done:

mv Dockerfile.pdf ./docs/

Do you know why the Dockerfile.pdf is changing even though my Dockerfile has not? Could the library include a timestamp?

Not sure if this helps at all, but the binary changed near the bottom of the RAW binary code.

Before binary:

stream
x�5�=��0������u1yS?Z��ݺH���!h�B1�������r�q������/sP�������
�)�W��_�a����7�B��m� Vׁ���^;D����%T$9���e-9��qj��~M�3q|�8��d�Vy��-��'�RAR���#���F���mM ��hW�G_'a4�
endstream
endobj

After binary:

stream
x�5�=��0������u1yS?P�E�.R:�t���P��8�߷F��p�s<��(C����欮��U�������y�����
�P�v_<�5M zg�}��ѠfgA	�I�h�~�*�Ck�Z�y���L��7N+?ۥS^#�*)�E�T���,����4j�Wok���@��1��'�4~
endstream

Hi Patrick,

Greetings.

I wanted to ask you if the dockerfilegraph project could be used to draw layers per stage as well.

As reported in #272

Spotted in #272

Currently there is only an empty error message, which is not really that helpful.