hercules-ci / arion Goto Github PK

Run docker-compose with help from Nix/NixOS

License: Apache License 2.0

Nix 68.81% Shell 0.71% Dockerfile 0.09% Haskell 30.39%

arion's Introduction

Arion is a tool for building and running applications that consist of multiple docker containers using NixOS modules. It has special support for docker images that are built with Nix, for a smooth development experience and improved performance.

Intro and Documentation

arion's People

Contributors

Stargazers

Watchers

arion's Issues

Figure out why `<unknown-file>` instead of real filename is reported on errors

NixOS systemd as default?

Try to make nixos.useSystemd the default when nixos.configuration is set by user.
May need to distinguish between user-created modules (intent: do NixOS) and builtin modules that only tweak the config (intent: if doing NixOS, do it better). We can do so by creating a separate nixos.autoConfiguration option.

If this turns out to be too messy, we should probably create a required option to be explicit about the container "type". Setting something like type = "nixos" everywhere is less painful than wondering why configuration doesn't apply.

Option to not build actual docker tarballs via nixlang invocation

Omitting the tarballs saves a lot of churn on CI. You do lose some testing of course, but unless you're experimenting with the dockerTools implementation you're probably better off not building the tarballs. They're as big as your entire closure but slightly different for each little change.

Integrate the container's systemd with the host's systemd

It would be nice to have a deep integration between the host's and containter's systemd such that the host's systemd recognises the docker containers:

machinectl
 MACHINE                          CLASS     SERVICE
 9aa1f884bdae630ecaee76c0ca85441f container docker
 cb7c2ddeba36e450d4214f51a8529e18 container docker

And we can see the container's logs on the host using:

journalctl --machine <container_id>

To implement this we could consider the tricks that RedHat pulled off to run systemd inside unprivileged docker containers.

In particular they used runc hooks to register the container with machinectl on the host and another hook (oci-systemd-hook) that does a bunch of things including mounting /var/log/journal into the container so that the journal entries from inside the container are visible on the host using.

Make it work on other Linux

Current implementation of useHostStore relies on NixOS symlinks to make it work. This is not necessary.
Instead, we can provide symlinks to a special location in the base image like /.arion-symlink-targets/{sh,env} and insert the symlinks to the binaries right before invoking the user command.

Evaluation cache (faster logs, restart etc)

I'm a bit annoyed that I have to eval arion everything I want to see logs. What's the reason for evaluation, can't it use the cached version of the yaml? There are other commands than logs that just query information.

environment.systemPackages ignored

{ pkgs, ... }:
{
  docker-compose.services = {
    webserver = { config, pkgs, ... }: {
      nixos.configuration = {config, pkgs, ...}: {
          boot.isContainer = true;
          environment.systemPackages = with pkgs; [
            tree
          ];
      };
      service = {
        command = [  "sh" "-c" "tree" ];
        useHostStore = true;
      };
    };
  };
}

Since environment.systemPackages seems to be ignored, what's the recommended way to manage dependencies?

Exec command requires `--` most of the time

Format JSON also via Nix

Currently only the arion command prints out formatted JSON.
If you generate the JSON via Nix only, it's a single line. Not terrible, not great.

Make sure to build new systemd with the right compile time options

Check the super version and try to add some overrides. (If not already configured for compatibility by NixOS/Nixpkgs itself)

https://twitter.com/ProgrammerDude/status/1142869595642048512

A more permanent GC root?

services outlive the arion command and may use the store
arion logs and friends are slower than necessary because of evaluation can be solved by having a known-good compose file around
better interop with Docker Compose and its ecosystem?

Upstream assert.nix back into NixOS as refactor

Makes assertions / warnings available as independent module. Exposes a function to apply their 'side effects'.

Use of nix-instantiate causes rare IFD error

Workaround: run a command that builds the yaml instead of instantiating. This causes the expression to be built and subsequent nix-instantiate to find it.

Complete the example section in README

Add some command invocations
Reference todomvc-nix for the actual files

Add test cases

Add test cases:

docker-compose json with all fields set
assertion at composition level
assertion at service level
same for s/assertion/warning

Secrets with ssh deployment

Local docker compose v3 deployment emulates secrets with bind mounts. This will not work for remote docker. We can work around this by creating volumes for each service that has secrets and adding an extra internal service for the purpose of receiving and writing the secrets.

Rename to composition

Use the word composition where applicable. Democracy ;)

Make project structure conform to todomvc-nix

Make project structure conform to todomvc-nix to a larger degree. Having a more familiar project structure should help everyone.

Relay docker-compose --help in command help

This is a regression compared to the bash script.

'Downgrade' function from kubenix to arion

Doing some translation seems quite feasible.

The goal is to have an alternative to, say, minikube. So really just for local development.

I can think of these approaches

big adapter: write a translation function that takes an entire kubenix and converts it to the right arion things, essentially replacing the arion frontend
small adapter: allow inclusion of specific kubenix objects into a composition or service
compatibility: modify arion naming to match kubenix objects

Disclaimer: this will never support all k8s functionality of course, but it may help with local development.

Support nix store on volume

Bring a special init and a nix store for copying.
Can we access a directory under a volume mount? If not, store the store in an alternate location, as a local store or file store.

Interesting useHostStore twist here. Host store may be mounted at the alternate location.
Can't seem to access what's under a volume mount. Probably best to ship nars in the image; is most robust and within the various "specs".

Pay attention to sharing limitations.

NixOS-based image support

Using NixOS inside a service is already possible, but not well integrated into Arion. A 'standard' interface seems helpful because it should make switching between init systems easier. For instance, it creates a standard way to add NixOS options to docker-compose services.

Service-level option for NixOS configuration to be written by user
- cast to list of modules
- merge function is list concatenation
Service-level option for evaluated NixOS configuration to be read by user or internal module
- defined by calling pkgs.nixos function with these modules
~~Internal option and module for nlewo's nix-container-images~~ moved to #24
- may or may not be able to use generalized call to NixOS
- may need to provide extra module to nix-container-images in order to support useHostStore properly
Internal option and module for services.systemd.*.runner-based containers

Try to generate a v2 docker compose file

UPDATE: docker-compose has a --compatibility flag that does most of the translation.

It turns out v2 is recommended for local development. This is annoying but also a fact of life. Example: docker/compose#4513 for memory limits. Arion can probably support either version with the flick of a switch.

add parallel raw options for v2
add composition level option for picking the docker compose file version
add command line switch to force a version to use
~~split the docker compose extension field into a separate file~~ v2.4 should support this
https://docs.docker.com/compose/compose-file/compose-file-v2/#scale vs https://docs.docker.com/compose/compose-file/#replicas
...

Weird Arion build error

I have the following build error:

arion up -d                                     1 ↵ 
these derivations will be built:
  /nix/store/9pdplsznmm30rys57g0d853qkx0dnrps-postgresql-and-plugins-11.6.drv
  /nix/store/pkgaklrx094j41i3bzxvc18cipqqg8c7-manage.drv
  /nix/store/bm3yqr79z6zq2cxagl7mv2q4hvy180v8-system-path.drv
  /nix/store/rjrr8aqyyy02lm7ip12rsgzq2apyiqz1-dbus-1.drv
  /nix/store/0ax8kglgmhl3l5sr6lahbmja0qzrldki-unit-dbus.service.drv
  /nix/store/0cl7lwf3fxi5kznj8b3myrwa5k4v52df-arion-base-granular-docker-layers.drv
  /nix/store/0k8ps060hccsymb07yl2hc0ipfp330i7-unit-migration.service.drv
  /nix/store/587gf69j4q6hd31vbkdxxd041khflkbp-unit-systemd-journald.service.drv
  /nix/store/5cb13kk3gd97pz1n8kvf4iqs9yllsprv-docker-image-arion-base.tar.gz.drv
  /nix/store/761k739by2mga61hcz0080kww0gyzqcg-unit-dbus.service.drv
  /nix/store/hlqif6a0khy8gylf86cdanzfaba7nyp8-stage-2-init.sh.drv
  /nix/store/pq9bg97gkgiq8hbpb04kx3g5b1dvzmfq-unit-postgresql.service.drv
  /nix/store/prgfqmzg2601gq9vsnvybalwnyry5lvq-unit-systemd-fsck-.service.drv
  /nix/store/q542d5aqnc6qkbbfzjj9pjm1dh36262s-unit-polkit.service.drv
  /nix/store/shkpqb78f6afphx47cja8ca89g0qvh41-unit-nscd.service.drv
  /nix/store/ysy3lzsjjfwfkbm77lms4zijcm5yq8na-unit-webServer.service.drv
  /nix/store/y4dcnr5ka2qg2ifyjwdrh2yi5mhkmz03-system-units.drv
  /nix/store/yq8cxw6vnpyzv1am0kly11gl1fwj6yv6-user-units.drv
  /nix/store/x2pxq6llfnp7mgkdzpjs3ajk18jvahf4-etc.drv
  /nix/store/xxlp4lqvv49gjyxnrphavr8yw7fazad7-nixos-system-unnamed-20.03pre209250.7184df6beb8.drv
  /nix/store/cb86hiq6cm2xcm9d29g0ihdjrqn6cm11-docker-compose.yaml.drv
building '/nix/store/hlqif6a0khy8gylf86cdanzfaba7nyp8-stage-2-init.sh.drv'...
building '/nix/store/pkgaklrx094j41i3bzxvc18cipqqg8c7-manage.drv'...
building '/nix/store/9pdplsznmm30rys57g0d853qkx0dnrps-postgresql-and-plugins-11.6.drv'...
building '/nix/store/shkpqb78f6afphx47cja8ca89g0qvh41-unit-nscd.service.drv'...
building '/nix/store/0cl7lwf3fxi5kznj8b3myrwa5k4v52df-arion-base-granular-docker-layers.drv'...
building '/nix/store/587gf69j4q6hd31vbkdxxd041khflkbp-unit-systemd-journald.service.drv'...
building '/nix/store/0k8ps060hccsymb07yl2hc0ipfp330i7-unit-migration.service.drv'...
/nix/store/kw1b29psgwibkqpbmdx3dcy3calzhjdf-store-path-to-layer.sh: line 5: $1: unbound variable
builder for '/nix/store/0cl7lwf3fxi5kznj8b3myrwa5k4v52df-arion-base-granular-docker-layers.drv' failed with exit code 123
cannot build derivation '/nix/store/5cb13kk3gd97pz1n8kvf4iqs9yllsprv-docker-image-arion-base.tar.gz.drv': 1 dependencies couldn't be built
building '/nix/store/ysy3lzsjjfwfkbm77lms4zijcm5yq8na-unit-webServer.service.drv'...
cannot build derivation '/nix/store/cb86hiq6cm2xcm9d29g0ihdjrqn6cm11-docker-compose.yaml.drv': 1 dependencies couldn't be built
error: build of '/nix/store/cb86hiq6cm2xcm9d29g0ihdjrqn6cm11-docker-compose.yaml.drv' failed
arion: FatalError {fatalErrorMessage = "nix-build failed with ExitFailure 100"}

With :

#arion-compose.nix
{
  services.webserver={pkgs,...}:{
    nixos.configuration=import ./configuration.nix;
    nixos.useSystemd=true;
    service.useHostStore = true;
    service.ports = [
      "80:80" # host:container
    ];

  };
}
#arion-pkgs.nix


import <nixpkgs> { 
  system = "x86_64-linux"; 
}

system: "x86_64-linux"
host os: Linux 5.4.8, NixOS, 20.03pre208413.e1eedf29e5d (Markhor)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.4pre20191022_9cac895
channels(root): "home-manager, nixos-20.03pre209250.7184df6beb8"
channels(paki): ""
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Improve documentation

Reuse bits from the NixOS manual options appendix and/or NixOS.org website. Host on GitHub pages.

for composition-level options
for service-level options.

Nix building packages unnecessarily when enabling service?

With the following config:

{
  services.webserver = { pkgs, ... }: {
    nixos.useSystemd = true;
    nixos.configuration.boot.tmpOnTmpfs = true;

    service.useHostStore = true;
    service.network_mode = "host";

    nixos.configuration.services.influxdb.enable = true;
  };
}

arion up seems to start building a bunch of packages including gtk+, while if i enable the the same influxdb service in the common Nixos configuration - the nixos-rebuild build completes significantly quicker. I've tested this on a fresh Nixos VM.

arion up
these derivations will be built:
  /nix/store/bz3bl6xra3bpwzbihr24hl34x4apv4qy-gtk+-2.24.32.drv
  /nix/store/rwi5khsigrx9p3m6gbl5naiz6xfm3wnr-gnome-vfs-2.24.4.drv
  /nix/store/d66jyz7n6jldhzz8wyw3m0qhraxvbznw-openjdk-8u222-ga.drv
  /nix/store/wh1na4rv2glvc7q3bkhh7c39avjamr51-libvirt-5.4.0.drv
  /nix/store/nk95n04j7iwls8gg952b4y1wsyxq49vy-collectd-5.8.1.drv
  /nix/store/x0mnll2r2wqcf9wndzw91dybnf8yy514-collectd-data-5.8.1.drv
  /nix/store/0y4n3yjxfn218qlhh0b3z58hlj810css-config.json.drv
  /nix/store/cqi9zi20cn8k0q0a5pv2j671kabc9v0i-config.toml.drv
  /nix/store/dwk87ii1f19a16xrg3nxh778iazy912s-unit-influxdb.service.drv
  /nix/store/az9z1jazzbwcbwi1lv4jjxmljaybadvp-system-units.drv
  /nix/store/knm6bfki1d7d97b7jqfcqxbkpmd09629-etc.drv
  /nix/store/g7sdjgb2qmg158cxllx7dhamvbpaj9i8-nixos-system-unnamed-19.09.2070.b9cb3b2fb2f.drv
  /nix/store/8lpri83gsja5rids0mmibvqajgmc7v9x-docker-compose.yaml.drv
these paths will be fetched (133.03 MiB download, 243.72 MiB unpacked):
  /nix/store/jwzhrb7nja8sjjllzw7fsamf9bcfdyzq-adoptopenjdk-hotspot-bin-8.0.232
  /nix/store/rlwril8s7s58dmcf5f85v3649i7vm294-jdk-jdk8u222-ga.tar.gz
copying path '/nix/store/rlwril8s7s58dmcf5f85v3649i7vm294-jdk-jdk8u222-ga.tar.gz' from 'https://cache.nixos.org'...
building '/nix/store/wh1na4rv2glvc7q3bkhh7c39avjamr51-libvirt-5.4.0.drv'...
copying path '/nix/store/jwzhrb7nja8sjjllzw7fsamf9bcfdyzq-adoptopenjdk-hotspot-bin-8.0.232' from 'https://cache.nixos.org'...
building '/nix/store/rwi5khsigrx9p3m6gbl5naiz6xfm3wnr-gnome-vfs-2.24.4.drv'...
building '/nix/store/bz3bl6xra3bpwzbihr24hl34x4apv4qy-gtk+-2.24.32.drv'...
unpacking sources
unpacking source archive /nix/store/sxrvwwrx2mqmhgvnki4rqb00m0plmfwl-gnome-vfs-2.24.4.tar.bz2
unpacking sources
unpacking source archive /nix/store/ki7v2i79b1fd1cga2065nn6x7l5km2d7-libvirt
unpacking sources
unpacking source archive /nix/store/7qbmp81hzfwlq8vcad7bs9kg4x809r4w-gtk+-2.24.32.tar.xz
source root is gnome-vfs-2.24.4
setting SOURCE_DATE_EPOCH to timestamp 1285666651 of file gnome-vfs-2.24.4/doc/html/volumes.html
patching sources
configuring
fixing libtool script ./ltmain.sh
configure flags: --disable-static --disable-dependency-tracking --prefix=/nix/store/5qbdaahi9vfi183dyfz6dkmqq2m7i1m5-gnome-vfs-2.24.4 --bindir=/nix/store/5qbdaahi9vfi183dyfz6dkmqq2m7i1m5-gnome-vfs-2.24.4/bin --sbindir=/nix/store/5qbdaahi9vfi183dyfz6dkmqq2m7i1m5-gnome-vfs-2.24.4/sbin --includedir=/nix/store/yjd0his1rdbxgrzk6mcvbsars4ars3zz-gnome-vfs-2.24.4-dev/include --oldincludedir=/nix/store/yjd0his1rdbxgrzk6mcvbsars4ars3zz-gnome-vfs-2.24.4-dev/include --mandir=/nix/store/5qbdaahi9vfi183dyfz6dkmqq2m7i1m5-gnome-vfs-2.24.4/share/man --infodir=/nix/store/5qbdaahi9vfi183dyfz6dkmqq2m7i1m5-gnome-vfs-2.24.4/share/info --docdir=/nix/store/5qbdaahi9vfi183dyfz6dkmqq2m7i1m5-gnome-vfs-2.24.4/share/doc/ --libdir=/nix/store/5qbdaahi9vfi183dyfz6dkmqq2m7i1m5-gnome-vfs-2.24.4/lib --libexecdir=/nix/store/5qbdaahi9vfi183dyfz6dkmqq2m7i1m5-gnome-vfs-2.24.4/libexec --localedir=/nix/store/5qbdaahi9vfi183dyfz6dkmqq2m7i1m5-gnome-vfs-2.24.4/share/locale
checking for a BSD-compatible install... /nix/store/9v78r3afqy9xn9zwdj9wfys6sk3vc01d-coreutils-8.31/bin/install -c
source root is libvirt
patching sources

Here is some output from the VM I was testing with:

[demo@nixos:~/test]$ arion up
these derivations will be built:
  /nix/store/032k1d58rz5vjs9isq633bx2xzi84527-unit-systemd-random-seed.service.drv
  /nix/store/bfz83fzlci3wa44ndqmcjxbkkqpb3flz-rrdtool-1.7.2.drv
  /nix/store/bz3bl6xra3bpwzbihr24hl34x4apv4qy-gtk+-2.24.32.drv
  /nix/store/zy4k7hbl1kdkp05b845sl96rb2qi4j5i-dbus-glib-0.110.drv
  /nix/store/nxi80q2xm19qawmgxxy8wrmf0pppjnnv-gconf-3.2.6.drv
  /nix/store/rwi5khsigrx9p3m6gbl5naiz6xfm3wnr-gnome-vfs-2.24.4.drv
  /nix/store/d66jyz7n6jldhzz8wyw3m0qhraxvbznw-openjdk-8u222-ga.drv
  /nix/store/gjjq1hzai9r1gjprvrx01wjkg79afydm-libnotify-0.7.8.drv
  /nix/store/ca82mlb962a1gnszg3r4nxz5wwlxs9ja-dnsmasq-2.80.drv
  /nix/store/rlsb51vanc8sv5sg8v113vdwlshqvj1p-pm-utils-1.4.1.drv
  /nix/store/wh1na4rv2glvc7q3bkhh7c39avjamr51-libvirt-5.4.0.drv
  /nix/store/nk95n04j7iwls8gg952b4y1wsyxq49vy-collectd-5.8.1.drv
  /nix/store/x0mnll2r2wqcf9wndzw91dybnf8yy514-collectd-data-5.8.1.drv
  /nix/store/0y4n3yjxfn218qlhh0b3z58hlj810css-config.json.drv
  /nix/store/b9w3720i1bs7fayhgj5gmawqzqnfly7n-minimal-contents.drv
  /nix/store/dyidprxqv2mk6dnyhncvrmbz5kmpcj0g-bulk-layers.drv
  /nix/store/sjl97md2blhiyzl9z2hiqs8akwqhrmrw-arion-base-config.json.drv
  /nix/store/0yzkq1iq4h4sp0ar7gd2xqzx54lajk15-closure.drv
  /nix/store/954yzzcpj70x3vslnz2fr451z0bk8qdl-pam-environment.drv
  /nix/store/100i0fb0462blf8nnvmhnzbc0lqn9164-systemd-user.pam.drv
  /nix/store/17c656ajjxd7kkdfa4ln1hba5lay6360-chsh.pam.drv
  /nix/store/182knvyycbangrvas26s6scjg63p8yz6-unit-systemd-vconsole-setup.service-disabled.drv
  /nix/store/1abbw4l5pa8rpkg1mwr5c1z99yah0p2y-etc-nixos.conf.drv
  /nix/store/23cr2y2zmbrhx7zcv10ipcv4m3cmkb4n-polkit-1.pam.drv
  /nix/store/2vngfwdiayl992nykcqk0bfg7z6bl4i0-cups.pam.drv
  /nix/store/6ax44rz1ack8p9j8naagdnqqkavxallb-client.conf.drv
  /nix/store/203ynhn563i8j513h6fvq9i4rdfsacgr-unit-network-local-commands.service.drv
  /nix/store/2pwh4w62ib0mgj1db104075j7h3bwzf7-unit-systemd-journal-flush.service.drv
  /nix/store/2zscgqfsk0k9bw532lf6rdfskcbqxijy-unit-rngd.service.drv
  /nix/store/33awhjnlzcaz6l2cvk9svvg64d5x2yrj-unit-systemd-udev-settle.service.drv
  /nix/store/vcw644hsp1443c1a6s5zinw6ybn76h1r-dbus-catalog.xml.drv
  /nix/store/w1hlj2y3bsx7wj2nn1vpkljd9dvahcqh-system-path.drv
  /nix/store/j73mpv69r6p3v28kikvcin4pqq19qv95-dbus-1.drv
  /nix/store/3r05mmm2ckb166vkg4ic46hhv03w6x2h-unit-dbus.service.drv
  /nix/store/q6plr77197gfhf6jw239b28w5f7mxxl4-etc-60-nixos.conf.drv
  /nix/store/4zpzgfiv68h3ss4vn342pz5nw7gly760-unit-systemd-sysctl.service.drv
  /nix/store/5n2ybd441xdrvmc32n165h723hf0w9q9-unit-systemd-fsck-.service.drv
  /nix/store/5vn0qyvnfph503qnns2hi0pk6pmkgqpg-unit-network-setup.service.drv
  /nix/store/5zbxhwky021ip7fshl62lqwi2dw3797f-unit-systemd-remount-fs.service.drv
  /nix/store/83y7wirlpisxfwlasbk3inyngs83f0d1-unit-nix-optimise.service.drv
  /nix/store/8b8ybc6aarxw0znkfn2glig07skm0wmb-etc-journald.conf.drv
  /nix/store/9x98jpkg9pv9sfqymcnd2smrg46422lc-unit-systemd-journald.service.drv
  /nix/store/aiy2cdlv4plwl1xq29c2ybvz4h4sghy4-unit-container-getty-.service.drv
  /nix/store/blsw6gqnv3wxsha1dwynds9g6031fia8-unit-nix-gc.service.drv
  /nix/store/c6j3zz00d936vhab7wss7g3fila3mz53-unit-prepare-kexec.service.drv
  /nix/store/c7dngd87qhy94k6qmzlqclw71f65fmdm-unit-user-runtime-dir-.service.drv
  /nix/store/397304103zh9jbss631xnzw4472jd09m-dhcpcd.conf.drv
  /nix/store/cmjbcpdv3xxwf91cvw3nbs4a5bgdhvbb-unit-dhcpcd.service.drv
  /nix/store/cqi9zi20cn8k0q0a5pv2j671kabc9v0i-config.toml.drv
  /nix/store/dwk87ii1f19a16xrg3nxh778iazy912s-unit-influxdb.service.drv
  /nix/store/fs8iqb1fjv078c929x3krh4s3ikpmjhx-unit-audit.service.drv
  /nix/store/gl866xv0bpwzycw5pmahr34ycy6xkhyk-unit-nix-daemon.socket-disabled.drv
  /nix/store/h01h23zajgln6afglqw8qg4a6n31v37i-unit-systemd-backlight-.service.drv
  /nix/store/hg5gkhw2x5xp66v2b9gykpqqgn09wfis-unit-resolvconf.service.drv
  /nix/store/ibz76l8w603sz9xsjwjcwcv3npzfjix5-unit-save-hwclock.service.drv
  /nix/store/a24cv32c6y61jz897i7m85piwpvdv8vz-etc-hosts.drv
  /nix/store/idysl23vsfnvvv8rnlh3fdwlvzya9q7q-unit-nscd.service.drv
  /nix/store/iyx9k38kc100lfzah5p3v72gdjpahilb-unit-systemd-timedated.service.drv
  /nix/store/ji0n8m881xsg3vpzzllyn516qmqv7vr2-unit-systemd-binfmt.service.drv
  /nix/store/l09hmsjsjv0x20s9lmxyniigpilvwmmk-unit-systemd-user-sessions.service.drv
  /nix/store/qv8bc2mjsjpkm1w0xgr4yqjrn0aywx1l-unit-systemd-logind.service-disabled.drv
  /nix/store/r7g1jwfhz1bhy7wr0sbhalbyj85f16pb-unit-getty-.service.drv
  /nix/store/riqjznwxpvd6d4sz09cxcf48yapmwgzz-unit-serial-getty-.service.drv
  /nix/store/s3xmxhdj96aval1j5ka9g0w54rqga1zl-unit-polkit.service.drv
  /nix/store/wmdqa5ahn0yqw30r6wadqbfcrb87kpqp-unit-nix-daemon.service-disabled.drv
  /nix/store/xzr5bn89yv0accv58lyn6mnagvwgdzb0-unit-user-.service.drv
  /nix/store/y7vixky6j8zy96ns4nlpqb4kbmlgmm3i-unit-systemd-update-utmp.service.drv
  /nix/store/zlyd4vbycxjvb2z6h4yp63wgi5wljwwq-unit-firewall.service.drv
  /nix/store/8a1hw9a75km9cz08rsswzdkabmfkqaam-system-units.drv
  /nix/store/9659bxj8xr2113qbwp1ivi12w8q8j2mh-runuser-l.pam.drv
  /nix/store/9jkkrrh0ri0h6i8h90vhk3fw4wc79rby-login.pam.drv
  /nix/store/a2wlzs88bppq8nj3pkhzpp337bsynxjv-userdel.pam.drv
  /nix/store/c4a6l0bl324sl1hbcyfk5yhbiik99wyy-i3lock-color.pam.drv
  /nix/store/zc61xfhdqcir0s1x2fz3rbccbmg7zm6s-set-environment.drv
  /nix/store/c53xc1yih63ah5vigfzanqir042g8acd-etc-profile.drv
  /nix/store/f7w6lkki0gy5mbhc5n809gazf5j9arma-chpasswd.pam.drv
  /nix/store/f96sl0wrq9h80vz6409pz9y75bmpiagj-groupdel.pam.drv
  /nix/store/f9ljlp5r1ws24czrli8mn9v5355s4vdp-issue.drv
  /nix/store/gw8sbih38syiyzf74fm5g5cl5kspkxqh-ftp.pam.drv
  /nix/store/hccjqrfpfyvc90vy9wdc7alw2p3fg5ng-sudo.pam.drv
  /nix/store/hqa9aby9j0l07g4jszvx77xh78p69nac-xscreensaver.pam.drv
  /nix/store/imvkmnb49h44k8fvrlm6y8zicysymvf6-su.pam.drv
  /nix/store/kb88irisqkp3qix0y8n1h2i7hap16rwl-usermod.pam.drv
  /nix/store/kczd4yfcbk7q5n3w1xshapgscl3pj452-passwd.pam.drv
  /nix/store/lzljcls8a6smx96hzlawxwb625rxxzrr-i3lock.pam.drv
  /nix/store/m83da80fc9qnv0mmlplm2pgfbvxr0dmx-screen.pam.drv
  /nix/store/qhmhycq4c27s85zlb7nvp4nhsrrmqyf3-etc-ssh_config.drv
  /nix/store/qz60y7c5dzx5s62zn2mshvfjp07k3yf5-xlock.pam.drv
  /nix/store/ri6b967m7s3fpswjj3qmzbg7nwhz3lby-etc-fstab.drv
  /nix/store/sjhila79v3h3ahwxm5hmgk59p9hgryg8-useradd.pam.drv
  /nix/store/slfm3px0zcis4dzx1af2xy3242pwz6am-groupmems.pam.drv
  /nix/store/snb5vsc9807d2cmmlmy6afi38g7lhpym-groupadd.pam.drv
  /nix/store/syicqs7km3bhhgkgk7cfynfk40czdsm3-vlock.pam.drv
  /nix/store/xkg1vi9n9rwkf0rbvs7vmvdm7j680v90-groupmod.pam.drv
  /nix/store/ybgf80jc46gk5376bgpijb79dp3kq2g8-chfn.pam.drv
  /nix/store/cgnq9jixa55s3s2q1fv4sgsyb9c0j576-unit-script-nixos-activation-start.drv
  /nix/store/a48637bck4zz35jczph2j8knkcypi3bp-unit-nixos-activation.service.drv
  /nix/store/h20aq1x528a4ypc20y80dwsb4vhf8czn-unit-dbus.service.drv
  /nix/store/yw23ns7b9f082k8b9jbxzkby1z6gyd0v-user-units.drv
  /nix/store/1lhi903axswr9kliarap4riyhampdzz8-etc.drv
  /nix/store/hgs6k2k5ia628nz2znkjcrnhy022cbwz-local-cmds.drv
  /nix/store/v2lcx817icccv4rbsvfbcvh31m6hgayy-mounts.sh.drv
  /nix/store/gyr7phzq2vf6d9183mvjnc0m9d5h32ny-stage-2-init.sh.drv
  /nix/store/vlg2invm6nn3zycfyi0bn1x70pb1ppy2-users-groups.json.drv
  /nix/store/9yyfwk67dnwvwc6vwmsijmh56kmfzapp-nixos-system-unnamed-19.09.2079.8731aaaf8b3.drv
  /nix/store/vg9n90h42zny6b30bl2fq0k0k4bsw521-closure-paths.drv
  /nix/store/zlbpl3x8s1siq093g34li4f0cxrq8r8n-store-path-to-layer.sh.drv
  /nix/store/lkkla11c4a33y1994gr7vcjagal51s3s-arion-base-granular-docker-layers.drv
  /nix/store/xfasjpyi6fr97jsglf94w3hbgqrdr3qr-arion-base-customisation-layer.drv
  /nix/store/h0ghnsl7l5ng4k693javba05dbjwqk3y-docker-image-arion-base.tar.gz.drv
  /nix/store/l9yks5f5k7fjcw0hmiqfk2alcfqrmkyx-container-system-env.drv
  /nix/store/paqym44g3h85lbb9fpkwxnyz5yqksyhk-docker-compose.yaml.drv
these paths will be fetched (359.23 MiB download, 1075.65 MiB unpacked):
  /nix/store/002aip2whyhngiw7wgaj21afdvnx29rx-libssh2-1.9.0-dev
  /nix/store/043jjnqkrm7y0zg28d3iy1jbqhwmq23f-meson-0.51.2
  /nix/store/06nq4z17fh43wrbn6hl1yq7bzs99lpr1-hook
  /nix/store/0a3imljkw5755ych0fh9gz5g7zs8vfrf-nghttp2-1.39.2-dev
  /nix/store/0b4m9hbykbx0h0k1nlr1zyl7ffza93mw-55eb69247fe2b479ea43311503042fc03bf4e67d.patch
  /nix/store/1jkdp9mvbk4laacp4ygh5j0rbnxnapxz-yajl-2.1.0
  /nix/store/1mj4gx6l22rrafb5n3g043m9x3srqk2q-dbus-1.12.16
  /nix/store/20jdniyabdz4f4mjbnql3bhgrj5l9wi5-acl-2.2.53-dev
  /nix/store/27c13j9glnqhb55s6xf58majhfy84q05-pigz-2.4
  /nix/store/291ldi6fqsbmkbvbs8is4mcg3jb64ld4-gcc-wrapper-8.3.0
  /nix/store/2mjp92nz4pyc65hv59aicbixa0hxqw74-varnish-5.2.1-dev
  /nix/store/2ypz5fdds2hzfqjda44ppjy7rcflmmh8-cyrus-sasl-2.1.27-dev
  /nix/store/384h3vls6zcnzzbwsdqd9gm0v9kb8yv0-iptables-1.8.3-dev
  /nix/store/3n1lyr4kv0ncav5kq475ikpvi28r0lh2-5.4.0-CVE-2019-10168.patch
  /nix/store/3nqjl8rqwnk22s4x7x1q986c5957isnc-cyrus-sasl-2.1.27-bin
  /nix/store/3z5np6z8dhgg3iq543x4jvzz01gq2xyb-pm-utils-1.4.1.tar.gz
  /nix/store/40gqaxg3js8zsxpr4j8ph4cjy13ngnk9-ORBit2-2.14.19
  /nix/store/4h0pmb16ln7wxy15yydi945p1rgkvjnl-atk-2.32.0-dev
  /nix/store/4mm0k3anmic3rsxdmkm9d9rvj3rmj468-net-snmp-5.8
  /nix/store/4qblslbyyic33pb85k52z6ffv0jp2il8-libdbi-0.9.0
  /nix/store/4y8asb5qabfdk6gc7js85j30lx77l7vh-bridge-utils-1.5
  /nix/store/53b0624c9s266iy2czzjcgsasv5mqdyp-corba-jdk8u222-ga.tar.gz
  /nix/store/5ch3bm2skx23wkwcz3rvkg61h1dc03nz-libuv-1.34.2
  /nix/store/5g3ih6zqawglxb8baak99f84wa64irw2-tarsum
  /nix/store/5xb06lyhrhjr9f1mbqqvvh270j71h88b-pcsclite-1.8.25-bin
  /nix/store/605125bvpxvp3pxj7ymvm81m4y1l8h2k-perl5.30.0-Net-DBus-1.1.0
  /nix/store/605l3mspl9bvjyg4lng6rjminskfhbzn-intltool-0.51.0
  /nix/store/61zfj9fyd2w555gx36mj3rgq1na6ss5k-go-1.12.16
  /nix/store/62mssjh0scfh2dn5a6npq21rw5br5q0s-numad-0.5
  /nix/store/62nx464pw43wx3fvg2dnfsaijl7nvvzq-jshon-20160111.2
  /nix/store/62x7m20m7lm8y8s17cbgha0sf3cmma19-gcc-8.3.0
  /nix/store/6jq6j9ym4x8kmzd09fwy4cn4lbhfrci3-util-linux-2.33.2-dev
  /nix/store/6s7gdq6aic1xpxcddc9lqpmih8j2k59w-pcsclite-1.8.25
  /nix/store/6sdarmqn2ylsxxaj427aq3mrb6d1crg0-5.4.0-CVE-2019-10167.patch
  /nix/store/74gvl6qr7zwf0na3gkw3c4napqkvka4q-libmodbus-3.1.6
  /nix/store/7a5c81gs807zabmfckzcd839b7hlpmzx-python3.7-python-dateutil-2.8.0
  /nix/store/7h86d7lflqhw89pqyvrar5mjjvkiam5c-python3.7-pytoml-0.1.20
  /nix/store/7qbmp81hzfwlq8vcad7bs9kg4x809r4w-gtk+-2.24.32.tar.xz
  /nix/store/8ds90qcjws2wn9v1jkhj99qwf14c2fbi-5.4.0-CVE-2019-10161.patch
  /nix/store/8pg237xh6ywrj7626f5z10cy8p27nh0m-perl5.30.0-XML-XPath-1.44
  /nix/store/931145dajawdwk35pd7a9fdkbcj7a1il-jaxp-jdk8u222-ga.tar.gz
  /nix/store/93iqhdwfg0cmwjbbig7bdv51labwxgs5-python3.7-PyYAML-5.1.1
  /nix/store/9caxfwks0fqzv09hk1h7zbw87sk5kmhk-automake-1.16.1
  /nix/store/9gpc8fd4ggrbakp5hprdn65m7wgb79qj-hotspot-jdk8u222-ga.tar.gz
  /nix/store/9nvwvhxmsykki97y8d9kxj2nn8gxwimv-c-ares-1.15.0
  /nix/store/9sci9f14y0y8g0r8ykfrs6xb04mi5dfh-libmicrohttpd-0.9.66-dev
  /nix/store/a82n1g48k89xyb8zcaym5szfv81sl18p-libbson-1.9.5
  /nix/store/ajrrkivdfvp8dp4vdg5hp1h5hblmanc9-binutils-2.31.1
  /nix/store/apfgni3w7sd7qnnzws0ky8j40sbigy4m-hook
  /nix/store/aq6q8gshzq89grw9srfndc19qnrbnjyw-libserialport-0.1.1
  /nix/store/aqsq6irl7dqwkgmlx8lyy6fl2jdvsq2q-zfs-user-0.8.2-dev
  /nix/store/axks09hzrzilf4zfs88riijm98afrijq-gnutls-3.6.11.1-dev
  /nix/store/b69lrb7466f7zllq3akvznh4mj7x16a7-libnotify-0.7.8.tar.xz
  /nix/store/bgxj218n6xq14ixrhlpqmk83dj5qqqyq-gdk-pixbuf-2.38.1-dev
  /nix/store/bmaq6jxpr8jj43r8ra0ahw8cs2n7fgi9-mosquitto-1.6.6
  /nix/store/bmw1dalvvgagcifyi459cxbzj5v57qb7-liboping-1.10.0
  /nix/store/bpg03jgqvxg7vfwhniiq98ncg78wpg8i-rabbitmq-c-0.9.0
  /nix/store/c1fnd3my4vyvip6mc203d0ml3bmi9a09-collectd-5.8.1.tar.bz2
  /nix/store/c2mdvakq1bwgnv916qs7crlzhq4jzaqm-autogen-5.18.12
  /nix/store/c8bsbm5xalrfwnmlgiplbwpcwg2mr6l2-zip-3.0
  /nix/store/ccqha51d7kcj725n4v946b90rpx6cwys-polkit-0.116-bin
  /nix/store/cvj75k5dk1pbxrgmyvs8gqxs51fw1g96-nghttp2-1.39.2-bin
  /nix/store/dbwly43yz5z5ia3f3m4v645vcynxhnmv-gnome-mime-data-2.18.0
  /nix/store/dh7xm5xaim7jrx3aaz38rys3yjhnvc8y-hiredis-0.14.0
  /nix/store/f3icmpi6q1hfydkkgnhb1gasyv1k8zpd-cross-file.conf
  /nix/store/f8x305m0yzfkbh31f6znfmc4x58888wf-libidn-1.35-dev
  /nix/store/fcviz6barhsqaj3kgd6vzpa4hmybsg1k-libtool-2.4.6
  /nix/store/fddcirvs2gcjhc5cmri99p417whqya5x-mongoc-1.8.0
  /nix/store/fl2vmp6v7wvp6snk1qilv0afq24fylz6-openssl-1.0.2u-bin
  /nix/store/fmpb41wkfmjpq89g339js01hn4zn9y5z-langtools-jdk8u222-ga.tar.gz
  /nix/store/fnf2nw5wcdlj4hhy7ypf0n9x9b3cf55p-expand-response-params
  /nix/store/fpyp0vyxds367cd9ajlw5hw142cvj7x7-libwebsockets-3.1.0
  /nix/store/fwaw1ljx26h07bc7hjjcdzbzsxmi6rmm-mariadb-connector-c-2.3.7-dev
  /nix/store/fz1184f1adin5fzs8l0qffklsc2b2c65-snappy-1.1.7-dev
  /nix/store/gindnb8698nxsgpnrzjdihwljw86r1fh-protobuf-c-1.3.1
  /nix/store/gwwycf3w6cbj0gd2mpgblrdjc24f3cys-binutils-wrapper-2.31.1
  /nix/store/gyy7cvnk5sxip4qvmjmrkk8asx9pksgz-python3.7-setuptools_scm-3.3.3
  /nix/store/h2xf9dlx46gz8iz83zpzjc2p4phck4nv-rrdtool-1.7.2.tar.gz
  /nix/store/hcpppvs5ar55x06byswqr8gdyllm5f01-perl5.30.0-Tk-804.034
  /nix/store/hkrwcwmd6p5iy0i263qh2bgsxj8zpfs9-zfs-user-0.8.2
  /nix/store/hrn5ky54k22h97r0axgxk335xsi0ahcr-avahi-0.7
  /nix/store/hvwp81lczh2mc7mgs7y78g3zvf9fapqd-libmemcached-1.0.18
  /nix/store/hwphyy0bkfgy640ckfgrj6pqn1p5b9ya-gobject-introspection-1.60.2
  /nix/store/i5zfpg6qhnhhn3zqarkpfgfprsiv7bky-libcredis-0.2.3
  /nix/store/idl3x7nyykjd66bqjgsmp1s4ikpxi69d-hook
  /nix/store/ifjz1sara92pcgb4adnq5rv1j59y5bmw-gobject-introspection-1.60.2-dev
  /nix/store/ik3mp2fxw3hh78hf0dkrdvr8lcndfmdj-5.4.0-CVE-2019-10166.patch
  /nix/store/ilv2kp5b7sxbg6zi5cdm7bj2x8pvicda-libgpg-error-1.36-dev
  /nix/store/irlhwsv7karalf84ia9kg55szah3jpa3-pango-1.43.0-dev
  /nix/store/ivd3dxagc5djhxf6p360cpmlpcbi7zw5-opensc-0.20.0
  /nix/store/izsji9y2x6xmmgvn33jsn3sgydp2inr4-libnl-3.4.0-dev
  /nix/store/izxy3bjlrck4b0pzy047qqjzabqxs4aa-libsigrok-0.3.0
  /nix/store/j08fcd1qhxbyphjgr9qcafgjx5pi0y1s-postgresql-9.6.15
  /nix/store/j763yzjwkq0xj4mma388h2h4f24dy4f8-dmidecode-3.2
  /nix/store/jvvxfaziglw1hna8kirf99xjdllaz6a3-parted-3.2-dev
  /nix/store/jwhd7yi26sfbi5djclxrr3ims6r5796i-ncurses-6.1-20190112-dev
  /nix/store/jwzhrb7nja8sjjllzw7fsamf9bcfdyzq-adoptopenjdk-hotspot-bin-8.0.232
  /nix/store/k04fdbirxd6sc0ixyqxbfg0dv6xl7z31-jq-1.6-bin
  /nix/store/ki7v2i79b1fd1cga2065nn6x7l5km2d7-libvirt
  /nix/store/kjzjyf90v7h6fqvsqrnw4z9is979n4lq-libtasn1-4.14-dev
  /nix/store/kkap7p1mqymf9qd44ydbjlqw5by4s7dz-autoconf-2.69
  /nix/store/kxwpajrvby75fvhw6c327mrrx890nv0b-nghttp2-1.39.2
  /nix/store/kz4i8i8nvimsa9cnn9b330p15csbxdww-alsa-lib-1.1.9-dev
  /nix/store/l4ph62j6kvs1bxsli4p5l5wvxzf8x14b-pango-1.43.0
  /nix/store/l51illz5x0ndqpnrs4py265h1yy84ivb-system-shutdown
  /nix/store/l8xpvx596a1zpdgl1zl29dyl849hdsd6-remarshal-0.10.0
  /nix/store/lc3hrwmsmkgdx3hqak7xv73j5sfc3svz-hook
  /nix/store/ld5aizbjvayk8wv2v4yxrqcq7gvia2mk-ORBit2-2.14.19-dev
  /nix/store/lmygr546wh5k1i27pd6f62p82l2qhls2-ebtables-2.0.10-4
  /nix/store/lrngbfgxvl76400p66mgagfwnw0vvzbx-dbus-1.12.16-lib
  /nix/store/md8frx76isqb3m6n3khnd2y1kbrlpksw-libidn-1.35-bin
  /nix/store/mdxxx9wc2qgk3mmcz1vshzbalyfyzx12-openssl-1.0.2u-dev
  /nix/store/mg102yb4a2cgisvkv2svi0vlbmxc76pw-libnl-3.4.0
  /nix/store/mpasxq5jwgw4qz1zazkyrw6hplgcdyy5-d5a3c020d33cc33ee8049f54c7b4dffcd123bf83.patch
  /nix/store/mpijc3nw7518c65vx5x3q6vgq19nyyql-attr-2.4.48-dev
  /nix/store/mspz17wygvxgrabrhbwqmq4bfymb7jl5-libnl-3.4.0-bin
  /nix/store/n2jvzimy2mm0rygi76jg0wsjg5d18z8p-libgcrypt-1.8.5-dev
  /nix/store/n8cbchp4bpyx3c5rj56l76rmvqwyhh96-libXt-1.1.5-dev
  /nix/store/ngyixxc9jlgk9a47bnap1rxhgh5v66y9-python3.7-six-1.12.0
  /nix/store/nryr486cqdvx9ibfa652b1xf8q577i6x-dbus-glib-0.110.tar.gz
  /nix/store/nwing04g1w0gx10xw6haiybc34nl7d6h-riemann-c-client-1.10.4
  /nix/store/p4mgm0wjq9bk4db3zb34nmv5bx2nnass-jaxws-jdk8u222-ga.tar.gz
  /nix/store/psyznznbs7qrmm1v8yrh90740m064b6n-varnish-5.2.1
  /nix/store/pv3733cr48pr5vbwjrjc7hhv222h4d4f-xhtml1-20020801
  /nix/store/pxj0y091xfdbi2dzgrhybr7jyy0nqw30-curl-7.65.3-dev
  /nix/store/q0nryj0cifax98fajy9r4jfgshcq04yh-libkrb5-1.17-dev
  /nix/store/q7m48qymjppgkyg53rdc7d3flw2hps4d-cups-2.2.12
  /nix/store/qaf3kpzkaj2xjn90m9fn7gjdn9m0dsbx-dnsmasq-2.80.tar.xz
  /nix/store/qghrkvk86f9llfkcr1bxsypqbw1a4qmw-stdenv-linux
  /nix/store/qlzqvd5zipcg2jr66pbjdjzvlpaczihy-nashorn-jdk8u222-ga.tar.gz
  /nix/store/qq1a8sn4k9f2qw62kc5d0slm3brrnr01-rng-tools-6.7
  /nix/store/r1mglgf6j6gmr1xv77jni942kk04i50d-cups-2.2.12-dev
  /nix/store/rf8vbdgdsbv05vnn0cbwcvpqck45b3jk-GConf-3.2.6.tar.xz
  /nix/store/rh9s9g1djzrkl2irihycm5pn84r6817s-glibc-locales-2.27
  /nix/store/rk4j1ppcsdh2d70g2s790vvsfg6jgw4h-polkit-0.116
  /nix/store/rlwril8s7s58dmcf5f85v3649i7vm294-jdk-jdk8u222-ga.tar.gz
  /nix/store/rm667fkz9cis2aqrj7yn8ivbj8lrhlb9-onig-6.9.4
  /nix/store/rskgp079mwp80m5yh6560mzh8vg0515y-cups-2.2.12-lib
  /nix/store/rvmsbk9f8nfpf1abmbnqb4ag8knhbpx0-atk-2.32.0
  /nix/store/s6xbwny8zv988jhl5pgz258fcza4habf-system-generators
  /nix/store/sxrvwwrx2mqmhgvnki4rqb00m0plmfwl-gnome-vfs-2.24.4.tar.bz2
  /nix/store/v0h6aqq5blf9f9kq2w18nny7aa1baglz-pango-1.43.0-bin
  /nix/store/vg1dwlfimgxd4dwys326qrzxq1dx2ywg-dhcpcd-8.0.6
  /nix/store/vsxgp3298dp8aax83kkynhsh3aszajp5-perl5.30.0-Path-Tiny-0.108
  /nix/store/vv4c9wiamwn0pnn36m7s0w6xy1c66q1b-autogen-5.18.12-lib
  /nix/store/vzwbs2039g4z7npmmvcc6wsq62pziv7v-check-0.12.0
  /nix/store/w0i9sbiq4m5akgcaf5b27jn1hq5i1yvz-gnutls-3.6.11.1-bin
  /nix/store/w37y08brlrxv4151sn726l6bqyn2kvk3-ninja-1.9.0
  /nix/store/w3zk97m66b45grjabblijbfdhl4s82pc-nettle-3.4.1-dev
  /nix/store/wbgs2graxg86cp4d6n789lh9bybvw0a5-jq-1.6-dev
  /nix/store/wj76rqlld4lr5zkdf86zbx2ap0dq6k16-libIDL-0.8.14
  /nix/store/wnr98qjhi8c0zasvrj37qs1jyangjjbz-jq-1.6-lib
  /nix/store/x0zlask03j72krzr0rbm32zy7cfqfz1r-gdk-pixbuf-2.38.1
  /nix/store/xbk8842znysc5dz4fdrsfhy9jxwrzl6m-openssl-1.0.2u
  /nix/store/xkaayw3snjqj67nj94spy21ags1fx6wr-rdkafka-1.0.1
  /nix/store/xpf3f6fyxmc70fpvqhkl716bbh2bfvmp-jdk8-jdk8u222-ga.tar.gz
  /nix/store/y730f98n42j63rj71hpknmg5wy3zvq70-polkit-0.116-dev
  /nix/store/ydbnsf8zbyma75f2ig1qfpgki47y4wjz-dbus-1.12.16-dev
  /nix/store/yrymiv17p32gbkxd3qh94xifc675xqky-nss-cacert-3.46.1
  /nix/store/zg6ipqwambyipywk8qdpzc9ngj3rnn86-unzip-6.0
  /nix/store/znsb638hg69m8lrn6vcqm7lpmrg3pskd-libev-4.27
  /nix/store/zrdpy1pw64qwqjjp2lm14bjyiir4pfp4-readline-6.3p08-dev

Support service logging field

Can be set to journald. I wonder what are the benefits of doing so and how will it interact with nixos?

First class images?

The way docker compose builds images, as part of services, is actually one of its pain points.

Changes:

provide a composition level attrsOf option to build images independently of services
make the service images use those through a naming convention
expose the images in a convenient way to the Nix language and document how to use it (something like (pkgs.arion.eval foo).config.images.<name>.tarball)

This also has the potential to make Arion usable inside a Dockerfile, to provide reuse of the Arion modules.

Note however, that image modules will have no way to set service options and, conversely, services that use separately defined images will not be able to change nixos options in the image.
The latter seems resolvable with a fancy inheritance scheme, but such a scheme is unlikely to be any better than the dockerTools layered images' transparent reuse scheme.
The good news is we can probably detect a services that tries to do both, by counting the number of defined nixos modules in the service.

Docker tries to create store path for volume mount

Store paths that don't exist can't be mounted into a container.
Docker or docker compose try to create missing directories. This is not a sensible behavior for volume mounts that mount nix store paths into a container.

On a single-user it corrupts the nix store.
On a multi-user install, you get this error:

$ docker start mycontainer
Error response from daemon: error while creating mount source path '/nix/store/...-container-system-env': mkdir /nix/store/...-container-system-env: read-only file system

This error may occur after garbage collection. arion up -d restores a valid store+container combination.
#39 prevents this error where GC roots can be applied.

Integration test Nix error when test VM has Nix sandboxing enabled

Discovered in #17
Error message
This seems to be an upstream Nix bug triggered by the test VM's filesystem mount structure - Nix wants to hard link but the source / target are on different filesystems.

base image should include /etc/protocols



Here is the error message I was able to extract:

    HttpExceptionRequest Request { host = "package.elm-lang.org" port = 443
    secure = True requestHeaders =
    [("User-Agent","elm/0.19.0"),("Accept-Encoding","gzip")] path =
    "/all-packages" queryString = "" method = "GET" proxy = Nothing rawBody =
    False redirectCount = 10 responseTimeout = ResponseTimeoutDefault
    requestVersion = HTTP/1.1 } (ConnectionFailure
    Network.BSD.getProtocolByName: does not exist (no such protocol name: tcp))

Currently work-arounding this with "${pkgs.iana-etc}/etc/protocols:/etc/protocols:ro"

Support image.contents (files outside store) with useHostStore

useHostStore currently sets the image to something with /bin/sh and /usr/bin/env symlinks, thus ignoring image.contents.

This can be implemented with a wrapper script (like entrypoint), but that may not run as root.
Otherwise, we should probably generate a small layer anyway with a symlink farm or the single derivation.

Restarting service doesn't run the latest service

Reproduce

Given arion container as:

    web = {
      service.useHostStore = true;
      service.depends_on = [ "hercules-server" ];
      service.ports = [
        "8088:80"
      ];
      nixos.useSystemd = true;
      nixos.configuration = {
        imports = [ ../../web/module.nix ];
        services.hercules-web = {
          domain = envConfig.domain;
          backend = apiInternalURL;
        };
      };
    };

When there's a change in the closure of the nixos system,
arion restart web will rebuild all the changes, but they won't be in effect which is confusing.

Workaround

arion up -d --always-recreate-deps web will recreate the image.

NixOS and /run/wrappers

/run/wrappers does not have the same attributs (like suid, exec) as under plain NixOS.
I suppose it's for security purposes, but it breaks the expected behaviours.
A services.tmpfs.usafe option which provides attributs as under NixOS will be great ?

Module for running specific NixOS service in isolation

We have an example which uses the runner. There's probably more we can do to support this use case. It might look a new module to support something like nixos.useService = services: services.nginx.

Include pre-defined services?

It seems to be that for local development one would very commonly need the same things.

nginx
ngrok (or another proxying server like https://www.netlify.com/products/dev/)
backend/frontend (maybe there's not much point abstracting this)
others?

So I'm proposing some of those modules can be already provided by arion.

arion watch

Once #34 is addressed, for really smooth development we could use target/lorri#64 (or low-level bits) to implement smooth development cycle :)

Sane defaults for development

Ideally, we'd turn on useHostStore, although that poses a security risk. It's debateable if that really should be the default, but since the whole project is aimed at sole development, I think we should do it.

There could be a hardened profile if you didn't trust your service source to turn this off.

Most of the time I imagine people will just use their own service set that can be audited by reading code, but those should be turned off when using third party images.

Configurable project name via composition option

Use -p or COMPOSE_PROJECT_NAME https://docs.docker.com/compose/reference/overview/

We might call it "composition name" instead because that's how we've been calling these things. Also it's less overloaded than project.

Document how we deal with https://docs.docker.com/compose/env-file/

support dockerTools images as input

The goal is to help users who are defining their images via dockerTools.

useHostStore = true -> avoid docker load
useHostStore = false -> do use docker load, for more production-like testing

Unanswered question: when using the host store, do we need to build a custom base image to get the attributes right, or can all the relevant attributes be set at the docker compose service level?

Darwin / macOS support

fix unit tests, add to CI #83
add linux builder setup instructions to README
user testing / instructions how to use correctly when system /= builtins.currentSystem
- pass system = "x86_64-linux" into nixpkgs in arion-pkgs.nix + anything else?

Arion.Nix evaluateComposition "matches an example" test failure (on unsupported Mac OS)

Hello, I'm currently experimenting with Arion on Mac OS X (10.14.5). Although not listed as a supported OS in the doc, I'm giving it a shot.

Using nix, I'm currently getting the following error when building. I'm unsure what could go wrong in this Haskell test on this platform, but maybe this issue can help troubleshoot something else.

assertion failed at /nix/store/f7rbfswfrx81y3v86i7gykj0js4pxrnh-vyqishq17q64pxfc6r627d42xddilh9z-source/pkgs/os-specific/linux/kernel/generic.nix:56:1

Arion.Nix
  evaluateComposition
    matches an example FAILED [1]

Failures:

  src/haskell/test/Arion/NixSpec.hs:16:41: 
  1) Arion.Nix.evaluateComposition matches an example
       uncaught exception: ExitCode
       ExitFailure 1

  To rerun use: --match "/Arion.Nix/evaluateComposition/matches an example/"

Randomized with seed 190610212

Finished in 0.4093 seconds
1 example, 1 failure

Test suite arion-unit-tests: FAIL
Test suite logged to: dist/test/arion-compose-0.1.0.0-arion-unit-tests.log
0 of 1 test suites (0 of 1 test cases) passed.
builder for '/nix/store/c7iia08jiab758w51fg3s7sl401achq6-arion-0.1.0.0.drv' failed with exit code 1
error: build of '/nix/store/c7iia08jiab758w51fg3s7sl401achq6-arion-0.1.0.0.drv' failed

Relevant part of nix derivation:

pkgs.fetchFromGitHub {
    owner = "hercules-ci";
    repo = "arion";
    rev = "v0.1.0.0";
    sha256 = "021biqdlmbh746ji3m35bhwag68p3qlnxrwf8iaw3sz552j9dcfj";
  }

DRY docs using build step?

Couldn't find a method to inline with asciidoctor alone, but the following produces an asciidoc

asciidoctor -b docbook5 -o - index.adoc | pandoc --from docbook --to asciidoc

is the quality ok?
command to build and overwrite README
CI for the README file
pre-commit hook

Integrated deployment via ssh

We've been deploying over ssh which is nice and simple, but not as easy as it should. More importantly, knowing how deployment is done will help with other features like secrets.

Options:

local docker (default) which allows bind mounts, useHostStore etc
remote docker, which doesn't allow bind mounts, requires ssh session management
in the future: swarm (out of scope here)

arion.build should build a directory

Currently builds only a yaml file. A directory provides opportunities to add more files to the output.

nscd doesn't start with NixOS

[155882.250676] systemd[225]: nscd.service: Failed to set up mount namespacing: Operation not supported
[155882.250868] systemd[225]: nscd.service: Failed at step NAMESPACE spawning /nix/store/hljwhpqjhpy5327zfmzjjh8xyly3hiy2-glibc-2.27-bin/sbin/nscd: Operation not supported

dockerTools target + skopeo

A better push command.
May need to introduce a composition-wide option that effectively overrides useHostStore.
Documentation
Update README

NixOps keys support

Quite feasible to support the NixOps interface for keys outside the store. Related: #52 #51

S6 init system support

Split out of #8

Argument parsing for common commands

arion needs to move to a command line scheme where all top-level commands are parsed by Arion itself. The escape hatch docker-compose subcommand can stay, but then at least the user knows they are digging down and inconsistencies may occur in features that Arion needs to be aware of.

This may demand a rewrite of the bash parts into something that's easier to write reliable logic in.

hercules-ci / arion Goto Github PK

arion's Introduction

arion's People

Contributors

Stargazers

Watchers

Forkers

arion's Issues

Reproduce

Workaround

Recommend Projects

Recommend Topics

Recommend Org