emk / compose_yml Goto Github PK

This is about to be released, and we need to support it. There's a bunch of (minor) new features here, some of which will be helpful.

New features, according to the release notes:

• Added support for setting volume labels and network labels in docker-compose.yml.
• NOPE (undocumented). Added support for the isolation parameter in service definitions.
• NOPE (obscure). Added support for link-local IPs in the service networks definitions.
• Added support for shell-style inline defaults in variable interpolation. The supported forms are ${FOO-default} (fall back if FOO is unset) and ${FOO:-default} (fall back if FOO is unset or empty).
• Added support for the group_add and oom_score_adj parameters in
  service definitions.
• Added support for the internal and enable_ipv6 parameters in network definitions.
• Overriding a logging configuration will now properly merge the options mappings if the driver values do not conflict.

This is permitted by Docker and it would be nice if we could handle it, too.

Docker for Windows allows to specify special paths in volumes sub-section of service section. These paths must NOT be converted to Windows equivalent. Examples:

services:
    test:
        volumes:
            - /dev/shm:/dev/shm
            - /var/run/docker.sock:/var/run/docker.sock
            - "/sys:/sys:ro"
            - "/var/run:/var/run:rw"

Including environment:, etc.

Much to my surprise, it looks like compose_yml is very close to working on Windows! But to fully support Windows, we'd need to make a number of changes.

Inside the container: Represent container paths with a new type

The following fields assume that the path format inside a container is the same as the path format on the host OS:

struct Service {
    pub working_dir: Option<RawOr<PathBuf>>,

pub struct VolumeMount {
    /// Where should we mount this volume in the container?  This must be
    /// an absolute path.
    pub container: PathBuf,

We could just make a trivial wrapper type named ContainerPathBuf around String and use that to replace all these instances of PathBuf, since we treat these fields as almost entirely opaque and we don't seem to need any Path operations on them. Or we could be lazy and just replace these with String.

Outside the container: Figure our how Windows paths work with docker-compose.yml

The first three fields are relatively easy, and we probably don't need to change anything:

enum Context {
    /// A regular local directory.
    Dir(PathBuf),

struct Service {
    pub env_files: Vec<RawOr<PathBuf>>,

struct Extends {
    pub file: Option<RawOr<PathBuf>>,

But the final field is going to require careful investigation:

enum HostVolume {
    /// This volume corresponds to a path on the host.  It may be a
    /// relative or absolute path.
    Path(PathBuf),
    /// A path relative to the current user's home directory on the host.
    /// Must be a relative path.
    UserRelativePath(PathBuf),

On Unix, this field can represent several different kinds of host paths:

version: "2"
services:
  voltest:
    image: "faraday/rails_hello"
    command: ["bash"]
    volumes:
      - "/absolute/path:/mnt/a"
      - "/absolute/path:/mnt/b:ro"
      - "./relative/path:/mnt/c"
      - "~/user/relative/path:/mnt/d"

If I save this as vol-test.yml and I run this with:

docker-compose -f vol-test.yml run voltest`

Then I can run the following inside the container:

root@2cd832d24351:/usr/src/app# ls /mnt/
a  b  c  d

I'd like to translate the above file to work natively on Windows, with particular attention paid to making sure the absolute and relative paths work correctly, and actually map to the expected directories on disk. In particular, what does an absolute path on the C: drive need to look like to get mounted into a container?

Just a minor cleanup.

The validate_file function in compose_yml is supposed to make sure that all the files we read and write correspond to an appropriate JSON schema. Unfortunately, this relies on valico, which appears to be currently unmaintained, and which only works with serde 0.8.

See s-panferov/valico#27 for a PR that would fix this.

Another special deserialization case

I think the most important reason for this library would be better error messages.

Serde 0.8

Invalid type. Expected `Bool`

Serde 0.9

font.use_thin_strokes: invalid type: string "tru", expected a boolean at line 51 column 21

Release notes: https://github.com/serde-rs/serde/releases/tag/v0.9.0

See the pull request

This affects cage, as described in faradayio/cage#104.

Using this issue to track the work I intend to be doing against this. The gist of the change is to add support for "long syntax" cached bind-mounted Docker volume definitions here so that this option is recognised when parsing docker-compose files.