weaveworks / footloose Goto Github PK

We're planning more examples, can't really cram them all in the README. We should split them into their own directory so we can provide example files and attach screenshots when necessary.

The main README should have the list of examples and link to them for discoverability.

It's often useful to pre-seeed the machine image with packages we want installed. This can be easily done by deriving a footloose image and adding a few RUN steps. With then need to indicate the image we want to run in the config file.

We should document the above.

While some tests need OS-specific commands to be executed (ie. to install an RPM), a lot of tests are really the same, just changing the container docker image.

https://github.com/dlespiau/footloose/blob/master/tests/test-create-delete-amazonlinux2.cmd
https://github.com/dlespiau/footloose/blob/master/tests/test-create-delete-centos7.cmd
https://github.com/dlespiau/footloose/blob/master/tests/test-create-delete-ubuntu18.04.cmd.

We should find a way to write that test only once, but executed for each image.

Maybe https://github.com/dlespiau/footloose/blob/master/tests/test-create-delete-all.cmd and, we'd detect within that test, %image would be replaced by the image name. We may be able to avoid having a special name ending with -all for the test: if we detect the %image replacements is really an array, we could instantiate tests for each element of the array. That would make it a bit more generic so the e2e tests could be reused elsewhere.

We don't currently use port mapping to access the SSH daemon because, on Linux, the container IP is routable from the host and we can just open a connection.

This doesn't work on macOS, we need to map the container port 22 to a localhost port before we can login. This will also be true for any service the user would like to access from the host.

We should expose port mappings, just like we do with volumes and default to mapping port 22 to either a fixed port or a port automatically picked up by docker.

Say export FOOTLOOSE_CLUSTER=$PWD/footloose.yaml

I like the pattern of pushing a tag and get CI to attach binaries to the Github release. Should do just that!

Releases should have docker images locked down to specific versions so users don't have the OS image changing between two invocations of footloose create.

Unfortunately, the macOS machines on Travis are VMs with no nested virtualization support. This means docker for mac doesn't work there.

travis-ci/travis-ci#7773
https://hackernoon.com/why-xod-switched-from-travis-to-circle-ci-98b6832f2d96

So we can't test footloose on macOS with travis. CircleCI should work though.

That's quite a bit of work but it'd be nice to have a footloose terraform provider so people can fake terraforming VMs on their development machine.

It'd be nice to be able to select the containers footloose creates with labels, exposing them in the configuration.

Question: Should we have a default footloose label?
Question: Do they belong to the Cluster or Machine object?

We have a simple example of image customization by adding a package, it'd be nice to have a slightly augmented version of this installing apache and exposing port 80 to the host.

I took care of isolating tests enough so they should be runnable in parallel. Now we need to test that hypothesis.

Another way to do it would be to shard the tests per OS, have n test runners in CI, each of them selecting an OS.

We probably need to start with a docker pull to retrieve the OS images so they are local when the tests need them to start the machines.

It's a bit annoying to have the command usage displayed on error and makes the error not the most prominent thing that appears on the console. Should find a way to not display the usage.

A very frequent test case is to be able to mount a local directory (say a git checkout, or $HOME) to expose parts of the host fs inside the machine.

There's one detail though, we are likely to want that, when writing files to that mount, we want to write them with the id, gid of the user of the host machine.

There are a few things that could do with validation, empty fields should default to reasonable values, machine names having to have a %d in them, validate machine names are valid hostnames, that both cluster name and machine names are valid docker container names, ...

it's ssh after all, we should be able to have:

$ footloose ssh node1 ls /

We should push the docker images used in the container machines from the CI run, not me pushing them by hand .

We should set the machine hostnames to config.machines[i].Name instead of leaving them to their default value of the container ID.

At the moment, cobra/pflag will bail out if we do something like:

$ footloose ssh node0 yum install -y iptables
Error: unknown shorthand flag: 'y' in -y

This can be worked around by explicitly stopping option parsing:

$ footloose ssh node0 -- yum install -y iptables

We could probably do better though? tell cobra/pfag to just leave unknown options alone?

It'd be nice to provide Amazon Linux 2 footloose base images. From a quick glance, it looks like a centos/rhel based image and version 2 comes with systemd.

Took the official image for as spin and systemd isn't installed by default but it's only a yum install systemd away

Given the above, building an amazon linux 2 base image should be quite close to what we do for centos7: https://github.com/dlespiau/footloose/blob/master/images/centos7/Dockerfile. We should put it in `

A detail is that currently, only I can push images to quay.io/footloose, this should be automated and pushed from CI (#23). It doesn't prevent people from creating and using a new locally though.

config can only be used to create a new config file at the moment, but we probably should also have something that can retrieve config values and something that can change an already existing config file.

A nice to do that would be to add sub-commands to config:

$ footloose config create --replicas 3
$ footloose config get cluster.name
$ footloose config set cluster.machines[0].privileged true

These are separate tasks, this issue is about the first line.

We should have a verbose option that shows everything we do, including the commands being run on the host and in the containers.

Hi @dlespiau,

I really like your project, would you consider adding it yourself to https://github.com/veggiemonk/awesome-docker#development-environment ?

If you have a idea for better suited section, let me know in the PR we can have discussion there with the other team members :D

Have a great day,

Julien

We should be able to use the STOPSIGNAL command to the dockerfiles to correctly shutdown systemd.

Source: https://www.freedesktop.org/wiki/Software/systemd/ContainerInterface/

We should start CI with some basic Travis tests: create a couple of CMs, check the containers are running, execute an ssh command.

We could show the cluster status and various information about the "machines", maybe.

The capture is a bit old and the CLI has seen two changes since then:

• footloose config create
• footloose ssh root@node0

It'd be nice to keep that capture in sync.

It'd be nice to have an exemple showing how to set up ansible with footloose machines.

Examples should go in examples/ and should be linked to from the README.

It'd be nice to be able to provision non-root users when creating container machines so we can login as non-root. Maybe something like:

cluster:
  name: cluster
  privateKey: cluster-key
machines:
- count: 3
  spec:
    image: quay.io/footloose/centos7
    name: node%d
    portMappings:
    - containerPort: 22
    groups:
    - name: damien
      id: 1001
    users:
    - name: damien 
      id: 1001
      gid: 1001
      groups:
      - docker    

This has some overlap with #11. I'm not sure how useful it is to create a user with a fixed (id,gid) compared to a dynamic user that mirrrors the current user inside the container.

Maybe we want both and they should be different features. The issue with implementing #11 with the idea above is that the configuration file would change depending on which machine/user is using the configuration.

This could mean #11 may be implemented in a different way:

cluster:
  name: cluster
  privateKey: cluster-key
machines:
- count: 3
  spec:
    image: quay.io/footloose/centos7
    name: node%d
    portMappings:
    - containerPort: 22
    mirrorUser: true

mirrorUser (or any better name!) would instruct footloose to mirror the user running footloose inside the CM.

Following on #27, we should have a set sub-command:

$ footloose config set cluster.machines[0].privileged true

Steps to reproduce the issue

$ footloose config create --replicas 2

### edit footloose.yaml to add hostPort, as per: https://godoc.org/github.com/dlespiau/footloose/pkg/config#PortMapping
$ cat footloose.yaml
cluster:
  name: cluster
  privateKey: cluster-key
machines:
- count: 2
  spec:
    image: quay.io/footloose/centos7
    name: node%d
    portMappings:
    - containerPort: 22
      hostPort: 2222

### check ports specified are free (under macOS):
$ sudo lsof -iTCP:2222 -sTCP:LISTEN
$ sudo lsof -iTCP:22 -sTCP:LISTEN
$ sudo lsof -iTCP:2223 -sTCP:LISTEN

$ footloose create
INFO[0000] Image: quay.io/footloose/centos7 present locally
INFO[0000] Creating machine: cluster-node0 ...
INFO[0001] Creating machine: cluster-node1 ...
ERRO[0001] 222c7f0c606a48beb919b4137a5d492ed985dbdf892de1d43b91ae6e2cf3a144
ERRO[0001] docker: Error response from daemon: driver failed programming external connectivity on endpoint cluster-node1 (b1bd36d39368f0af75b48987f1d970f6bb7b2fbe38e43cf952ef8c94033c8490): Bind for 0.0.0.0:22 failed: port is already allocated.
FATA[0001] exit status 125

$ docker ps
CONTAINER ID        IMAGE                       COMMAND             CREATED             STATUS              PORTS                  NAMES
426cf434fa00        quay.io/footloose/centos7   "/sbin/init"        7 seconds ago       Up 6 seconds        0.0.0.0:22->2222/tcp   cluster-node0

Expected behaviour

CONTAINER ID        IMAGE                       COMMAND             CREATED             STATUS              PORTS                  NAMES
deadbeefbabe        quay.io/footloose/centos7   "/sbin/init"        7 seconds ago       Up 6 seconds        0.0.0.0:2222->22/tcp   cluster-node0
deadbeefcafe        quay.io/footloose/centos7   "/sbin/init"        6 seconds ago       Up 5 seconds        0.0.0.0:2223->22/tcp   cluster-node1

Using docker directly has been great to bring the tool to a usable state. An idea is to be able to use CRI instead to create containers, that would open the door to very interesting container runtimes out there, including ones that actually create VMs!

I tuned a bit kind's docker.Inpect in kubernetes-sigs/kind@7e8624b, we should update to this version.

At the moment, running create while the cluster is running will result in an error (because the containers already exist):

$ footloose create
INFO[0000] Image: quay.io/footloose/centos7 present locally 
INFO[0000] Creating machine: cluster-node0 ...          
ERRO[0000] docker: Error response from daemon: Conflict. The container name "/cluster-node0" is already in use by container "5134f80b733ecf4561f154c7c43eca2fb5273ae8da0f7c73402b2205ceef1a3e". You have to remove (or rename) that container to be able to reuse that name. 
ERRO[0000] See 'docker run --help'.                     

Same with delete.

We should make them idempotent.

Now that we have our first release, we should get a version command! Use git describe --tags and inject the version through a linker flag.

The main thing is to create a /var/lib/docker volume.

At the moment we hardcode "root@nodeIP" as the ssh argument, we should probably make the user explicit and default to the current user, like the ssh command does, to avoid surprising the user:

$ footloose ssh root@nodes

This should also play better with the plan of letting users login with heir own id/gid as described in #11.

tests/ should have a readme explaining:

• How to run the tests (footloose in path, go test -v)
• How to create a new tests
• Document the various variables available in the .cmd files.

Following on #27, we should have a get sub-command:

$ footloose config get cluster.name

We should check the machine name correspond to the configuration.

to integrate with external tools, it would be very useful to have a 'list' command that outputs all the machine hostname, and maybe optionally the current host-container port mapping. This can be done with docker but there can be other containers running, we need only the ones created by footloose.

Just like ubuntu 18.04, we should add the 16.04 LTS base image to footloose.

We should be able to create configuration files with new port mappings from the command line, probably using the same syntax as docker for binding an address or supplying a port: https://docs.docker.com/config/containers/container-networking/.

footloose config create --port 8080:80
footloose config create -p 192.168.1.100:8080:80

(yes, -p is for --publish with docker, I like --port a tiny bit better for the option long name 🤷‍♂️ )

Container images tend to be quite stripped down, but for container machines, we want a basic set of tools to be available. Defining the exact criteria to decide if a command should be there or not is going to be tricky so we'll have to play it by ear. eg.

• hostname
• ps
• ifconfig/ip
• netstat?
• vi/vim
• ping
• curl
• wget

Extracted from #76

footloose could run provisioning commands when creating a machine. That means we could run an ansible playbook on machine creation automatically.

Of course there are security concerns and we shouldn't allow to execute arbitrary commands. By default I think we shouldn't allow any command to be run unless the user tells us it's ok:

$ footlose create --allow-provisioning=ansible # <- name(s) of the command(s) the user allows us to exec.

Two more desirable behaviours:

• Error out if a file exists
• "Merge" the existing config file with the option(s) given on the command line