Giter Club home page Giter Club logo

dockermanager's Introduction

Go Report Card

luzifer / dockermanager

The intention of this project is to have a running daemon on a docker host server which is able to realize a configuration of docker containers. For this it manages all containers and images on the docker host. This includes starting and stopping containers which are or are not defined by the configuration file.

Requirements

  • One or more host servers with latest docker-ce / docker-ee
  • A config file or config URL to serve the configuration from
  • Docker daemon listening on tcp port
  • The dockermanager set up
  • If you want to use images from a private registry put a .dockercfg file (docker login) to the homedir of the user running dockermanager

Wasn't this supposed to be a cluster manager?

Yeah, speaking of pre-1.0 versions this is true. In the early development the dockermanager was intended to use a serf cluster to manage a whole cluster of machines. Because I never had the need to manage a cluster on my private projects I never really worked on the cluster logic.

In the meantime a bunch of cluster managers / schedulers having the capability to run Docker containers emerged and therefore I decided to cut out the cluster functionality. Following the principle "do one thing and do it well" the dockermanager now concentrates on managing single machines.

I'm managing a handfull of servers running as single nodes and that's where I'm optimizing the dockermanager. If you are searching for a solution to manage a cluster of machines you might want to take a look at these ones:

  • Amazon ECS (Running Docker containers on a cluster of EC2s)
  • Hashicorp Nomad (Full cluster solution including ability to run Docker containers)
  • Kubernetes (Automated container deployment, scaling, and management)
  • Mesosphere DC/OS (OS built around containers and services)

Configuration

CLI parameters

# ./dockermanager --help
Usage of ./dockermanager:
  -c, --config string         Config file or URL to read the config from (default "config.yaml")
      --configInterval int    Sleep time in minutes to wait between config reloads (default 10)
      --docker-certs string   Directory containing cert.pem, key.pem, ca.pem for the registry
      --docker-host string    Connection method to the docker server (default "unix:///var/run/docker.sock")
      --fullHost              Manage all containers on host (default true)
      --log-level string      Set log level (debug, info, warning, error) (default "info")
      --refreshInterval int   fetch new images every <N> minutes (default 30)

Configuration file

The configuration is written in YAML format and reloaded regularly by the daemon:

  • container-name: Name of the container on the host. Needs to be unique
    • command: Override CMD value set by Dockerfile
    • hosts: Array of hostnames to deploy the container to or ALL
    • image: Name of the image registry or luzifer/jenkins or my.registry.com:5000/secret
    • tag: Tag for the image, probably latest
    • links: Links to other containers in format othercontainername:alias
    • volumes: Volume mapping in form <localdir>:<containerdir>
    • ports: Array of port configurations
      • container: Exported port in the container e.g. 80/tcp or 12201/udp
      • local: IP/port combination in the form <ip>:<port>
    • environment: Array of environment variables in form <key>=<value>
    • update_times: Array of allowed time frames for updates of this container in format HH:MM-HH:MM (Optional, if not specified container is allowed to get updated all the time.)
    • start_times: Cron-style time specification when to start this container. Pay attention to choose a container quitting before your specified interval for this. Containers having this specification will not get started by default and are not restarted after they quit. Use this for starting cron-like tasks.
    • stop_timeout: Time in seconds to wait when stopping a deprecated container to be exchanged. (default: 5s)
    • labels: Labels to attach to the container
    • add_cap: Array of capabilities to add to this container
    • depends_on: Array of container names to start before this one

Example configuration for a jenkins container:

---
jenkins:
  hosts:
    - docker01
  image: luzifer/jenkins
  tag: latest
  links:
    - "othercontainername:alias"
  volumes:
    - "/home/ubuntu/data/jenkins_home:/var/jenkins_home"
  ports:
    - container: 8080/tcp
      local: 0.0.0.0:1000
  environment:
    - MYVAR=value
  update_times:
    - 04:00-06:00
  stop_timeout: 20


scheduletest:
  hosts:
    - docker01
  image: jlekie/curl
  tag: latest
  command:
    - "http://example.com/page"
  start_times: "*/2 * * * *"

dockermanager's People

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

jonyboy2000

dockermanager's Issues

Container are started in wrong order

When containers are linked the containers are probably started in wrong order: There is no dependency graph inside dockermanager so it does not know which container needs which container to be started.

Jul 07 07:35:32 fwwedel01 dockermanager[851]: time="2017-07-07T07:35:32+02:00" level=error msg="Unable to create container 'jenkins': API error (500): {\"message\":\"Could not get container for crowd\"}\n"

Additionally things break when linked containers are restarted so when restarting a linked container all containers linked to that one needs to be restarted too.

Index out of range [1.2.0-rc6]

panic: runtime error: index out of range

goroutine 131 [running]:
main.(*scheduler).imageManager(0xc420141340)
        /go/src/github.com/Luzifer/dockermanager/scheduler.go:328 +0x7b0
created by main.newScheduler
        /go/src/github.com/Luzifer/dockermanager/scheduler.go:90 +0x2a3

Cleanup is removing intermediate images

When cleaning exited containers the cleanup process destroys intermediate containers for docker builds:

Installing collected packages: BeautifulSoup, boto, pycrypto, requests, six, coinbase, CouchDB, feedparser, haversine, MarkupSafe, Jinja2, oauthlib, python-dateutil, python-memcached, python-pushover, pytz, PyYAML, redis, requests-oauthlib, tzlocal, prometheus-client, hvac
Successfully installed BeautifulSoup-3.2.1 CouchDB-0.9 Jinja2-2.8 MarkupSafe-0.23 PyYAML-3.12 boto-2.2.2 coinbase-2.0.6 feedparser-5.2.1 haversine-0.1 hvac-0.2.17 oauthlib-1.1.2 prometheus-client-0.0.19 pycrypto-2.6.1 python-dateutil-2.6.0 python-memcached-1.52 python-pushover-0.2 pytz-2016.10 redis-2.10.3 requests-2.12.4 requests-oauthlib-0.5.0 six-1.10.0 tzlocal-1.3
---> 0c34b63f8030
Error removing intermediate container 6e4196519bc1: No such container: 6e4196519bc15f66e63610ba8937cf42629e2d40079bb5081126582ded64456f
Successfully built 0c34b63f8030

todo

  • Only remove exited containers after they are already dead for time X (X=1h?) or are to be replaced by a new container
  • Only remove dangling images after they are time X (X=1h?) old

Use networks instead of container links

Warning: The --link flag is a deprecated legacy feature of Docker. It may eventually be removed. Unless you absolutely need to continue using it, we recommend that you use user-defined networks to facilitate communication between two containers instead of using --link.
(https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/)

  • Support specifying networks
  • Create network if not already persistent
  • Cleanup networks with no more containers attached if created by dockermanager
  • Deprecate links attribute

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.