In your OS or VM:

• Install docker
• Install docker-compose

This demo uses Ubuntu Linux (16.04 LTS version) in Virtual Box.

The purpose of the repository is to showcase how to dockerize a Spring Boot application as well as a Kafka Broker (along with Zookeeper) on either a Single Host Server or on a Swarm Cluster.

• Stack (Swarm Cluster)
• Services (Single Host)
• Containers (The lowest layer)

We are focusing on "Services" and "Swarm" layers for this demo.

We know dockerization can be used for:

• Development environment
• Automated testing environment
• Single host deployment
• Swarm cluster deployment

On a Swarm cluster, the Swarm mode features:

• Scaling
• Load balancing
• Decentralized design
• Multi-host networking
• Declarative service model
• Service discovery
• Rolling updates
• etc.

The Spring Boot application uses Spring Cloud Stream to communicate (publish/receive messages) with Kafka message broker. The application simulates a compute engine to perform simple calculating tasks generated by REST 'send' request, and return the result to the user by REST 'receive' request. It uses Kafka topics to store requests and responses.

To facilitate the demostration of dockerization, I created two docker images:

• Spring Boot Application Docker Image
• Kafka Docker Image

Also it's worth noting how Kafka (Zookeeper) docker image is built:

• Kafka Docker Repo @ Github

Both docker images are used to create multiple docker containers, which are managed through docker-compose. The docker-compose prescribes the docker stack, services, networks, and deployment, etc.

docker pull michaeldqin/springboot-kafka

docker pull michaeldqin/kafka

In order to run the docker containers, we need a docker-compose.yml to orchestrate the services.

File: docker-compose-single-host.yml

version: '3'
services:
  springboot-webapp:
    image: michaeldqin/springboot-kafka
    ports:
    - 8080:8080
    depends_on:
    - kafka-server
    networks:
    - springboot-network
    logging:
      driver: json-file
  kafka-server:
    image: michaeldqin/kafka
    hostname: kafka-server
    ports:
    - "2181:2181"
    - "9092:9092"
    environment:
    - ADVERTISED_LISTENERS=PLAINTEXT://kafka-server:9092    
    volumes:
    - ${PWD}/.:/opt/kafka_2.12-1.0.0/log
    networks:
    - springboot-network
    logging:
      driver: json-file
networks:
  springboot-network:
    driver: bridge

docker-compose -p dev -f docker-compose-single-host.yml up

Please note, that dev is the project name, and it serves as the development environment for developers.

docker ps

In browser, type: http://localhost:8080

In browser, type: http://localhost:8080/send

In browser, type: http://localhost:8080/receive

docker-compose -p dev -f docker-compose-single-host.yml down

In a Swarm cluster mode, we can have 3 application instances running, and 1 Kafka broker serving all the messages.

File: docker-compose-swarm-cluster.yml

version: '3'
services:
  springboot-webapp:
    image: michaeldqin/springboot-kafka
    ports:
    - 8080:8080
    depends_on:
    - kafka-server
    networks:
    - webnet
    deploy:
      replicas: 3
      restart_policy:
        condition: on-failure
  kafka-server:
    image: michaeldqin/kafka
    hostname: kafka-server
    ports:
    - "2181:2181"
    - "9092:9092"
    environment:
    - ADVERTISED_LISTENERS=PLAINTEXT://kafka-server:9092    
    volumes:
    - ${PWD}/.:/opt/kafka_2.12-1.0.0/log
    networks:
    - webnet
    deploy:
      placement:
       constraints: [node.role == manager]
networks:
  webnet:

Please note, the only difference between docker-compose-single-host.yml and docker-compose-swarm-cluster.yml is the deployment instructions in the docker-compose-swarm-cluster.yml.

# Initialize a Swarm cluster
docker swarm init

# Deploy the stack as 'dev' environment
docker stack deploy -c docker-compose-swarm-cluster.yml dev

docker swarm init

docker stack deploy

docker service ls
## OR
docker container ls

## Check dev_springboot-webapp instances
docker service ps dev_springboot-webapp

Open your favorite browser, in address bar you type: http://localhost:8080, then enter key. You can see Hostname is different from different browser windows, which means multiple application instances are serving the requests.

You can scale the app by changing the replicas value in docker-compose.yml, saving the change, and re-running the docker stack deploy command.

Update: replicas: 5

services:
  springboot-webapp:
    ...
    deploy:
      replicas: 5
    ...

Re-Run:

docker stack deploy -c docker-compose-swarm-cluster.yml dev

# Take down the stack (application)
docker stack rm dev

# Take down the Swarm cluster
docker swarm leave --force

# Remove containers
docker container rm $(docker container ls -a -q)

docker stack ls                                            # List stacks or apps
docker stack deploy -c <composefile> <appname>  # Run the specified Compose file
docker service ls                 # List running services associated with an app
docker service ps <service>                  # List tasks associated with an app
docker inspect <task or container>                   # Inspect task or container
docker container ls -q                                      # List container IDs
docker stack rm <appname>                             # Tear down an application
docker swarm init             # Create a single node (manager) swarm and join it
docker swarm leave --force      # Take down a single node swarm from the manager