Giter Club home page Giter Club logo

kubernetes-django's Introduction

Scalable and resilient Django with Kubernetes

This repository contains code and notes to get a sample Django application running on a Kubernetes cluster. It is meant to go along with a related blog post that provides more context and explains some of the theory behind the steps that follow.

Preliminary steps

  1. Fetch the source code for this example.

    git clone https://github.com/hnarayanan/kubernetes-django.git
    
  2. Install Docker.

  3. Take a look at and get a feel for the example Django application used in this repository. It is a simple blog that’s built following the excellent Django Girls Tutorial.

  4. Setup a cluster managed by Kubernetes. The effort required to do this can be substantial, so one easy way to get started is to sign up (for free) on Google Cloud Platform and use a managed version of Kubernetes called Google Container Engine (GKE).

    1. Create an account on Google Cloud Platform and update your billing information.

    2. Install the command line interface.

    3. Create a project (that we'll refer to henceforth as $GCP_PROJECT) using the web interface.

    4. Now, we're ready to set some basic configuration.

      gcloud config set project $GCP_PROJECT
      gcloud config set compute/zone europe-west1-d
      
    5. Then we create the cluster itself.

      gcloud container clusters create demo
      gcloud container clusters list
      
    6. Finally, we configure kubectl to talk to the cluster.

      gcloud container clusters get-credentials demo
      kubectl get nodes
      

Create and publish Docker containers

For this example, we'll be using Docker Hub to host and deliver our containers. And since we're not working with any sensitive information, we'll expose these containers to the public.

PostgreSQL

Build the container, remembering to use your own username on Docker Hub instead of hnarayanan:

cd containers/database
docker build -t hnarayanan/postgresql:9.5 .

You can check it out locally if you want:

docker run --name database -e POSTGRES_DB=app_db -e POSTGRES_PASSWORD=app_db_pw -e POSTGRES_USER=app_db_user -d hnarayanan/postgresql:9.5
# Echoes $PROCESS_ID to the screen
docker exec -i -t $PROCESS_ID bash

Push it to a repository:

docker login
docker push hnarayanan/postgresql:9.5

Django app running within Gunicorn

Build the container:

cd containers/app
docker build -t hnarayanan/djangogirls-app:1.2-orange .

Push it to a repository:

docker push hnarayanan/djangogirls-app:1.2-orange

We're going to see how to perform rolling updates later in this example. For this, let's create an alternative version of our app that simply has a different header colour, build a new container app and push that too to the container repository.

cd containers/app
emacs blog/templates/blog/base.html

# Add the following just before the closing </head> tag
    <style>
      .page-header {
        background-color: #ac4142;
      }
    </style>

docker build -t hnarayanan/djangogirls-app:1.2-maroon .
docker push hnarayanan/djangogirls-app:1.2-maroon

Deploy these containers to the Kubernetes cluster

PostgreSQL

Even though our application only requires a single PostgreSQL instance running, we still run it under a (pod) replication controller. This way, we have a service that monitors our database pod and ensures that one instance is running even if something weird happens, such as the underlying node fails.

cd  kubernetes/database
kubectl create -f replication-controller.yaml

kubectl get rc
kubectl get pods

kubectl describe pod <pod-id>
kubectl logs <pod-id>

Now we start a service to point to the pod.

cd  kubernetes/database
kubectl create -f service.yaml

kubectl get svc
kubectl describe svc database

Django app running within Gunicorn

We begin with three app pods (copies of the orange app container) talking to the single database.

cd kubernetes/app
kubectl create -f replication-controller-orange.yaml
kubectl get pods

kubectl describe pod <pod-id>
kubectl logs <pod-id>

Then we start a service to point to the pod. This is a load-balancer with an external IP so we can access the site.

cd kubernetes/app
kubectl create -f service.yaml
kubectl get svc

Before we access the website using the external IP presented by kubectl get svc, we need to do a few things:

  1. Perform initial migrations:

    kubectl exec <some-app-orange-pod-id> -- python /app/manage.py migrate
    
  2. Create an intial user for the blog:

    kubectl exec -it <some-app-orange-pod-id> -- python /app/manage.py createsuperuser
    
  3. Have a CDN host static files since we don't want to use Gunicorn for serving these. This demo uses Google Cloud storage, but you're free to use whatever you want. Just make sure STATIC_URL in containers/app/mysite/settings.py reflects where the files are.

    gsutil mb gs://demo-assets
    gsutil defacl set public-read gs://demo-assets
    
    cd django-k8s/containers/app
    virtualenv --distribute --no-site-packages venv
    source venv/bin/activate
    pip install Django==1.9.5
    export DATABASE_ENGINE='django.db.backends.sqlite3'
    ./manage.py collectstatic --noinput
    gsutil -m cp -r static/* gs://demo-assets
    

At this point you should be able to load up the website by visiting the external IP for the app service (obtained by running kubectl get svc) in your browser.

Go to http://app-service-external-ip/admin/ to login using the credentials you setup earlier (while creating a super user), and return to the site to add some blog posts. Notice that as you refresh the site, the name of the app pod serving the site changes, while the content stays the same.

Play around to get a feeling for Kubernetes' API

Now, suppose your site isn't getting much traffic, you can gracefully scale down the number of running application pods to one. (Similarly you can increase the number of pods if your traffic starts to grow!)

kubectl scale rc app-orange --replicas=1
kubectl get pods

You can check resiliency by deleting one or more app pods and see it respawn.

kubectl delete pod <pod-id>
kubectl get pods

Notice Kubernetes will spin up the appropriate number of pods to match the last known state of the replication controller.

Finally, to show how we can migrate from one version of the site to the next, we'll move from the existing orange version of the application to another version that's maroon.

First we scale down the orange version to just one copy:

kubectl scale rc app-orange --replicas=1
kubectl get pods

Then we spin up some copies of the new maroon version:

cd kubernetes/app
kubectl create -f replication-controller-maroon.yaml
kubectl get pods

Notice that because the app service is pointing simply to the label name: app, both the one orange and the three maroon apps respond to http requests to the external IP.

When you're happy that the maroon version is working, you can spin down all remaining orange versions, and delete its replication controller.

kubectl scale rc app-orange --replicas=0
kubectl delete rc app-orange

Cleaning up

After you're done playing around with this example, remember to cleanly discard the compute resources we spun up for it.

gcloud container clusters delete demo
gsutil -m rm -r gs://demo-assets

And coming in the future

Future iterations of this demo will have additional enhancements, such as using a Persistent Volume for PostgreSQL data and learning to use Kubernetes' Secrets API to handle secret passwords. Keep an eye on the issues for this project to find out more. And you're free to help out too!

kubernetes-django's People

Contributors

hnarayanan avatar tdumitrescu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kubernetes-django's Issues

No External ip for service

  • When I ran kubectl get svc , I got all services but their was no External ip assigned to my service.
  • So how can I ran the dev server of my app ?
app              10.0.0.204   <pending>     80:31987/TCP     17h

Introduce Persistent Volume (Claim) for Postgresql data

From older notes:

  • (WIP!) Setup a persistent store for the database. In this example we're
    going to be using Persistent Disks from Google Cloud Platform. In
    order to make one of these, we create a disk and format it (using an
    instance that's temporarily created just for this purpose).
gcloud compute disks create pg-data-disk --size 50GB
gcloud compute instances create pg-disk-formatter
gcloud compute instances attach-disk pg-disk-formatter --disk pg-data-disk
gcloud compute config-ssh
ssh pg-disk-formatter.$GCP_PROJECT
    sudo mkfs.ext4 -F /dev/sdb
    exit
gcloud compute instances detach-disk pg-disk-formatter --disk pg-data-disk
gcloud compute instances delete pg-disk-formatter
- PostgreSQL Persistent Volume (Claims)
  kubectl create -f kubernetes/database/persistent-volume.yaml
  kubectl get pv
  kubectl create -f kubernetes/database/persistent-volume-claim.yaml
  kubectl get pvc

Error while doing migration

I am getting this error while migrating:

Is the server running on host "127.0.0.1" and accepting TCP/IP connections on port 5432?

I think that is because DATABASE_SERVICE_HOST env variable is not set.
How can I fix this?

current point of view?

Hi,

you were some of the early adopters. What is your experience after five years?

Do you still recommend the text you wrote some years ago?

Step for static assets on CDN missing some dependencies

When you get to the step

./manage.py collectstatic 

you'll need Django installed. Using a Python virtualenv, something like

mkvirtualenv django
pip install django

Unfortunately, you also need psycopg2 which suffers from the leaky abstraction of needing Postgres in the host OS. So, on a mac, you also need to do

brew install postgresql
pip install psycopg2

Woe unto you if you also need to go install Homebrew.

Unpin the postgresql install

Hi, thanks for the awesome blog post (came here from HN). It's inspired me to finally get around to trying out GCE.

When I run

docker build -t hnarayanan/postgresql:9.5  .

I'm getting the following errors:

E: Version '9.5.0-1.pgdg80+2' for 'postgresql-9.5' was not found
E: Version '9.5.0-1.pgdg80+2' for 'postgresql-contrib-9.5' was not found

If I just unpin the postgres versions in the apt-get installs, the image builds fine:

diff --git i/containers/database/Dockerfile w/containers/database/Dockerfile
index 5058b6b..aa09a76 100644
--- i/containers/database/Dockerfile
+++ w/containers/database/Dockerfile
@@ -31,8 +31,8 @@ RUN apt-get update \
        && apt-get install -y postgresql-common \
        && sed -ri 's/#(create_main_cluster) .*$/\1 = false/' /etc/postgresql-common/createcluster.conf \
        && apt-get install -y \
-               postgresql-$PG_MAJOR=$PG_VERSION \
-               postgresql-contrib-$PG_MAJOR=$PG_VERSION \
+               postgresql-$PG_MAJOR \
+               postgresql-contrib-$PG_MAJOR \
        && rm -rf /var/lib/apt/lists/*

replication-controller.yaml volumeMounts error

There is an error with the "volumeMounts" section (which is commented by default).

You should change it by the correct one. Like this:

     # TODO: User persistent disks, volumes and claims for the following
      volumeMounts:
        - name: pg-data
          mountPath: /var/lib/postgresql/data
  volumes:
    - name: pg-data
      persistentVolumeClaim:
        claimName: pg-data-claim

Show how to test the site locally within Docker

You can check it out locally if you want:

# TODO: The following needs to be fixed, and needs to be connected
# with the running local postgres container above
# docker run --name some-app --link some-postgres:postgres -d application-that-uses-postgres

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.