Giter Club home page Giter Club logo

ci-for-git-ops-pipeline's Introduction

CI For GitOps Pipeline

This repository contains source code for a project that demonstrates creation of a CI/CD pipeline to automate deployments to a Kubernetes cluster using AWS CodeBuild for CI and Fleet as a GitOps tool.

CI/CD Diagram

Requirements/Prerequisites

Create Kubernetes Clusters with Terraform & Installation of Fleet GitOps tool

In order for you to have the full CI/CD workflow, you will need Kubernetes clusters for Fleet manager and other downstream clusters for deployments. The repository with the source code to provision K8s clusters in EKS, GKE and AKS can be found here. There is a README on how to install Fleet on EKS to manage deployments to the other downstream clusters.

Project Structure

├── README.md
├── application
├── iac
├── import-source-credentials.json
├── k8s-manifests
└── kustomizations

Application

This section refers to the application directory. This folder contains a basic Node.js application with a Dockerfile to containerize the application. The application listens for traffic on port 8080 and has a single endpoint (/test).

IaC (Terraform)

This section refers to the iac directory. It contains all the Terraform source code to provision a CI pipeline with GitHub as the source and a CodeBuild project for the CI stage. Additional resources being created are the IAM permissions for the relevant service roles and the S3 buckets for CodeBuild caching and CodePipeline source artefacts.

Provisioning IaC

AWS Account & Profile

You need to have an AWS account and a profile configured with the AWS CLI on your workstation.

GitHub Personal Access Token & Docker Hub Account

You need to create a GitHub Personal Access Token with the relevant permissions for cloning and pushing to your profile's repo. You also need to create a Docker Hub account. Store both the GitHub PAT and your Docker Hub credentials in AWS Secrets Manager.

Keeping Sensitive Values in sensitive.tfvars

Before provisioning the infrastructure, you need to create a sensistive.tfvars file structure in the following way:

profile="your-aws-profile"
github_secret_name="your-github-secret-name"
docker_secret_name="your-dockerhub-credentials-secret-name"

Provision & Destroy Pipeline Infrastructure in AWS

terraform apply -var-file="sensitive.tfvars" -auto-approve

Destroy Infrastructure

terraform destroy -var-file="sensitive.tfvars" -auto-approve

Kustomizations

This section refers to the kustomizations directory. This folder contains the configuration files for the customizations to be made to the base deployment.yaml manifest file and exports the kustomize build to the k8s-manifests directory.

K8s Manifests

This section refers to the k8s-manifests directory. This folder contains two Kubernetes manifest files:

  • Deployment (deployment.yaml) - This file is generated as part of the CI stage using kustomize and has the Pod container configurations for the new version pushed to Docker Hub.
  • Service (service.yaml) - Creates a service load balancer for the Node.js application.

The Fleet manager should be configured to watch this folder for its deployments to the relevant downstream clusters.

Ideally, this folder should be in a separate repository.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.