The purpose of the project is to establish the Poor Man's High Availability architecture on AWS using the right and most efficient tools for each step.

Having multiple Jenkins controller instances, attached to the same file system, might lead to inconsistencies. So, when the instance goes down, there is a few moments of downtime (VM + Java startup time).

Adopting an Immutable Infrastructure approach makes the upgrading and patching process easy. All we need is to upgrade the AMI instance an configure the ASG to use the latest version and as long as the data persists in EFS all the configuration will remain the same.

• Terraform: Provisioning AWS resources.
• Packer: Building Jenkins controller and agent AMIs (Amazon Machine Images).
• Ansible: Configuring both Jenkins controller and agent during the AMI build process.
• AWS CLI: Interacting with AWS Services through terminal.

• IAM Roles: Grant permissions to Jenkins controller and agent instances.
• EFS Filesystem: Persist Jenkins data (backup data when EC2 instance is recreated).
• AWS Parameter Store: Store SSH keypair.
• Autoscaling Group (ASG): Recreate the Jenkins controller instance whenever terminated.
• Application Load Balancer (ALB): Routes to the Jenkins controller instance deployed in a random AZ.

You need to follow the following steps

i. Generate SSH keys

ssh-keygen  

ii. upload ssh keypair

cd terraform/ssm ; terraform init && terraform apply -auto-approve

iii. Build AWS AMIs

$ make jenkins-agent
$ make jenkins-controller efs_id=<efs-filesystem-id> region=<aws-region>

iv. Provision resources accordingly through terraform modules

The diagram below shows how we designed the tools and services above to work together and achieve our goal.

Jenkins is deployed in an AutoScaling Group (ASG) with a Min and Max 1 count. Hence, we have a single instance of Jenkins controller running all the time.

we also have a dedicated EFS disk that holds all Jenkins data. The Jenkins controller AMI will have teh EFS disk mount configuration in the EFS entry.

If the Jenkins controller instance goes down, the ASG policie will bring another instance. In the process the data disk gets attached from the terminating instance to the new instance for serving the previous Jenkins data.

All the existing jobs will fail during the downtime and continue when the instance is ready.

We need to keep this project as close as possible to be production-ready. Here are a few things to keep in mind regarding a real-world comparison.

Everyone can track upcoming features here and contributions are more than welcome :)

1. Private network, DNS and TLS
2. Jenkins access through a VPN/VDI connection
3. Jenkins user authentication will be based on LDAP / SSO (Eg. Okta)

• VPC: 3 public subnets and 3 private subnets
• Client to Site VPN
• ASG in Private Subnet:
• EFS in Private Subnet: Inbound access only from Private Subnets CIDR.
• Load Balancer with TLS: TLS configured in Load Balancer
• CloudWatch Monitoring and Logging: Both for AWS Resources and Jenkins.
• EFS Backup: AWS Backup Service / EFS to EFS Backup.

The Apache 2.0 License is permissive. It allows you to use, modify, and distribute the licensed software, including creating derivative works, without requiring those derivative works to be licensed under the same terms. You can release the modified parts of the code under any license you prefer.