inscapist / terraform-k3s-private-cloud Goto Github PK

Private cluster with k3s. Why have 1 huge complicated cluster (pet) when you can have many simple, cheap clusters (cattle)?

License: MIT License

HCL 92.79% Shell 7.21%

k3s k3s-cluster aws terraform kubernetes session-manager cloud cloud-init microservices ci-cd

terraform-k3s-private-cloud's Introduction

K3s Private Cluster

Asciicast for accessing cluster can be viewed here

Requirements

Name	Version
terraform	>= 0.14
SSM Agent	>= 3.0.161.0

This module is published at Terraform registry

Why would you want this?

You are cost-conscious. RDS, Elasticache and EKS are too expensive and perhaps even too complicated
You want a cluster with low maintenance overhead and just works
You want to have this cluster for only 1 or 2 apps (App Clusters)

Why would I do this?

I wish to empower individual developers to have their own test/staging environment
Big clusters fail, coordination is hard, service mesh is expensive
I wish our code can simply run anywhere on any cloud, even if its stateful

Features

Private cluster with no exposed ports
Low memory usage and simpler architecture (thanks to K3S)
It has most of the expected features of kubernetes (PVC, LoadBalancer, MetricsServer)
Simple codebase with minimal magic
Network agnostic with ability to reuse VPC

What is this?

This module is designed for kubernetes workload that runs within private subnet. A private subnet is simply a subnet not associated with an internet gateway. This results in lower cost and reduction in moving parts, as you do not need to purchase a domain name, certificate and manage perimeter security.

If you don't need egress connectivity for cluster, you can use the VPC endpoints for SSM. Otherwise, session manager would not work.

This module currently supports only AWS cloud. However, I wish to support more clouds with minimal differences between API.

Various networking configurations are demonstrated in the following examples:

How do I start?

Refer Getting Started and other documentation

terraform-k3s-private-cloud's People

Contributors

Stargazers

Watchers

Forkers

florianstoeber avineshwar felipecruz91 mat-work rolling-space maniacs-oss sethodessa niazhussain ryanoatz99 shaunc peter-lankton bridgecrew-perf4 avk999

terraform-k3s-private-cloud's Issues

Supports more cloud (GCP, Azure, Digital Ocean, etc)

References:

https://github.com/kubernetes/kops/tree/master/pkg/resources

WIP: Volume CSI interface

tba

Implement cloud alarm similar to cloudposse's

https://github.com/cloudposse/terraform-aws-ec2-instance

Add port tunneling capability to GoSSM

GoSSM is a nice project because it adds more ease of use to the project.

However, it currently does not support port tunneling yet.

https://github.com/gjbae1212/gossm

doc/feature: add/remove node to cluster

Thank you for this repo! I plan to use this for an "app cluster" in which one service will be running continuously, and another task will run periodically (once daily). I don't want to have to provision a backing node to support the periodic task; rather, I'd like to have a 2nd scaling group that I can scale up when I need it and then scale down. (2nd ASG because different type of node needed. Automatic autoscaling via cluster autoscaler would be even better, but not sure if possible easily outside of EKS.)

Perhaps a "recipe" for this could be added to the doc? Can I just copy the node pool and node code, and use it?

hmm... it seems the cluster_token is not part of the outputs. Maybe as well as a recipe, a few changes could be made to enable this?

Alternately, you could support multiple autoscaling groups directly.

A node pool can run as ondemand or spot-instances

module "worker-pool" {
  source = ".."

  node_pool            = "r6g.medium"
  min_size             = 0
  max_size             = 10
  on_demand_percentage = 0

  equivalent_instance_types = [
    "r6g.large",
    "r5.large",
    "r4.large",
  ]

  target_group_arns      = [] # for egress-ready instances
  cluster_server         = k3s_server._private_dns
  vpc_security_group_ids = module.common.vpc_security_group_ids
  node_pool_tags         = module.common.node_pool_tags
}