jwdobken / terraform-hcloud-kubernetes Goto Github PK

View Code? Open in Web Editor NEW

41.0 3.0 14.0 176 KB

Terraform module to provide Kubernetes for the Hetzner Cloud (non-official)

Home Page: https://registry.terraform.io/modules/JWDobken/kubernetes/hcloud/latest

License: MIT License

HCL 84.02% Shell 15.98%

hetzner kubernetes kubernetes-cluster kubernetes-provider terraform-modules hcloud

terraform-hcloud-kubernetes's Introduction

read 📚 books, play 🥁 drums and drink lots of ☕ espresso...

terraform-hcloud-kubernetes's People

Contributors

Stargazers

Watchers

Forkers

mysticaltech raphael-gosteli omegion kscarlett xavimf87 ryanoatz99 rodriguesl3 michaelgroening teksuo kuzyashin abhishekparihari pavelsinelnik marsch petrmac

terraform-hcloud-kubernetes's Issues

Error: file provisioner error

I create file values.tfvars

cluster_name = "my_cluster"
hcloud_token = "hcloud_token"
hcloud_ssh_keys = ["ssh_key_name"]
worker_count = 1

and after i run command
terraform apply -var-file values.tfvars

manually i add private_key
Enter a value: file("/home/USER/.ssh/private_key")

after that i see

module.cluster.hcloud_server.control_plane_node[0]: Provisioning with 'file'...
╷
│ Error: file provisioner error

Failed to read ssh private key: no key found

Pls, write how i can properly set value for private_key

Node/instance connection not working anymore after node memory overload

Problem
When a node is memory overloaded, the state of that node will be defined as unknown. Meanwhile the instance is running fine when looking in the Hetzner dashboard. The node is not working in Kubernetes and all running pods are stuck in either terminated or restarting state.
A terraform plan would return no changes.

Tested solutions that do not work
I tried a drain and delete for the node and removed the Hetzner instance manually.
After that I retried terraform plan and apply, that would successfully recreate the Hetzner instance, but the instance was not joined to the kubernetes cluster (node not added to cluster).

To be researched

Potentially when a Kubernetes node is running on the instance and crashes, this provider will not restart the node on that instance. This can be tested by memory overloading the node while checking the instance.
When removing the node and instance manually, a kubectl join command seems not to be executed on the instance. This can be tested by manually executing this process while monitoring the processes on the new instance.

ssh: unable to authenticate

I'm getting the following error when applying plan:
Error: timeout - last error: SSH authentication failed (root@NODE_PUBLIC_IP:22): ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
But provider doesn't contain the inputs for ssh private key.

Add connection to Hetzner storage box for ReadWriteMany + cheaper storage then SSD

Example using Samba, but requires manual creating of storage box.
After generating storage box, the $HETZNER_STORAGE_USER is the id of your storage box.
You have to create a password for that specific storage box and use it as $HETZNER_STORAGE_PASSWORD

source: https://github.com/kubernetes-csi/csi-driver-smb

helm repo add csi-driver-smb https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts
helm install csi-driver-smb csi-driver-smb/csi-driver-smb --namespace kube-system --version v1.9.0

Create secret

kubectl create secret generic smbcreds --from-literal username="$HETZNER_STORAGE_USER" \
--from-literal password="$HETZNER_STORAGE_PASSWORD"

smb.yaml:

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-samba
spec:
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: smb
  mountOptions:
    - dir_mode=0777
    - file_mode=0777
  csi:
    driver: smb.csi.k8s.io
    readOnly: false
    volumeHandle: unique-samba  # make sure it's a unique id in the cluster
    volumeAttributes:
      source: "//$HETZNER_STORAGE_USER.your-storagebox.de/backup"
    nodeStageSecretRef:
      name: smbcreds

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-samba
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  volumeName: pv-samba
  storageClassName: smb

Deploy smb.yaml:

kubectl create -f smb.yaml

usage in deployment:

volumes:
  - name: hetzner-storage
    persistentVolumeClaim:
      claimName: pvc-samba

Ubuntu 20 to 22

Try to access list of worker nodes ips

Hi,

I tried to access the list of worker nodes to add them automatically to a hcloud loadbalancer.

Can you provide me with information on how to do that with your module?

Thanks :)

I trying to achieve something like this:

resource "hcloud_load_balancer_target" "load_balancer_target" {
  for_each = toset(module.hcloud_kubernetes_cluster.worker_nodes)

  type = "server"
  load_balancer_id = hcloud_load_balancer.load_balancer.id
  hcloud_load_balancer_target  = each.value.id
}

kubernetes installation on nodes not working

Hi Joost,

I have a strange issue. I was using your awesome little hcloud kubernetes implementation the last years. But now I got a new Mac and had to setup everything freshly and took the chance to update to "JWDobken/kubernetes/hcloud" 0.4.0.

So far so good. My configurations looks like in your example:

# providers
terraform {
  required_providers {
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = ">= 2.0.0"
    }
    hcloud = {
      source = "hetznercloud/hcloud"
    }
  }
}

variable "hcloud_token" {
  type = string
}

resource "hcloud_ssh_key" "geeksven_cluster" {
  name       = "geeksven-cluster"
  public_key = file("~/.ssh/id_rsa.pub")
}

module "hcloud_kubernetes_cluster" {
  source          = "JWDobken/kubernetes/hcloud"
  version         = "0.4.0"
  cluster_name    = "geeksven-cluster"
  hcloud_token    = var.hcloud_token
  hcloud_ssh_keys = [hcloud_ssh_key.geeksven_cluster.id]
  private_key     = file("~/.ssh/id_rsa")
  control_plane_type     = "cx11" # optional
  worker_type     = "cx11" # optional
  worker_count    = 1
}

resource "hcloud_load_balancer" "load_balancer" {
  name               = "geeksven-lb"
  load_balancer_type = "lb11"
  location           = "nbg1"
}

resource "hcloud_load_balancer_network" "cluster_network" {
  load_balancer_id = hcloud_load_balancer.load_balancer.id
  network_id       = module.hcloud_kubernetes_cluster.network_id
}

output "kubeconfig" {
  value = module.hcloud_kubernetes_cluster.kubeconfig
}

But somehow it is not installing the nodes correctly. It feels like the modules/kubernetes/scripts/install.sh is not executed at all. Did you have this issue too?

the error output also states kubectl is missing and that's why I actually checked the nodes. I ssh to them and yes nothing is installed.

Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.hcloud_kubernetes_cluster.module.kubernetes.module.endpoint.null_resource.contents to include new values learned so far during
│ apply, provider "registry.terraform.io/hashicorp/null" produced an invalid new value for .triggers["stderr"]: was cty.StringVal("https://5.75.169.22:6443"), but
│ now cty.StringVal("Warning: Permanently added '5.75.169.24' (ED25519) to the list of known hosts.\r\nbash: kubectl: command not found").
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

I can also provide you with the trace logs from kubernetes if you like to.

I would be very happy to get your help :)

resource "hcloud_server" "server_test" {

  public_net {
    ipv4_enabled = false
  }
}

failed to pull image "registry.k8s.io/kube-apiserver:v1.25.2"

Errorwhile upgrading to kubectl 1.25.2:

module.hcloud_kubernetes_cluster.module.kubernetes.null_resource.install[0] (remote-exec): failed to pull image
"registry.k8s.io/kube-apiserver:v1.25.2": output: E1010 18:34:39.458633    8681 remote_image.go:222] "PullImage from 
image service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.ImageService" 
image="registry.k8s.io/kube-apiserver:v1.25.2"

Reproduce with:

$ ssh root@${control_plane_ip}
$ kubeadm config images pull

replace provisioners with `user_data`

user_data - (Optional, string) Cloud-Init user data to use during server creation