NameServer limits were exceeded about cluster-api-provider-hetzner HOT 7 CLOSED

@FischerLGLN thank you very much. The problem comes from the automated setting of the nameservers from the hcloud cloud-init scripts. These are automatically executed by startup. These set a total of 4 nameservers. However, many libraries like glibc can only handle 3 nameservers and i think the kubelet is using it under the hood...

In Fedora, however we cannot change the /etc/resolv.conf directly as it's managed by systemd with the NetworkManager.
You have two options. Fixing it by providing a custom resolv.conf only for the kubelet and pointing to it by providing the kubelet flag --resolv-conf=/etc/kubernetes/resolv.conf.
Therefore add something like this:

cat <<'EOF' >/etc/kubernetes/resolv.conf
nameserver 1.1.1.1
nameserver 1.0.0.1
nameserver 2606:4700:4700::1111
EOF

And/or by changing the node-wide dns settings.

You could use such a script:

cat <<'EOF' >/opt/set-dns-servers.sh
#!/usr/bin/env bash
nmcli -g name,type connection  show  --active | awk -F: '/ethernet/ { print $1 }' | while read connection
do
  nmcli con mod "$connection" ipv6.ignore-auto-dns yes
  nmcli con mod "$connection" ipv4.ignore-auto-dns yes
  nmcli con mod "$connection" ipv4.dns "1.1.1.1 1.0.0.1"
  nmcli con mod "$connection" ipv6.dns "2606:4700:4700::1111"
  nmcli con down "$connection" && nmcli con up "$connection"
done
EOF

P.S. If you like the project please leave a ⭐ ;)

from cluster-api-provider-hetzner.

@FischerLGLN you're welcome! Also check out this helm chart for more "natural" usage of capi with hetzner: https://github.com/syself/charts/tree/main/charts/capi-cr

We will also release a new helm chart for hcloud CSI in the next days in the same mentioned repo.

from cluster-api-provider-hetzner.

@batistein Thanks for the quick fix!
I'll test more features like CSI and additional CNI like Calico and come back here for questions and providing ideas.
cluster-api feels more natural than using terraform.

from cluster-api-provider-hetzner.

@batistein okay, so I could use the helm chart to be more flexible in kinds of worker node names, versioning in git for multiple target hetzner accounts and would avoid the

clusterctl generate ... 

loop.

Can't wait for autoscaling to integrate with cluster-api: kubernetes-sigs/cluster-api#5658

Regarding CSI, Hetzner is mentioning a limit here: https://www.hetzner.com/cloud

With our flexible scalable volumes, each cloud server can be expanded with additional SSD-based storage as needed. Volumes of up to 10TB and up to 16 volumes per cloud server means our volumes are suitable for memory-intensive applications.

Would you recommend to create more smaller nodes to allow more apps to reserve a PersistentVolume?

from cluster-api-provider-hetzner.

@FischerLGLN right this would avoid the clusterctl generate ;)
Autoscaling is already implemented see: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/clusterapi/README.md
For the beginning more smaller nodes later if you have round about 30 nodes it makes more sense to have bigger ones. But only in terms of Volumes and HA. From price perspective that's not the best.

from cluster-api-provider-hetzner.

So Google points here. Hetzner seems to by default set 4 nameservers, that is two nameservers per IPv4 and IPv6 each. This is cleary mis-configuration but it's built into the Ubuntu22 image apparently?
Please fix? Or what am I not getting?

from cluster-api-provider-hetzner.

@MichaelVoelkel this is a mis-configuration in the base image. The above fix could be used in the configuration to avoid the error.

from cluster-api-provider-hetzner.