Load balancer with failover IP about hcloud-cloud-controller-manager HOT 22 CLOSED

Transferring FloatingIPs with health checking will not be part of this project, but this can be provided via an additional application.

I will pass this on to the Hetzner Cloud Product Management.

from hcloud-cloud-controller-manager.

When you guys have Floating IPs, you can actually use metallb with L2 and it will be correctly working as a K8s LoadBalancer.

from hcloud-cloud-controller-manager.

Hi all,
I created a for of https://github.com/kubernetes/contrib/tree/master/keepalived-vip and modified it in a way that it can use notify scripts.
I wanted did this to solve exactly this problem. I already tested it in one of my development clusters at hetzner and it seems to be working.
I commited the example resources I used so you can try it.
This is work in progress, and I would apreciate any feedback and testing.
Resource files are here: https://github.com/cornelius-keller/contrib/tree/master/keepalived-vip/notify-example-hetzner

from hcloud-cloud-controller-manager.

We had no problem with metallb we just pass addresses as

data:
  config: |
    address-pools:
    - name: ip-space
      protocol: layer2
      addresses:
      - 111.111.111.1/32

using it only with one IP right now, though.

edit: But a hetzer k8s loadbalancer would be awesome. Issuing new floating ips as needed (and attaching it to nodes)

@fentas did you do any extra configuration? I am not able to access my service w / metallb It seems that the service is assigned the external ip but is not accessible from outside

from hcloud-cloud-controller-manager.

@schmitch I've looked into metallb and didn't consider it for my cluster since it relies on DHCP according the docs. How did you get it to work with Hetzner Cloud's floating ips? I can't find any information on how to use it in combination with the Hetzner Cloud API to dynamically assign the floating ip.

from hcloud-cloud-controller-manager.

@vitobotta I've hit the same wall now. metallb only responds to ARP requests on the node it assigned to the LoadBalander service's IP. It's working in combination with https://github.com/cbeneke/hcloud-fip-controller if you also configure all floating IPs as secondary IPs for each node. But that should actually not be necessary as metallb does ARP.

So you either need to modify metallb to issue ARP replies for all LB IPs on all nodes or have some tool always assign the floating IP to the node metallb elected. hcloud-fip-controller has its own independent election unfortunately.

from hcloud-cloud-controller-manager.

Yes, the CNI might make a difference. Also my goal is to rely solely on metallb's ARP implementation instead of configuring the FIP on the node directly.

from hcloud-cloud-controller-manager.

Hi @pierreozoux,
I looked into this and this feature would be out of the scope of this tool to provide and would not work well without additional components for healthchecking the nodes.
Hetzner Failover IPs can only be assigned to bare metal servers and this tool only operates on Hetzner Cloud resources.

FloatingIPs via a separate tool can be used to implement this via service external ips and its own port keepalive check.

from hcloud-cloud-controller-manager.

And what about FloatingIPs support?
(If yes, then I'll open a new issue)

from hcloud-cloud-controller-manager.

also commented on xetys/hetzner-kube#58 . I guess that is the better place .

from hcloud-cloud-controller-manager.

I'm using Azure Load Balancer (https://azure.microsoft.com/en-us/services/load-balancer/) and it's working very good. Health check is every 2 seconds for specified port. I'm using it for SMTP servers load balancing and if I restart one server when updating, service is still available. If I can use the same load balancer with health checks in Hetzner Cloud, I will migrate (cheaper price).

from hcloud-cloud-controller-manager.

@niklaskorz @schmitch Same here, didn't work for me. I set up MetalLB with L2, provided a floating IP from Hetzner but it just didn't work.

from hcloud-cloud-controller-manager.

We had no problem with metallb we just pass addresses as

data:
  config: |
    address-pools:
    - name: ip-space
      protocol: layer2
      addresses:
      - 111.111.111.1/32

using it only with one IP right now, though.

edit: But a hetzer k8s loadbalancer would be awesome. Issuing new floating ips as needed (and attaching it to nodes)

from hcloud-cloud-controller-manager.

We had no problem with metallb we just pass addresses as

data:
  config: |
    address-pools:
    - name: ip-space
      protocol: layer2
      addresses:
      - 111.111.111.1/32

using it only with one IP right now, though.

edit: But a hetzer k8s loadbalancer would be awesome. Issuing new floating ips as needed (and attaching it to nodes)

Can we use pool of floating IPs and services would get assigned to floating IPs from this pool?
Do we need to assign/associate these floating IPs to some worker node as well?

from hcloud-cloud-controller-manager.

Hi @fentas I have followed your suggestion and installed metallb and nginx ingress controller. It did create the load balancer with the floating IP I specified, but it didn't auto assign the floating IP to a node. So I assigned it to a node manually but when I visit the IP in the browser it hangs forever instead of loading nginx's default backend. Any idea of what I could have done wrong? Thanks!

from hcloud-cloud-controller-manager.

@tckb Hi, did you find a solution? I'm having the same problem

from hcloud-cloud-controller-manager.

Interesting! If I assign the floating IP to the node where the metallb controller is running then it works! But that means that I have to manually fix if the controller pod is rescheduled on another node. How to avoid that?

from hcloud-cloud-controller-manager.

Weird, it doesn't matter if the metallb controller pod is on the same node I assign the floating IP to. However the thing only works if I assign the floating IP to a specific node and not with the other two nodes of the cluster. What could be causing this?

from hcloud-cloud-controller-manager.

@mstarostik I am not sure if it's not necessary in theory, but the only way I got it working with my current setup is to ensure the network interface on all the nodes is configured with the floating IPs. So far I haven't had any problems this way.

from hcloud-cloud-controller-manager.

@vitobotta just trying to avoid this. Makes changes to the FIPs more intrusive as it feeds back into the node config. And with k3OS tweaking configs like this is deliberately uncomfortable.

from hcloud-cloud-controller-manager.

Sorry for not responding. Quite bussy.
We have one floating ip which we configure to one node (ansible) and configure metallb as described above. This worked for multible provisionings up to now.
This node becomes a single point of failure for sure but for now this is ok. So a operator making sure the floating ip is attached to a node would be nice.

Never had the problem that metallb had to be placed on the node where the floating ip is configured. Maybe this is an issue within the CNI? (or kube-proxy)
We use cilium - maybe this makes the difference?

from hcloud-cloud-controller-manager.

Any news or better approach to automatically reassign floating IPs to the elected node to run MetalLB?

from hcloud-cloud-controller-manager.