cloud-controller-manager is deleting non-hetzner nodes about hcloud-cloud-controller-manager HOT 6 CLOSED

@pierreozoux Hetzner Hybrid setups are indeed a interesting idea and because of our network setup there should be no latency issues.
I will look into this in #9, but it might take a while to clarify how the different integrations might work for hybrid cloud.

from hcloud-cloud-controller-manager.

Hi @xetys,
when adding nodes of a different provider to the cluster you have to remove the --cloud-provider=external flag from the kubelets config so the cloud controller manager is not responsible for managing these nodes.

It might also be possible to run multiple cloud controller managers on the same cluster, but I had no time testing this yet.

from hcloud-cloud-controller-manager.

After I wrote this issue I took a look at this to my self. I see it's not that easy to solve, but it's possible.

The basic idea is still to do this stuff with labeling, and do this while InstanceExistsByProviderID is called. So if we find a server does not exists in hetzner, but never was stated as this, then return true.

The other question is, if this is a good practice. We state then, that instances exist, while they actually do not. That all that ensureNodeExistsByProviderIDorExternalID keep the node in the cluster.

I think I could PR my idea, if you are ok with that approach

from hcloud-cloud-controller-manager.

I think the hcloud-cloud-controller manager already works as expected.

When new kubelets with the --cloud-provider=external flag register themselves into the cluster and no cloud controller manager can find the node, it is removed.
So you either need to run multiple cloud controller managers for each provider or don't specify the flag when there is no cloud controller manager running/configured for the node's provider.

Edit: see post below

from hcloud-cloud-controller-manager.

I did some testing and reading in the kubernetes documentation, this is what my state of this issue is:

• Nodes that cannot be found in the cloud provider are always deleted also when --cloud-provider=external is NOT set.
• A multi provider kubernetes cluster is not supported
• To support a multi cloud/provider kubernetes deployment federation was created

A simple multi cloud kubernetes setup is cool, but right now this is not a use case kubernetes does support and considering the network latency issues, when the kubernetes cluster gets larger, I can see why.

If you really want to support this setup, your approach should work, but I would not merge it into this project.

from hcloud-cloud-controller-manager.

@thetechnick what about hybrid in the sense of Hetzner cloud VM and hetzner baremetal (this is my use case.)

Do you plan to support this kind of hybrid?

from hcloud-cloud-controller-manager.