Elastic IP about cluster-api-provider-packet HOT 6 CLOSED

Update: This is the plan under v1alpha3, and will enable multi-master, but requires proper management of IPs from the Packet API. It is blocked by a Packet API issue, known to the core engineering team as ENG-4090.

from cluster-api-provider-packet.

I am working on this right now. It is not enough to assign an IP via Packet API. We have to configure the network interface to use it.

Let's suppose the elastic IP is: 149.72.10.12 on ubuntu the command should be (this is not the right command even because it gets lost after a reboot):

sudo ip addr add 149.72.10.12 dev lo

How do we do pass that information, and how do we run the command in the target device?

We can pass information tagging a device, and we can resume the tag from inside the server via metadata (or we can figure out a way to get the right ElasticIP directly from the metadata, but it is better to pass it via tag I presume.

We can run the command as part of the preKubeadmCommand configuring the template cluster accordingly.

Problem: It sounds hard to extend across OS and probably a bit flaky as a workflow. I am wondering if we can do something better with the Packet API.

/cc. @deitch

from cluster-api-provider-packet.

preKubeadmCommand makes sense. But, yes, it would need to survive reboots, so rather than applying it to the right interface (public interface, rather than lo), we need to persist it. The way to do that depends on the distro, but is via /etc/network/interfaces in Ubuntu/Debian, or /etc/sysconfig/network-scripts/ifcfg-{ifname} in RedHat/Fedora/CentOS.

You then need to re-up the interface to catch the change. You could add it manually using ip addr add and rely on the config file to catch it on reboot, but that isn't a great idea, as you might not catch an error in the file until much later; better to exercise it right away.

I don't believe there is much we can do with the Packet API. This is an OS thing, and we need the dynamism, e.g. what happens when one host goes down? How does the other pick it up? You need dynamic control.

FWIW, the Packet Elastic IP guide is here and contains examples for multiple OS types.

You might want to think about using the IP with BGP. It would require adding a BGP speaker to each node, which complicates installs, but then you don't have to worry about changing the IP on each node.

from cluster-api-provider-packet.

I don't believe there is much we can do with the Packet API. This is an OS thing, and we need the dynamism, e.g. what happens when one host goes down? How does the other pick it up? You need dynamic control.

This is not part of the scope of this issue but as we discussed worst-case scenario we will have a few goroutines checking the status of the IPs and they will take action accordingly.

I doubt all this provisioning can be done adding lines of bash in the preKubeadmCommand it looks a flaky approach to me, wdyt? The alternative is to provide and distribute a binary for that. Or we have to do what other projects do and pre-build images.

from cluster-api-provider-packet.

we will have a few goroutines checking the status of the IPs and they will take action accordingly

So an agent on each node, that picks it up? I don't know if there is any other way; all VIPs have something to allocate them.

Actually, now that I think about it, given how all of Packet is L3, could you actually add the Elastic IP to all of the nodes (as far as the OS is concerned), and just switch it at the Packet API level? I think that might work. Then you don't need any dynamic switching.

Except that something needs to update Packet API to tell it which server to switch it to, but that can be within the CAPI control loop.

pre-build images

Non-trivial with Packet. There is no, "run this image from this object store" option on Packet. Either you run their stock images, or run an iPXE server, which we shouldn't get into here.

from cluster-api-provider-packet.

This is done

from cluster-api-provider-packet.