cluster-api-provider-proxmox is a Cluster API infrastructure provider implementation for Proxmox VE.

cluster-api-provider-proxmox provides only infrastructure controller (ProxmoxCluster and ProxmoxMachine). To bootstrap your cluster/machine you need to provide Control Plane provider and Bootstrap provider. For example KubeadmControlPlane and KubeadmBootstrap.

You’ll need a Kubernetes cluster to run against. You can use KIND to get a local cluster for testing, or run against a remote cluster. Note: Your controller will automatically use the current context in your kubeconfig file (i.e. whatever cluster kubectl cluster-info shows).

1. Initialize Management cluster

for more information : https://cluster-api.sigs.k8s.io/user/quick-start.html#initialize-the-management-cluster

# install cluster-api components
export EXP_CLUSTER_RESOURCE_SET=true
clusterctl init --infrastructure=proxmox:v0.2.2 --config https://raw.githubusercontent.com/sp-yduck/cluster-api-provider-proxmox/main/clusterctl.yaml

Note: container images are available at here

2. Create your first workload cluster

# export env variables
export CONTROLPLANE_HOST=X.X.X.X   # for control-plane node
export GATEWAY_ADDRESS=X.X.X.X     # for control-plane node
export NAMESERVER_ADDRESS=X.X.X.X  # for contro-lplane node
export PROXMOX_URL=X.X.X.X:8006
export PROXMOX_PASSWORD=password
export PROXMOX_USER=user@pam
export NODE_URL=node.ssh.url:22
export NODE_USER=node-ssh-user
export NODE_PASSWORD=node-ssh-password

# generate manifests
clusterctl generate cluster cappx-test --infrastructure=proxmox:v0.2.2 --config https://raw.githubusercontent.com/sp-yduck/cluster-api-provider-proxmox/main/clusterctl.yaml > cappx-test.yaml

# inspect and edit
vi cappx-test.yaml

# apply manifests
kubectl apply -f cappx-test.yaml

3. Access your first workload cluster !!

Usually it takes 2~10 mins to complete bootstrapping the nodes.

# get workload cluster's kubeconfig
clusterctl get kubeconfig cappx-test > kubeconfig.yaml

# get node command for workload cluster
kubectl --kubeconfig=kubeconfig.yaml get node

4. Tear down your workload cluster

kubectl delete cluster cappx-test

• No need to prepare vm templates. You can specify any vm image in ProxmoxMachine.Spec.Image.

• Supports custom cloud-config (user data). CAPPX uses ssh for bootstrapping nodes so it can applies custom cloud-config that can not be achieved by only Proxmox API.

CAPPX is compatible with qcow2 image. You can build your own node image and use it for ProxmoxMachine.

CAPPX relies on a few prerequisites which have to be already installed in the used operating system images, e.g. a container runtime, kubelet, kubeadm,.. .

To build your custom node image, you can use kubernetes-sigs/image-builder project.

Also there are some available out-of-box images published other communities such as Metal3. For example https://artifactory.nordix.org/ui/native/metal3/images/. Example MD can be found metal3-ubuntu2204-k8s127.yaml.

If it isn't possible to pre-install those prerequisites in the image, you can always deploy and execute some custom scripts through the ProxmoxMachine.spec.cloudInit or KubeadmConfig.spec.preKubeadmCommands etc. . Example MD can be found ubuntu2204.yaml.

CAPPX is tested with pve-manager/7.4-3/9002ab8a (running kernel: 5.15.102-1-pve).

CAPPX is tested with KubeadmControlPlane and KubeadmBootstrap.

This project aims to follow the Cluster API Provider contract.

Because Proxmox-VE does not provide LBaaS solution, CAPPX does not follow the typical infra-cluster logic. ProxmoxCluster controller reconciles only Proxmox storages used for instances. You need to prepare control plane load balancer by yourself if you creates HA control plane workload cluster.

ProxmoxMachine controller follows the typical infra-machine logic. To bootstrap your machine, CAPPX supports only cloud-config type bootstrap data secret. CAPPX is mainly tested with KubeadmControlPlane and KubeadmBootstrap.

Are you interested in contributing to cluster-api-provider-proxmox? Do not hesitate to open GitHub issues.

Copyright 2023 Teppei Sudo.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.