Giter Club home page Giter Club logo

karmada-operator1's Introduction

Karmada-Operator

A Karmada Operator based on the K8s Operator model built by DaoCloud, Karmada Operator(Kubernetes Armada operator) simplifies the deployment, operation and maintenance of Karmada.

TL;DR

Switch to the root directory of the repo.

helm install karmada-operator -n karmada-operator-system --create-namespace --dependency-update ./charts/karmada-operator

Prerequisites

  • Kubernetes 1.16+
  • helm v3+

Demo

screenshot_gif

Installing the Chart

To install the chart with the release name karmada-operator in namespace karmada-operator-system:

  • local installation

Switch to the root directory of the repo.

helm install karmada-operator -n karmada-operator-system --create-namespace --dependency-update ./charts/karmada-operator

Tip: If you need to modify values.yaml, please refer to Configuration

  • remote installation

First, add the Karmada-Operator chart repo to your local repository.

$ helm repo add karmada https://release.daocloud.io/chartrepo/karmada
$ helm repo list
NAME            URL
karmada   https://release.daocloud.io/chartrepo/karmada

With the repo added, available charts and versions can be viewed.

$ helm search repo karmada
NAME                    	CHART VERSION	APP VERSION	DESCRIPTION
karmada/karmada-operator	0.0.5        	0.0.5      	A Helm chart for karmada-operator

Install the chart and specify the version to install with the --version argument. Replace --version=<x.x.x> with your desired version. such as the above karmada/karmada-operator chart version 0.0.5 , so you can use --version=0.0.5.

$ helm --namespace karmada-operator-system upgrade -i karmada-operator karmada/karmada-operator --version=<x.x.x> --create-namespace
Release "karmada-operator" does not exist. Installing it now.
NAME: karmada-operator
LAST DEPLOYED: Thu Aug 11 15:24:01 2022
NAMESPACE: karmada-operator-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing karmada-operator.

Your release is named karmada-operator.

To learn more about the release, try:

  $ helm status karmada-operator -n karmada-operator-system
  $ helm get all karmada-operator -n karmada-operator-system
  • example there is an example to install a karmada instance on current cluster.

Tip: List all releases using helm list

Create a KarmadaDeployment

After installing karmada-operator, use cr to create karmadaDeployment

Create cr using below command in the cluster where the karmada-operator install

Tip:

  • If no secretRef is specified in cr, the karmada instance is installed in the cluster where karmada-operator is located
  • If no namespace is specified in cr, the default install namespace is the same as the karmadaDeployment name
#deploy KarmadaDeployment demo-kmd in the cluster where the karmada-operator install 
$ kubectl apply -f https://github.com/DaoCloud/karmada-operator/blob/main/examples/karmadaDeployment.yaml

The process of deploying karmada may take a few minutes, check the operation of karmada's components

$ kubectl -n demo-kmd get pod
NAME                                                        READY   STATUS    RESTARTS      AGE
etcd-0                                                      1/1     Running   0             19m
karmada-demo-kmd-aggregated-apiserver-6fd745748-hjbq7       1/1     Running   0             19m
karmada-demo-kmd-apiserver-429bfb7c1-6fkrk                  1/1     Running   0             19m
karmada-demo-kmd-controller-manager-7b8f5699c-mvx8s         1/1     Running   3 (21m ago)   21m
karmada-demo-kmd-controller-manager-7b8f5699c-7fc8c         1/1     Running   2 (19m ago)   19m
karmada-demo-kmd-kube-controller-manager-9b56855fdc-699c5   1/1     Running   0             19m
karmada-demo-kmd-scheduler-685b997cb7-9467w                 1/1     Running   0             19m
karmada-demo-kmd-scheduler-685b997cb7-85c84                 1/1     Running   0             20m
karmada-demo-kmd-webhook-85c84d84c7-g6t9n                   1/1     Running   0             19m

ok, karmada installation is complete.

Create multiple KarmadaDeployments

karmada-operator supports creating multiple karmadaDeployment instances in the same cluster,You can deploy multiple karmada instances multiple times using the cr of KarmadaDeployment with different content.

List multiple KarmadaDeployments：

$ kubectl get kmd
NAME         MODE   PHASE       STATUS   AGE
demo-kmd     Helm   Completed    true    2m42s
demo-kmd-1   Helm   Completed    true    2m57s
$ kubectl get pod -n demo-kmd-1
NAME                                                        	READY   STATUS    RESTARTS      AGE
etcd-0                                                      	1/1     Running   0             14m
karmada-demo-kmd-1-aggregated-apiserver-9b56855fd-gwrmf         1/1     Running   0             14m
karmada-demo-kmd-1-apiserver-6f9bbcb9c-vmxbq                    1/1     Running   0             14m
karmada-demo-kmd-1-controller-manager-5d6777b75-7rmq8           1/1     Running   2 (17m ago)   17m
karmada-demo-kmd-1-controller-manager-5d6777b75-g6xzd           1/1     Running   2 (16m ago)   16m
karmada-demo-kmd-1-kube-controller-manager-76b94fb97c-6lqwv     1/1     Running   0             14m
karmada-demo-kmd-1-scheduler-797b7b787b-klcs4                   1/1     Running   0             14m
karmada-demo-kmd-1-scheduler-797b7b787b-r829t                   1/1     Running   0             15m
karmada-demo-kmd-1-webhook-755d649cb4-wbx5h                     1/1     Running   0             15m
$ kubectl get pod -n demo-kmd
NAME                                                        READY   STATUS    RESTARTS      AGE
etcd-0                                                      1/1     Running   0             19m
karmada-demo-kmd-aggregated-apiserver-6fd745748-hjbq7       1/1     Running   0             19m
karmada-demo-kmd-apiserver-429bfb7c1-6fkrk                  1/1     Running   0             19m
karmada-demo-kmd-controller-manager-7b8f5699c-mvx8s         1/1     Running   3 (21m ago)   21m
karmada-demo-kmd-controller-manager-7b8f5699c-7fc8c         1/1     Running   2 (19m ago)   19m
karmada-demo-kmd-kube-controller-manager-9b56855fdc-699c5   1/1     Running   0             19m
karmada-demo-kmd-scheduler-685b997cb7-9467w                 1/1     Running   0             19m
karmada-demo-kmd-scheduler-685b997cb7-85c84                 1/1     Running   0             20m
karmada-demo-kmd-webhook-85c84d84c7-g6t9n                   1/1     Running   0             19m

Export Karmada KubeConfig

You can get karmada kubeconfig to view karmada instance information

$ KARMADA_CONFIG=$(kubectl get secret  -n ${your_karmada_operator_namespace} $(kubectl get kmd demo-kmd -o jsonpath='{.status.secretRef.name}') -o jsonpath='{.data.kubeconfig}')
$ echo $KARMADA_CONFIG | base64 -d > karmada-apiserver.config

You can export KUBECONFIG for easy login the karmada instance

export KUBECONFIG=./karmada-apiserver.config

We can simply test karmada-apiserver kubeconfig ,such as List namespaces

$ kubectl --kubeconfig karmada-apiserver.config get ns
NAME              STATUS   AGE
default           Active   124m
demo-kmd          Active   89m
karmada-cluster   Active   89m
kube-node-lease   Active   124m
kube-public       Active   124m
kube-system       Active   124m

Simple Karmada Demo

We can test karmada's multi-cloud capabilities with a simple demo

Join Clusters

1. Use kubectl karmada join cluster to Karmada.

Currently you need to download karmadactl to join cluster

Karmada provides kubectl-karmada plug-in download service since v1.2.1. You can choose proper plug-in version which fits your operator system form karmada release.

Take v1.2.1 that working with linux-arm64 os as an example:

$ wget https://github.com/karmada-io/karmada/releases/download/v1.2.1/karmadactl-linux-arm64.tgz

$ tar -zxf karmadactl-linux-arm64.tgz

$ cp karmadactl /usr/local/bin

then we join member1 and member2 cluster into karmada

--cluster-kubeconfig is your members kubeconfig.

--kubeconfig is your karmada apiserver kubeconfig

$ karmadactl --kubeconfig ${your_karmada_apiserver_config} join member1 --cluster-kubeconfig=${your_member1_kubeconfig}
$ karmadactl --kubeconfig ${your_karmada_apiserver_config} join member2 --cluster-kubeconfig=${your_member2_kubeconfig}

2. Show members of karmada

$ kubectl  --kubeconfig ${your_karmada_apiserver_config} get clusters
NAME         VERSION    MODE   READY   AGE
member1      v1.21.12   Push   True    2m53s
member2      v1.22.6    Push   True    21s

Propagate a deployment by Karmada

1. Create nginx deployment in Karmada

First, create a deployment named nginx:

# create a deployment name nginx
$ kubectl create -f https://github.com/DaoCloud/karmada-operator/blob/main/examples/nginx/deployment.yaml --kubeconfig ${your_karmada_apiserver_config}

2. Create PropagationPolicy that will propagate nginx to member cluster

Then, we need to create a policy to propagate the deployment to our member cluster.

# create a PropagationPolicy 
$ kubectl create -f https://github.com/DaoCloud/karmada-operator/blob/main/examples/nginx/propagationpolicy.yaml --kubeconfig ${your_karmada_apiserver_config}

3. Check the deployment status from Karmada

You can check deployment status from Karmada, don't need to access member cluster:

$ kubectl get deployment --kubeconfig ${your_karmada_apiserver_config}
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   2/2     2            2           20s

Configuration

Global parameters

Name Description Value
global.imageRegistry Global Docker image registry ""
global.imagePullSecrets Global Docker registry secret names as an array []

Common parameters

Name Description Value
controllerManager.labels deployment labels of the karmada-operator-controller-manager "app: karmada-operator-controller-manager"
controllerManager.replicaCount replicas of the karmada-operator-controller-manager deployment 2
controllerManager.podAnnotations pod Annotations of the karmada-operator-controller-manager {}
controllerManager.podLabels pod Labels of the karmada-operator-controller-manager {}
controllerManager.image.registry Image registry of the karmada-operator-controller-manager release.daocloud.io
controllerManager.image.repository Image repository of the karmada-operator-controller-manager kairship/karmada-operator-controller-manager
controllerManager.image.tag Image tag of the karmada-operator-controller-manager "v0.0.1"
controllerManager.image.pullPolicy Image registry of the karmada-operator-controller-manager "IfNotPresent"
controllerManager.image.pullSecrets Image pullSecrets of the karmada-operator-controller-manager []
controllerManager.resources resources of the karmada-operator-controller-manager {}
controllerManager.nodeSelector nodeSelector of the karmada-operator-controller-manager {}
controllerManager.affinity affinity of the karmada-operator-controller-manager {}
controllerManager.tolerations tolerations of the karmada-operator-controller-manager key: node-role.kubernetes.io/master operator: Exists
controllerManager.kubeconfigPath kubeconfig Path of the karmada-operator-controller-manager "/root/.kube"
controllerManager.localKubeconfig Image registry of the karmada-operator-controller-manager "true"

Uninstalling

Uninstalling the KarmadaDeployment

Deleting karmadaDeployment is a dangerous operation and needs to be deleted carefully. Please delete it according to the following process.

Only Delete KarmadaDeployment CR

When you want to remove CR but keep the instance on the environment，you can execute the following command

# patch disable-cascading-deletion label
$ kubectl label --overwrite kmd demo-kmd karmada.install.io/disable-cascading-deletion="true"
# disable cascade delete KarmadaDeployment
$ kubectl delete kmd demo-kmd -n demo-kmd

Delete KarmadaDeployment CR and Instance

When you want to remove CR and the instance in your environment，you can execute the following command

# cascade delete KarmadaDeployment
$ kubectl delete kmd demo-kmd -n demo-kmd

Uninstalling the Karmada-Operator

To uninstall/delete the karmada-operator helm release in namespace karmada-operator-system:

helm uninstall karmada-operator -n karmada-operator-system

The command removes all the Kubernetes components associated with the chart and deletes the release.

kubectl delete ns karmada-operator-system

karmada-operator1's People

Contributors

calvin0327 avatar carlory avatar chaunceyjiang avatar lixingchendaocloud avatar wawa0210 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar

Forkers

calvin0327 githubxsy chaunceyjiang lixingchendaocloud

karmada-operator1's Issues

gather overview information of the member cluster to karmadaDeployment

After deployed a karmada instance, We need to gather overview of the common resources in the member cluster.
This information can give us a very intuitive understanding of member cluster resources.

  1. Include Namespace, Node, Deployment, Pod, Service, Secret
  2. count of OP and PP
  3. member cluster healthy.

When installing the second karmada instance, an error port conflict is reported

Second karamda instance installation failed because of port conflict.

error message

{"component":"helm","info":"purge requested for karmada-demo-kmd2","release":"karmada-demo-kmd2","targetNamespace":"karmada-system1","version":"v3"}
E0809 07:14:40.251753       1 install.go:226] [helm-installer]:failed to install karmada chart for demo-kmd2: release karmada-demo-kmd2 failed, and has been uninstalled due to atomic being set: Service "karmada-demo-kmd2-apiserver" is invalid: spec.ports[0].nodePort: Invalid value: 32443: provided port is already allocated
E0809 07:14:40.251867       1 installer_controller.go:158] "Failed to reconcile karmadaDeployment" err="release karmada-demo-kmd2 failed, and has been uninstalled due to atomic being set: Service \"karmada-demo-kmd2-apiserver\" is invalid: spec.ports[0].nodePort: Invalid value: 32443: provided port is already allocated" cluster="demo-kmd2" num requeues=15

about ETCD spec

  • current spec is
type ETCD struct {
	// TODO: support external etcd.
	// StorageMode etcd data storage mode(emptyDir,hostPath, PVC), default "emptyDir"
	// +optional
	StorageMode string `json:"storageMode,omitempty"`

	// If specified StorageMode is PVC, StorageClass must be set.
	// +optional
	StorageClass string `json:"storageClass,omitempty"`

	// If specified StorageMode is PVC, Size must be set.
	// +optional
	Size string `json:"size,omitempty"`
}
  • advice spec is
type ETCD struct {
	// Storage about etcd data storage config
	// +optional
	Storage corev1.Volume `json:"storage,omitempty"`

	// Servers is config external etcd
	// +optional
	Servers string `json:"servers,omitempty"`
}

karmada operator retry mechanism

after the kmd be reconcile 5 times, the cr will not be requeue. in some case example the network environment is not very stable, we hope there is a retry mechanism to keep the kmd always be reconcile util the karmada is running.

whether to delete etcd data when deleting Karmada instance

when install karmada instance with hostPath to karmada etcd. we not delete the data of ectd currently when delete karmada instance. if we create a new instance named the same string with previous instance. there are dirty data on new karmada instance.

so, we should designe a optional to handle whether to delete etcd data. wolcome to discussion

Unable to use sample to create karmada

I wanted to use the sample to create a karmada, but I got some errors

image

kubectl apply -f https://raw.githubusercontent.com/DaoCloud/karmada-operator/main/examples/karmadaDeployment.yaml

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.