add node failed
logs:
[Info] 2022-06-16 16:58:14.38258995 +0800 CST m=+1655.018892666 Adding kubernetes node, please wait ...
Kubernetes Cluster Label: K8s-Cluster1
Kubernetes Node: 8.8.88.86
SSH Operation Port: 22
Operating System Type: rhel7
Automatically Upgrade OS Kernel: NO
System User for Operation: root
PLAY [addnode] *****************************************************************
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/genfile : 0.Distributing deployment files to target host, please wait...] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/genfile : file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/genfile : copy] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/genfile : copy] ***
changed: [8.8.88.86]
PLAY [addnode] *****************************************************************
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 1.Create base package directory] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : file] ***
ok: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 2.Perform installation pretreatment] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : shell] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 3.Turn off firewall] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : service] ***
fatal: [8.8.88.86]: FAILED! => {"changed": false, "msg": "Could not find the requested service ufw: host"}
...ignoring
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : service] ***
fatal: [8.8.88.86]: FAILED! => {"changed": false, "msg": "Could not find the requested service SuSEfirewall2: host"}
...ignoring
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 4.Turn off selinux] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 5.Turn off postfix] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 6.Set system limit] ***
changed: [8.8.88.86] => (item=* soft nofile 65536)
changed: [8.8.88.86] => (item=* hard nofile 65536)
changed: [8.8.88.86] => (item=* soft nproc 65536)
changed: [8.8.88.86] => (item=* hard nproc 65536)
changed: [8.8.88.86] => (item=* soft memlock unlimited)
changed: [8.8.88.86] => (item=* hard memlock unlimited)
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 7.Set ipvs_modules] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 8.Run ipvs_modules] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 9.Add sysctl parameter configuration] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 10.Set sysctl parameter configuration] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 11.Create journal directory] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 12.Create journal.conf.d directory] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 13.Add prophet file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 14.Start systemd-journald service] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 15.Install TTY software] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : copy] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 16.Create resolv config file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 17.Create k8s cert directory] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 18.Decompress k8s software package] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 19.Decompress cfssl software package] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/all : 20.Decompress helm software package] ***
changed: [8.8.88.86]
PLAY [addnode] *****************************************************************
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 1.Cleaning up garbage files left in history.] ***
fatal: [8.8.88.86]: FAILED! => {"changed": false, "msg": "Could not find the requested service kubelet: host"}
...ignoring
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : service] ***
fatal: [8.8.88.86]: FAILED! => {"changed": false, "msg": "Could not find the requested service kube-proxy: host"}
...ignoring
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : service] ***
fatal: [8.8.88.86]: FAILED! => {"changed": false, "msg": "Could not find the requested service containerd: host"}
...ignoring
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : shell] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
ok: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
ok: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
ok: [8.8.88.86] => (item=containerd)
ok: [8.8.88.86] => (item=containerd-shim)
ok: [8.8.88.86] => (item=containerd-shim-runc-v1)
ok: [8.8.88.86] => (item=containerd-shim-runc-v2)
ok: [8.8.88.86] => (item=nerdctl)
ok: [8.8.88.86] => (item=ctr)
ok: [8.8.88.86] => (item=runc)
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
ok: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
ok: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
ok: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
ok: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
ok: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 2.Create CNI plug-in directory] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 3.Decompress CNI plug-in package] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : unarchive] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 4.Create runtime config dir] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 5.Decompress runtime software package] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : template] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 6.Create containerd service] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 7.Create ipvsinit service] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 8.Create ipvsinit shell] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 9.Enable ipvsinit service] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 10.Start runtime service] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 11.Decompress runtime images package] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 12.Load runtime images] ***
changed: [8.8.88.86] => (item=metrics-server-amd64.tar)
changed: [8.8.88.86] => (item=metrics-scraper.tar)
changed: [8.8.88.86] => (item=heapster-amd64.tar)
changed: [8.8.88.86] => (item=pause-amd64.tar)
changed: [8.8.88.86] => (item=traefik.tar)
changed: [8.8.88.86] => (item=alpine.tar)
changed: [8.8.88.86] => (item=kubernetes-dashboard-v2.4.0-amd64.tar)
changed: [8.8.88.86] => (item=coredns.tar)
changed: [8.8.88.86] => (item=registry.tar)
changed: [8.8.88.86] => (item=calico-cni.tar)
changed: [8.8.88.86] => (item=calico-kube-controllers.tar)
changed: [8.8.88.86] => (item=calico-node.tar)
changed: [8.8.88.86] => (item=calico-pod2daemon-flexvol.tar)
changed: [8.8.88.86] => (item=flannel-cni-plugin.tar)
changed: [8.8.88.86] => (item=flannel.tar)
changed: [8.8.88.86] => (item=kube-router.tar)
changed: [8.8.88.86] => (item=weave-kube.tar)
changed: [8.8.88.86] => (item=weave-npc.tar)
changed: [8.8.88.86] => (item=cilium.tar)
changed: [8.8.88.86] => (item=cilium-operator-generic.tar)
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : 13.Set runtime images tag and create pushsysimages shell] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000base/runtime : shell] ***
changed: [8.8.88.86] => (item=metrics-server-amd64:v0.5.0)
changed: [8.8.88.86] => (item=metrics-scraper:v1.0.7)
changed: [8.8.88.86] => (item=heapster-amd64:v1.5.4)
changed: [8.8.88.86] => (item=pause-amd64:3.5)
changed: [8.8.88.86] => (item=traefik:v2.0.7)
changed: [8.8.88.86] => (item=alpine:3.6)
changed: [8.8.88.86] => (item=kubernetes-dashboard-amd64:v2.4.0)
changed: [8.8.88.86] => (item=coredns:1.3.1)
changed: [8.8.88.86] => (item=registry:v2.7.1)
changed: [8.8.88.86] => (item=calico-node:v3.19.3)
changed: [8.8.88.86] => (item=calico-pod2daemon-flexvol:v3.19.3)
changed: [8.8.88.86] => (item=calico-cni:v3.19.3)
changed: [8.8.88.86] => (item=calico-kube-controllers:v3.19.3)
changed: [8.8.88.86] => (item=flannel-cni-plugin:v1.0.0)
changed: [8.8.88.86] => (item=flannel:v0.15.1)
changed: [8.8.88.86] => (item=kube-router:v1.3.2)
changed: [8.8.88.86] => (item=weave-kube:2.8.1)
changed: [8.8.88.86] => (item=weave-npc:2.8.1)
changed: [8.8.88.86] => (item=cilium:v1.9.0)
changed: [8.8.88.86] => (item=cilium-operator-generic:v1.9.0)
PLAY [addnode] *****************************************************************
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kubelet : 1.Distribution kubelet cert] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kubelet : 2.Clean up invalid certificates] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kubelet : 3.Copy kubelet config and shell file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kubelet : copy] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kubelet : 4.Set IPVS rules] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kubelet : 5.Create /home/kube-install/k8s/kubernetes directory] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kubelet : file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kubelet : 6.Create kubelet service startup file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kubelet : 7.Start kubelet service] ***
changed: [8.8.88.86]
PLAY [addnode] *****************************************************************
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kube-proxy : 1.Create kube-proxy.config.yaml configuration] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kube-proxy : 2.Create /home/kube-install/k8s/kubernetes/kube-proxy directory] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kube-proxy : 3.Create kube-proxy service startup file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/kube-proxy : 4.Start kube-proxy service] ***
changed: [8.8.88.86]
PLAY [master1] *****************************************************************
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : 1.Enable csr-crb configuration] ***
changed: [8.8.88.81]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : shell] ***
changed: [8.8.88.81]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : shell] ***
changed: [8.8.88.81]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : shell] ***
changed: [8.8.88.81]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : shell] ***
changed: [8.8.88.81]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : 2.Wait 90s] ***
ok: [8.8.88.81]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : 3.Approve node csr] ***
fatal: [8.8.88.81]: FAILED! => {"changed": true, "cmd": "/usr/local/bin/kubectl get csr | awk '/Pending/ {print $1}' | xargs kubectl certificate approve", "delta": "0:00:00.118209", "end": "2022-06-16 17:06:44.673905", "msg": "non-zero return code", "rc": 123, "start": "2022-06-16 17:06:44.555696", "stderr": "error: one or more CSRs must be specified as or -f ", "stderr_lines": ["error: one or more CSRs must be specified as or -f "], "stdout": "", "stdout_lines": []}
...ignoring
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : 4.Wait 30s] ***
ok: [8.8.88.81]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : 5. Setting scheduling label for k8s node] ***
changed: [8.8.88.81]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : shell] ***
changed: [8.8.88.81]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x0000000000node/approve-csr : shell] ***
changed: [8.8.88.81]
PLAY [addnode] *****************************************************************
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x00000000finish/addnode : Create reboot config file] ***
changed: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x00000000finish/addnode : Congratulations, node successfully joined kubernetes cluster! "The operating system will automatically restart to take effect on the cluster configuration."] ***
ok: [8.8.88.86]
TASK [/root/kube-install/data/output/K8s-Cluster1/sys/0x00000000finish/addnode : shell] ***
changed: [8.8.88.86]
PLAY RECAP *********************************************************************
8.8.88.81 : ok=11 changed=9 unreachable=0 failed=0 skipped=0 rescued=0 ignored=1
8.8.88.86 : ok=77 changed=62 unreachable=0 failed=0 skipped=0 rescued=0 ignored=5
[Error] 2022-06-16 17:13:19.840843904 +0800 CST m=+2560.477146626 Kubernetes node add failed!