Comments (12)
Thanks @krsna1729 Yeah, I think the time anomaly is because sn-nuc001 was my first test node, and I have a feeling it did not shutdown/clean/delete cleanly before I ran up the 'full cluster' - so, the master thought it had still been online.
Sure, let me close this. I was expecting to maybe run into some issues wrt accessing prometheus etc. via the first ether nic when the nodes/setup are on the second nic - but I'll open a new Issue if I run into anything.
from cloud-native-setup.
To note. I'm running the setup scripts on a bare metal Clear Linux cluster btw, not in the vagrant VM setup.
from cloud-native-setup.
From https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#initializing-your-master
I probably need to:
--apiserver-advertise-address= argument to kubeadm init
as by default kubeadm
picks the interface with the default gateway on it, and for me that is not the nic with the nodes on.
Do we need to add support to the script command line for this, or are we happy leaving it up to the user to modify to their needs?
from cloud-native-setup.
I'm having trouble getting this to work yet.
I have a feeling maybe the ability to put this in the yaml init file was only added in 1.13.
See:
kubernetes/kubernetes#70761
I've tried adding in the option like:
apiVersion: kubeadm.k8s.io/v1alpha3
kind: InitConfiguration
nodeRegistration:
criSocket: /var/run/crio/crio.sock
#localAPIEndpoint:
#aPIEndpoint:
APIEndpoint:
advertiseAddress: "192.168.xxx.yyy"
bindPort: 6443
---
to the kubeadm.yaml
with both the three different section name types (APIEndpoint
and the two commented out), but no luck - I still end up with the IP address of my primary ether port.
I'm going to see if I can edit my ~/.kube/config
and edit the cluster server IP address there and re-start the script part way through....
Any input/help appreciated here ;-)
from cloud-native-setup.
@grahamwhaley yes, that is a thing that @mcastelino and i have observed and was something that we were discussing as well as that affects my vagrant setup too. we thought of settign it as an env variable that a user can set which the script can read and then add the necessary bits to kubeadm init
.
from cloud-native-setup.
grr, I tried to hack the IP into the kube config, but the cert is encoded to the IP address, so it nacked it....
sigh. So, looking for input on how to fix this in the kubeinit.yaml....
from cloud-native-setup.
apiVersion: kubeadm.k8s.io/v1alpha3
kind: InitConfiguration
nodeRegistration:
criSocket: /var/run/crio/crio.sock
apiEndpoint:
advertiseAddress: "192.52.100.11"
bindPort: 6443
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
featureGates:
RuntimeClass: true
---
apiVersion: kubeadm.k8s.io/v1alpha3
kind: ClusterConfiguration
kubernetesVersion: v1.12.0
controlPlaneEndpoint: "192.52.100.11:6443"
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
apiServerExtraArgs:
advertise-address: "192.52.100.11"
feature-gates: RuntimeClass=true
results in kube config of
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: <>
server: https://192.52.100.11:6443
name: kubernetes
But kubeadm join
from node 2 did not work
[discovery] Trying to connect to API Server "192.52.100.11:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.52.100.11:6443"
[discovery] Failed to request cluster info, will try again: [Get https://192.52.100.11:6443/api/v1/namespaces/kube-public/configmaps/cluster-info: net/http: TLS handshake timeout]
[discovery] Failed to request cluster info, will try again: [Get https://192.52.100.11:6443/api/v1/namespaces/kube-public/configmaps/cluster-info: net/http: TLS handshake timeout]
from cloud-native-setup.
@grahamwhaley i am testing in vagrant. Can you test the above in your setup? Just to make sure, the join failing is not specific to vagrant setup
from cloud-native-setup.
@krsna1729 @mcastelino @ganeshmaharaj ...
thanks @krsna1729 - that worked for me!
kubeadm join 192.168.42.1:6443 --token XXXXXXXX --discovery-token-ca-cert-hash sha256:YYYYYYYYYYYYYYYYYYYYYYY --cri-socket=/run/crio/crio.sock
[preflight] running pre-flight checks
[WARNING RequiredIPVSKernelModulesAvailable]: the IPVS proxier will not be used, because the following required kernel modules are not loaded: [ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack_ipv4] or no builtin kernel ipvs support: map[ip_vs:{} ip_vs_rr:{} ip_vs_wrr:{} ip_vs_sh:{} nf_conntrack_ipv4:{}]
you can solve this problem with following methods:
1. Run 'modprobe -- ' to load missing kernel modules;
2. Provide the missing builtin kernel ipvs support
[discovery] Trying to connect to API Server "192.168.42.1:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.168.42.1:6443"
[discovery] Requesting info from "https://192.168.42.1:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.42.1:6443"
[discovery] Successfully established connection with API Server "192.168.42.1:6443"
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.12" ConfigMap in the kube-system namespace
[kubelet] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[preflight] Activating the kubelet service
[tlsbootstrap] Waiting for the kubelet to perform the TLS Bootstrap...
[patchnode] Uploading the CRI Socket information "/run/crio/crio.sock" to the Node API object "sn-nuc001" as an annotation
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the master to see this node join the cluster.
and then once create_stack.sh
had finished, as the user who ran the script:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
sn-nuc-ctl Ready master 7m48s v1.12.3
sn-nuc001 NotReady <none> 5m39s v1.12.3
OK, so, we should discuss if those missing kernel modules on the node are an issue or if the node status of 'NotReady' is an issue, but things look connected across my second ether port!
from cloud-native-setup.
@grahamwhaley /cc @mcastelino @ganeshmaharaj
[WARNING RequiredIPVSKernelModulesAvailable]
Currently list of modules pertaining to IPVS proxier loaded in clear (master and node) as per https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/README.md.
$ cut -f1 -d " " /proc/modules | grep -e ip_vs -e nf_conntrack
nf_conntrack_netlink
ip_vs_sh
ip_vs_wrr
ip_vs_rr
ip_vs
nf_conntrack
Seems like have we all. NOTE: Our kernel is 4.19.5-297.kvm
and the check is looking for nf_conntrack_ipv4
, which the readme says to use nf_conntrack
instead for 4.19 and above.
This bug has been fixed in 1.13
https://sourcegraph.com/github.com/kubernetes/[email protected]/-/blob/pkg/util/ipvs/kernelcheck_linux.go#L47:17
vs
IPTables
will be used as fallback in 1.12. So we could potentially ignore the warning for now.
NotReady
It is generally because of networking. Can you paste the output of
kubectl get po --all-namespaces -owide
Check if the canal
pod on the second node is running, paste the pod describe output too if possible and there is a config file under /etc/cni/net.d/
.
from cloud-native-setup.
Hi @krsna1729 - I think the NotReady
was transient, and all my nodes are now up and ready. I can post the pod info if you still want, but I don't think it's a real problem. If I see it again and it 'sticks', then I'll re-post.
from cloud-native-setup.
@grahamwhaley ah yes the network pod runs as a daemonset pod which might take time to spin up on the new node added. Suspected it was transient, asked because of 5+ min in the get node output.
Can I close this?
from cloud-native-setup.
Related Issues (20)
- kubeadm yaml needs to be updated to match the version of k8s HOT 1
- setup_system.sh fails when OS version is specified and packages have been removed
- kubeadm displaying warnings HOT 2
- cilium CNI does not work with containerd
- Trying to setup k8s with cri-o and kata-fc runtime on bare metal without Vagrant HOT 3
- consider shallow git clone for create `get()` function
- create: should not print out 'Not-documented' in the help
- rook: fails to come up on single node cluster HOT 4
- Systemd level proxy instead of per service HOT 2
- Problems with no_proxy settings HOT 1
- Add information about Rook to readme
- Rook needs updates
- Does Kubernetes actually work now with Clear Linux? HOT 5
- Missing steps in Vagrant install document HOT 3
- create_stack.sh no longer works
- System level proxy breaks containerd HOT 2
- "swapoff -a" not persistent across reboot HOT 3
- crio runnner is not properly installed with setup_system.sh for automated installation of Kubernetes HOT 1
- vfioveth - POD with DPDK HOT 1
- No bundle update? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cloud-native-setup.