chukaofili / kubernetes-on-digitalocean Goto Github PK

I am not sure if you can assist here, but I'm getting desperate! I have followed your guide and it works flawlessly - the only issue is when I try to connect to the cluster through the load balancer, I always get 503 Service Unavailable - even though the Load Balancer in Digital Ocean is showing as Healthy.... I am stumped....

Hi Chuka,

First, thanks for this tutorial. I follow it and it really helps. And sorry for opening issue - this is not an issue, I just didn't know how to communicate differently.

I'm imitating your initial 'kubernetes on digitalocean' setup, only i would like to make a different nodes configuration. I would like to create 3 environments in DO, test with 1 node, stage with 1 node, and prod with 2 nodes + load balancer (more is not needed for now, just want to add some HA to the current setup and to make sure i could easily scale in future). Thing is, in this setup I would like all my nodes to be both masters and to host pods.

So, for example, I would like to build a cluster to:

• Have two nodes that are both masters and both can host pods
• Have one LB
• Cluster works when any of nodes is down (single node)
• Local kubectl works when any of nodes is down (single node)

I was trying to create such a setup by imitating your step for creating master, then imitating your step for creating one worker, and then doing kubectl --kubeconfig=${CLUSTER_KUBECTL_CONFIG} patch node ${WORKER_NODE} -p '{"metadata":{"labels":{"node-role.kubernetes.io/master":""}},"spec":{"taints":[{"effect":"NoSchedule","key":"node-role.kubernetes.io/master","timeAdded":null}]}} and kubectl --kubeconfig=${CLUSTER_KUBECTL_CONFIG} --all node-role.kubernetes.io/master-. It seems that this setup doesn't actually helps - kubectl --kubeconfig=${CLUSTER_KUBECTL_CONFIG} get nodes says that I have two masters, when I put second master (the one created as worker) down everything still works (local kubectl and cluster is up), but when I put initial master (the one created as master) down I cannot connect to the cluster anymore via kubectl. Changing IP in kubectl config to second node doesn't help. Error is The connection to the server 159.89.27.117:6443 was refused - did you specify the right host or port?.

So I guess my setup for second master is flaky and that I should run some other commands, but googling didn't help. It would be super-nice if you would know what I'm doing wrong and help me with advice.

Thanks, Aleksandar

Most of my pods have the following error in the log file:

Get https://<cluster-ip>:<port>/containerLogs/kube-system/digitalocean-flexplugin-deploy-47d5g/flex-deploy?tailLines=5000&amp;timestamps=true: dial tcp <cluster-ip>:<port>: connect: no route to host

I am seeing this is pods like: digitalocean-flexplugin-deploy, digitalocean-provisioner, monitoring-grafana and even the busy-pod.

Running: kubectl get po -n kube-system returns:

NAME                                        READY     STATUS    RESTARTS   AGE
coredns-78fcdf6894-lpnkg                    1/1       Running   0          1d
coredns-78fcdf6894-n8bbp                    1/1       Running   0          1d
digitalocean-flexplugin-deploy-47d5g        1/1       Running   0          1d
digitalocean-flexplugin-deploy-7dchm        1/1       Running   0          1d
digitalocean-flexplugin-deploy-jnflb        1/1       Running   0          1d
digitalocean-provisioner-6dc747b88d-hh7b5   1/1       Running   0          1d
digitalocean-provisioner-6dc747b88d-nn49l   1/1       Running   0          1d
etcd-k8s-master-node                        1/1       Running   0          1d
heapster-6cc56b8ff9-gzmvx                   1/1       Running   0          1d
kube-apiserver-k8s-master-node              1/1       Running   0          1d
kube-controller-manager-k8s-master-node     1/1       Running   0          1d
kube-flannel-ds-gpbbx                       1/1       Running   0          1d
kube-flannel-ds-jmjv8                       1/1       Running   0          1d
kube-flannel-ds-xspq6                       1/1       Running   0          1d
kube-proxy-l5fpv                            1/1       Running   0          1d
kube-proxy-vzb6z                            1/1       Running   0          1d
kube-proxy-zr4vm                            1/1       Running   0          1d
kube-scheduler-k8s-master-node              1/1       Running   0          1d
kubernetes-dashboard-6948bdb78-b49t4        1/1       Running   0          1d
monitoring-grafana-764779695-lcqc6          1/1       Running   0          1d
monitoring-influxdb-6d7bc5546f-dd2k8        1/1       Running   0          1d

In the video I see you are running kube v1.10.* and I am running v1.11.0.

Suggestion: Cloud the move from kubedns to coredns be the issue?

Also: This may be causing the pod to fail to mount the volume:

pod has unbound PersistentVolumeClaims (repeated 2 times)
Unable to mount volumes for pod "busy-pod_default(97000798-9669-11e8-984f-e2571728a743)": timeout expired waiting for volumes to attach or mount for pod "default"/"busy-pod". list of unmounted volumes=[vol1]. list of unattached volumes=[vol1 default-token-72khc]