Giter Club home page Giter Club logo

onprem-k8s-calico-oss's Introduction

Getting up and running with Calico on your on-premises K8s Cluster

Agenda

  • Kubernetes networking considerations
    • Choose a pod cidr
    • Choose a service cidr
    • Choose which kube-proxy mode to run
  • Calico networking considerations
    • Choose which dataplane to use
    • Choose the initial IP Pool cidr
    • Choose the initial IP Pool block size
    • Choose the initial IP Pool nat outgoing mode
    • How to determine if encapsulation is required
  • Install Kubernetes with Calico using kubeadm
  • Explore Kubernetes with Calico networking
  • Simple Calico network policy example

Notes

Kubernetes networking configuration

  • Cluster pod cidr: 10.48.0.0/16
  • Cluster service cidr: 10.49.0.0/16
  • Cluster kube-proxy mode: iptables

Calico networking configuration

  • Calico dataplane: Standard Linux networking
  • Calico initial IP Pool cidr: 10.48.0.0/24
  • Calico initial IP Pool block size: /26 (default)
  • Calico initial IP Pool nat mode: enabled (default)
  • Calico additional IP Pool cidr: 10.48.1.0/24

Install Kubernetes with Calico using kubeadm

All nodes (Ubuntu 18.04 LTS)

Initial setup
  1. Install kubeadm, cluster dependencies, and friends
K8SVERSION=1.15.4-00
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
sudo apt update -y
sudo apt install \
	docker.io \
	watch \
	ipvsadm \
	ipset \
	tcpdump -y
sudo apt install kubeadm=${K8SVERSION} kubelet=${K8SVERSION} kubectl=${K8SVERSION} -y
sudo systemctl enable docker
sudo docker --version
kubeadm version
sudo swapoff -a
sudo kubeadm config images pull
sudo hostnamectl set-hostname `hostname -f`

Master node

Initial setup
  1. Initialize the kubernetes cluster with iptables mode kube-proxy
echo 'apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
networking:
  serviceSubnet: 10.49.0.0/16
  podSubnet: 10.48.0.0/16
  dnsDomain: cluster.local
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: iptables' > kubeadm-config-iptables-mode.yaml
sudo kubeadm reset -f
sudo kubeadm init --config kubeadm-config-iptables-mode.yaml
  1. Copy the kubectl config into place
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
  1. You should see an output similar to below. It will be different and unique to your environment.
sudo kubeadm join 10.0.0.10:6443 --token 0d3aqz.u2bmp0zwlfdh5pmt \
    --discovery-token-ca-cert-hash sha256:726cf64d358aded6a6584271c5342178f10834e254bfe8ff08357dcc3c6af877
  1. Install and configure calicoctl
curl -O -L  https://github.com/projectcalico/calicoctl/releases/download/v3.13.2/calicoctl
chmod +x calicoctl
sudo mv calicoctl /usr/local/bin
echo 'apiVersion: projectcalico.org/v3
kind: CalicoAPIConfig
metadata:
spec:
  datastoreType: "kubernetes"
  kubeconfig: "/home/ubuntu/.kube/config"' > calicoctl.cfg
sudo mkdir -p /etc/calico
sudo cp calicoctl.cfg /etc/calico

Worker nodes

Initial setup
  1. Login to workers

  2. Reset kubeadm

sudo kubeadm reset
  1. Copy the command from step 3 above and run it on each of your worker nodes. It will look similar to below, but different.
sudo kubeadm join 10.0.0.10:6443 --token 42sy8h.gg7su3eb12dvbu76 --discovery-token-ca-cert-hash sha256:b34cc7c3ee43d7476639624d9b2da9fed9365b7f79525b5c15030f37114a4ccb
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.15" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Activating the kubelet service
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
  1. Login to the other workers and repeat steps 2 and 3.
Master node
Configure and Install Calico
  1. Pull down Calico
curl https://docs.projectcalico.org/manifests/calico.yaml -o calico.yaml
  1. Edit the calico-node DaemonSet
open calico.yaml
  1. Configure the initial IP Pool by setting CALICO_IPV4POOL_CIDR to 10.48.0.0/24

  2. Disable encap by setting CALICO_IPV4POOL_IPIP to Never

  3. Apply the calico.yaml

kubectl apply -f calico.yaml
Explore Kubernetes with Calico networking

Let's look around and explore

  1. Check out the Calico node status
sudo calicoctl node status
Calico process is running.

IPv4 BGP status
+--------------+-------------------+-------+----------+-------------+
| PEER ADDRESS |     PEER TYPE     | STATE |  SINCE   |    INFO     |
+--------------+-------------------+-------+----------+-------------+
| 10.0.0.236   | node-to-node mesh | up    | 00:11:57 | Established |
| 10.0.0.30    | node-to-node mesh | up    | 00:12:10 | Established |
+--------------+-------------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.
  1. Verify our IP pool settings
calicoctl get ippools default-ipv4-ippool -o yaml
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  creationTimestamp: "2020-04-15T23:50:00Z"
  name: default-ipv4-ippool
  resourceVersion: "1063"
  uid: 89bbe3a0-4191-4234-93eb-be5aec34d7ab
spec:
  blockSize: 26
  cidr: 10.48.0.0/24
  ipipMode: Never
  natOutgoing: true
  nodeSelector: all()
  vxlanMode: Never
  1. Check out our IP pool utilization
calicoctl ipam show
+----------+--------------+-----------+------------+-----------+
| GROUPING |     CIDR     | IPS TOTAL | IPS IN USE | IPS FREE  |
+----------+--------------+-----------+------------+-----------+
| IP Pool  | 10.48.0.0/24 |       256 | 3 (1%)     | 253 (99%) |
+----------+--------------+-----------+------------+-----------+
calicoctl ipam show --show-blocks
+----------+----------------+-----------+------------+-----------+
| GROUPING |      CIDR      | IPS TOTAL | IPS IN USE | IPS FREE  |
+----------+----------------+-----------+------------+-----------+
| IP Pool  | 10.48.0.0/24   |       256 | 3 (1%)     | 253 (99%) |
| Block    | 10.48.0.0/26   |        64 | 1 (2%)     | 63 (98%)  |
| Block    | 10.48.0.192/26 |        64 | 2 (3%)     | 62 (97%)  |
+----------+----------------+-----------+------------+-----------+
Simple Calico network policy example
  1. Inspect the network policies
calicoctl get networkpolicies --all-namespaces
NAMESPACE   NAME

  1. Inspect the global network policies
calicoctl get globalnetworkpolicies
NAME

  1. Deploy the zone-based segmentation policies
calicoctl apply -f FirewallZonesPolicies.yaml
  1. Verify the zone-based segmentation policies
calicoctl get networkpolicies --all-namespaces
NAMESPACE   NAME
default     dmz
default     restricted
default     trusted

References

onprem-k8s-calico-oss's People

Contributors

jsabo avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.