Kubernetes/Cluster
Jump to navigation
Jump to search
Create Cluster
References:
How To Setup Kubernetes Cluster Using Kubeadm - Easy Guide https://devopscube.com/setup-kubernetes-cluster-kubeadm/
Creating a cluster with kubeadm | Kubernetes https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/
Install kubeadm
Reference:
Installing kubeadm | Kubernetes https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
Install dependencies:
sudo apt-get update sudo apt-get install -y apt-transport-https ca-certificates curl gpg curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
Install kubelet kubeadm kubectl
sudo apt-get update sudo apt-get install -y kubelet kubeadm kubectl sudo apt-mark hold kubelet kubeadm kubectl
Setup Docker Runtime with Shim
Note: Technically Docker seems to have had a falling out with Kubernetes, but as it is still popular, you can still use it with cri-dockerd shim...
Install Docker:
apt install docker.io
Install shim: [1]
wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.8/cri-dockerd-0.3.8.amd64.tgz tar xvf cri-dockerd-0.3.8.amd64.tgz sudo mv ./cri-dockerd /usr/local/bin/ ln -s /usr/local/bin/cri-dockerd /usr/bin/cri-dockerd
Initalize Cluster
Run pre-flight checks:
sudo kubeadm init phase preflight
Pre pull images:
sudo kubeadm config images pull
Fake initalize cluster:
# kubeadm init --dry-run sudo kubeadm init --dry-run --cri-socket=unix:///var/run/cri-dockerd.sock
Initalize cluster:
# kubeadm init sudo kubeadm init --cri-socket=unix:///var/run/cri-dockerd.sock --pod-network-cidr=10.244.0.0/16
Note: the "--pod-network-cidr=10.244.0.0/16" is needed for Flannel. Silly Flannel.
Initalized Success
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.x.x.x:6443 --token exxxxxxx.xxxxxxxxxxxx \
--discovery-token-ca-cert-hash sha256:4xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Initalize Pod Network - Flannel
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
kubectl get pods --all-namespaces
reference:
GitHub - flannel-io/flannel: flannel is a network fabric for containers, designed for Kubernetes https://github.com/flannel-io/flannel#deploying-flannel-manually
Enable iptables Bridged Traffic on all the Nodes
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf overlay br_netfilter EOF sudo modprobe overlay sudo modprobe br_netfilter # sysctl params required by setup, params persist across reboots cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 EOF # Apply sysctl params without reboot sudo sysctl --system
Disable swap on all the Nodes
Kubernetes doesn't like Swap. Disable it.
root@kube1:~# kubeadm init
[init] Using Kubernetes version: v1.29.0
[preflight] Running pre-flight checks
[WARNING Swap]: swap is supported for cgroup v2 only; the NodeSwap feature gate of the kubelet is beta but disabled by default
Disable swap:
sudo swapoff -a (crontab -l 2>/dev/null; echo "@reboot /sbin/swapoff -a") | crontab - || true
Install Docker
ref [2]
Setup Repo
sudo apt-get update sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo \ "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
ref [3]
Install Docker Enginer
sudo apt-get update sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
ref [4]
Install docker and plugin
sudo apt-get update sudo apt-get install docker-compose-plugin
ref [5]
Install Container Runtime
Install cri-dockerd binary:
wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.8/cri-dockerd-0.3.8.amd64.tgz tar xvf cri-dockerd-0.3.8.amd64.tgz sudo mv cri-dockerd/cri-dockerd /usr/local/bin/ ln -s /usr/local/bin/cri-dockerd /usr/bin/cri-dockerd
wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.2.0/cri-dockerd-v0.2.0-linux-amd64.tar.gz tar xvf cri-dockerd-v0.2.0-linux-amd64.tar.gz sudo mv ./cri-dockerd /usr/local/bin/ ln -s /usr/local/bin/cri-dockerd /usr/bin/cri-dockerd
cri-dockerd --help
Install service:
wget https://raw.githubusercontent.com/Mirantis/cri-dockerd/master/packaging/systemd/cri-docker.service wget https://raw.githubusercontent.com/Mirantis/cri-dockerd/master/packaging/systemd/cri-docker.socket sudo mv cri-docker.socket cri-docker.service /etc/systemd/system/ sudo sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service
systemctl daemon-reload systemctl enable cri-docker.service systemctl enable --now cri-docker.socket
systemctl start cri-docker.service systemctl start cri-docker.socket
systemctl status cri-docker.service systemctl status cri-docker.socket
journalctl -u cri-docker.service journalctl -u cri-docker.socket
crictl info crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock info
/etc/crictl.yaml
runtime-endpoint: unix:///var/run/containerd/containerd.sock image-endpoint: unix:///var/run/containerd/containerd.sock timeout: 10 debug: true
runtime-endpoint: unix:///var/run/cri-dockerd.sock image-endpoint: unix:///var/run/cri-dockerd.sock timeout: 10 debug: true
- /var/lib/kubelet/kubeadm-flags.env
ref [6]
Old:
git clone https://github.com/Mirantis/cri-dockerd.git
cd cri-dockerd
mkdir bin
VERSION=$((git describe --abbrev=0 --tags | sed -e 's/v//') || echo $(cat VERSION)-$(git log -1 --pretty='%h')) PRERELEASE=$(grep -q dev <<< "${VERSION}" && echo "pre" || echo "") REVISION=$(git log -1 --pretty='%h')
go build -ldflags="-X github.com/Mirantis/cri-dockerd/version.Version='$VERSION}' -X github.com/Mirantis/cri-dockerd/version.PreRelease='$PRERELEASE' -X github.com/Mirantis/cri-dockerd/version.BuildTime='$BUILD_DATE' -X github.com/Mirantis/cri-dockerd/version.GitCommit='$REVISION'" -o cri-dockerd
...
ref [7]
Install Kubeadm & Kubelet & Kubectl on all Nodes
sudo apt-get update && \ sudo apt-get install -y apt-transport-https ca-certificates curl && \ sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
#echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B53DC80D13EDEF05
sudo apt-get update -y && \ sudo apt-get install -y kubelet kubeadm kubectl && \ sudo apt-mark hold kubelet kubeadm kubectl
sudo apt-get update && \ sudo apt-get install -y kubelet=1.26.5-00 kubeadm=1.26.5-00 kubectl=1.26.5-00 docker.io && \ sudo apt-mark hold kubelet kubeadm kubectl docker.io
sudo mkdir /etc/containerd && \ sudo containerd config default > /etc/containerd/config.toml && \ sudo sed -i 's/ SystemdCgroup = false/ SystemdCgroup = true/' /etc/containerd/config.toml && \ sudo systemctl restart containerd && \ sudo systemctl restart kubelet
Initialize Kubeadm On Master Node To Setup Control Plane
sudo kubeadm init
Sockets found:
unix:///var/run/containerd/containerd.sock unix:///var/run/cri-dockerd.sock
sudo kubeadm init --cri-socket=unix:///var/run/cri-dockerd.sock
Tear Down Cluster
If you need to try again:
kubeadm reset
Sort of, this doesn't do a great job...
Notes
How To Setup Kubernetes Cluster Using Kubeadm - Easy Guide https://devopscube.com/setup-kubernetes-cluster-kubeadm/
How to Set Up Three Node Kubernetes Cluster on Ubuntu https://k21academy.com/docker-kubernetes/three-node-kubernetes-cluster/
5 Fastest Ways to Install a k8s Cluster - Your DevOps Mentor https://yourdevopsmentor.com/blog/5-ways-to-install-a-kubernetes-cluster/
Kubernetes Cluster setup on Ubuntu 22.04 using kubeadm with Calico, By Sir Babar Zahoor https://www.linkedin.com/pulse/kubernetes-cluster-setup-ubuntu-2204-using-kubeadm-calico-md-sajjad
How to Set Up Three Node Kubernetes Cluster on Ubuntu 20.04 https://www.atlantic.net/dedicated-server-hosting/how-to-set-up-three-node-kubernetes-cluster-on-ubuntu-20-04/
Using Kubeadm to Create a Cluster | Airplane https://www.airplane.dev/blog/kubeadm
Install Mirantis cri-dockerd as Docker Engine shim for Kubernetes | ComputingForGeeks https://computingforgeeks.com/install-mirantis-cri-dockerd-as-docker-engine-shim-for-kubernetes/
Install Kubernetes Cluster on Ubuntu 22.04 with kubeadm | ComputingForGeeks https://computingforgeeks.com/install-kubernetes-cluster-ubuntu-jammy/
Install Kubernetes 1.26 on Ubuntu 20.04 or 22.04 LTS | by Akriotis Kyriakos | Medium https://akyriako.medium.com/install-kubernetes-on-ubuntu-20-04-f1791e8cf799
How to Install Kubernetes Cluster on Ubuntu 22.04 https://www.linuxtechi.com/install-kubernetes-on-ubuntu-22-04/