Kubernetes

Subpage Table of Contents

• Kubernetes/Cluster
• Kubernetes/Cluster/CoreDNS
• Kubernetes/Cluster/HAProxy
• Kubernetes/Cluster/Ingress-NGINX
• Kubernetes/Cluster/Keepalived
• Kubernetes/Cluster/MetalLB
• Kubernetes/Cluster/Node Roles
• Kubernetes/Rancher
• Kubernetes/kustomization.yaml
• Kubernetes/kustomize

Kubernetes

Kubernetes, also known as K8s.

Kubernetes is a container or microservice platform that orchestrates computing, networking, and storage infrastructure workloads.

The Kubernetes Project was open-sourced by Google in 2014 after using it to run production workloads at scale for more than a decade. Kubernetes provides the ability to run dynamically scaling, containerised applications, and utilising an API for management. Kubernetes is a vendor-agnostic container management tool, minifying cloud computing costs whilst simplifying the running of resilient and scalable applications.

k8s

"By the way, if you’re wondering where the name “Kubernetes” came from, it is a Greek word, meaning helmsman or pilot. The abbreviation K8s is derived by replacing the eight letters of “ubernete” with the digit 8." [1]

Download

Download Kubernetes | Kubernetes

https://kubernetes.io/releases/download/

List of container images:

curl -Ls "https://sbom.k8s.io/$(curl -Ls https://dl.k8s.io/release/stable.txt)/release" | grep "SPDXID: SPDXRef-Package-registry.k8s.io" |  grep -v sha256 | cut -d- -f3- | sed 's/-/\//' | sed 's/-v1/:v1/'

kubectl install

Install and Set Up kubectl on Linux | Kubernetes

https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/

Download binary:

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

Verify checksum:

curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check

Install binary:

sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

Verify version:

kubectl version
kubectl version --client
kubectl version --client --output=yaml

Make backup, in case you need previous version to match Kube cluster:

 ls -la /usr/local/bin/kubectl*
sudo cp /usr/local/bin/kubectl /usr/local/bin/kubectl-1.31.0   # match version
 ls -la /usr/local/bin/kubectl*

Bash completion:

# should already be installed...
apt-get install bash-completion

echo 'source <(kubectl completion bash)' >>~/.bashrc

Aliases:

echo 'alias k=kubectl' >>~/.bashrc
echo 'alias k8s=kubectl' >>~/.bashrc
echo 'alias k9=kubectl' >>~/.bashrc

echo 'alias kw=kubectl' >>~/.bashrc
echo 'alias k8sw=kubectl' >>~/.bashrc
echo 'alias k9w=kubectl' >>~/.bashrc

echo 'complete -o default -F __start_kubectl k' >>~/.bashrc
echo 'complete -o default -F __start_kubectl k8s' >>~/.bashrc
echo 'complete -o default -F __start_kubectl k9' >>~/.bashrc

echo 'complete -o default -F __start_kubectl kw' >>~/.bashrc
echo 'complete -o default -F __start_kubectl k8sw' >>~/.bashrc
echo 'complete -o default -F __start_kubectl k9w' >>~/.bashrc

ref: [2]

Kubectl config

.kube/config

Definitions

Basic objects include:

Pod. A group of one or more containers.

Service. An abstraction that defines a logical set of pods as well as the policy for accessing them.

Volume. An abstraction that lets us persist data. (This is necessary because containers are ephemeral—meaning data is deleted when the container is deleted.)

Namespace. A segment of the cluster dedicated to a certain purpose, for example a certain project or team of devs.

Create Simple Bash Pod

kubectl run my-shell --rm -i --tty --image ubuntu -- bash

• my-shell: This ends up being the name of the Deployment that is created. Your pod name will typically be this plus a unique hash or ID at the end.
• --rm: Delete any resources we've created once we detach. When you exit out of your session, this cleans up the Deployment and Pod.
• -i/--tty: The combination of these two are what allows us to attach to an interactive session.
• --: Delimits the end of the kubectl run options from the positional arg (bash).
• bash: Overrides the container's CMD. In this case, we want to launch bash as our container's command

ref [3]

Create simple bash pods in background

kubectl run shell1 --image ubuntu sleep infinity
kubectl run shell2 --image ubuntu sleep infinity
kubectl run shell3 --image ubuntu sleep infinity

Basic Commands

Get Nodes

Get Nodes:

kubectl get nodes

## Example:
# kubectl get nodes
NAME               STATUS     ROLES                  AGE     VERSION
ci-2210            Ready      <none>                 509d    v1.22.2
ci-2211            Ready      <none>                 509d    v1.22.2
ci-2212            Ready      <none>                 509d    v1.22.2
ci-4010            NotReady   <none>                 157d
ci-0               Ready      control-plane,master   299d    v1.22.2

Get Nodes Wide Output

kubectl get nodes -o wide

## Example:
# kubectl get nodes -o wide
NAME               STATUS     ROLES                  AGE    VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
ci-infra-01        Ready      <none>                 83d    v1.22.2   10.x.x.x         <none>        Ubuntu 20.04.3 LTS   5.4.0-97-generic    docker://20.10.18

Get Nodes with Labels

k get nodes --show-labels

## Example:
$ kubectl kubectl get nodes --show-labels
NAME               STATUS     ROLES                  AGE    VERSION   LABELS
ci-infra-01        Ready      <none>                 83d    v1.22.2   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=ci-infra-01,kubernetes.io/os=linux

Label Node

# kubectl label nodes <node-name> <label-key>=<label-value>

k label nodes somenode nonmaster=nonmaster

spec:
...
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
          - key: nonmaster
            operator: In
            values:
              - nonmaster

Evict Pods from Node

kubectl drain <nodename>

Make Node Unschedulable

kubectl cordon <nodename>

Make Node Scheduable

kubectl uncordon <nodename>

Delete Node

kubectl delete node <nodename>

Get Pods

Get Pods (in the default name space)

kubectl get pods

# Example:
# kubectl get pods
NAME     READY   STATUS    RESTARTS   AGE
kshell   1/1     Running   0          3d20h

Get Pods (more details):

kubectl get pods -o wide

# Example
# kubectl get pods -o wide
NAME     READY   STATUS    RESTARTS   AGE     IP          NODE              NOMINATED NODE   READINESS GATES
kshell   1/1     Running   0          3d20h   50.0.40.7   kubeserver1       <none>           <none>

Get Pods from All Namespaces:

# kubectl get pods -A
NAMESPACE   NAME     READY   STATUS    RESTARTS   AGE
default     kshell   1/1     Running   0          3d20h

# kubectl get pods -A -o wide
NAMESPACE   NAME     READY   STATUS    RESTARTS   AGE     IP          NODE              NOMINATED NODE   READINESS GATES
default     kshell   1/1     Running   0          3d20h   50.0.40.7   kubeserver1       <none>           <none>

Filter for specific pod:

kubectl get pods -o wide | grep <nodename>

To specify a different name space add "-n [namespace]"

kubectl get pods -n MyNamespace

ref [4]

Get Specific Pod Details

kubectl get pod [pod-name]

# Example:
# kube get pod kshell
NAME     READY   STATUS    RESTARTS   AGE
kshell   1/1     Running   0          3d20h

Describe Pod

kubectl describe pod [pod-name]

# kubectl -n kshell describe pod kshell
Name:         kshell-234234234
Namespace:    kshell
...

Pod Logs

kubectl get logs [pod-name]

# kubectl -n kshell logs kshell

Connect to Pod and Run Bash

kubectl exec -it [single-container-pod] -- bash

# example:
kubectl exec --stdin --tty shell-demo -- /bin/bash

Get Logs of Pod

kubectl logs [pod-name]

Create Simple Pod from Image

kubectl run my-shell --rm -i --tty --image ubuntu -- bash

# run in background
kubectl run my-shell -i --tty --image ubuntu -- bash
ctrl+p ctrl+q

Note: "image" is Docker image name

Attach to Simple Pod

kubectl attach my-shell -c my-shell -i -t

kubectl exec my-shell -it -- /bin/bash

Delete Pod

kubectl delete pod [pod-name]

NAME SPACE MANAGEMENT

kubectl create namespace [namespace]

Use name space

kubectl -ns [namespace] ...other_k8_commands...

Set default namespace for commands: [5]

kubectl config set-context --current --namespace=[namespace]

To unset:

kubectl config set-context --current --namespace=""

View current context namespace: [6]

kubectl config view | grep namespace:
# or
kubectl config view -o jsonpath={.contexts[].context.namespace}

CONTEXT MANAGEMENT

ref: https://stackoverflow.com/questions/55373686/how-to-switch-namespace-in-kubernetes

kubectl config set-context gce-dev --user=cluster-admin --namespace=dev
kubectl config use-context gce-dev

With aliases:

$ alias kubens='kubectl config set-context --current --namespace '
$ alias kubectx='kubectl config use-context '

// Usage
$ kubens kube-system    // Switch to a different namespace
$ kubectx docker        // Switch to separate context

With addons like kubectx & kubens

• https://github.com/ahmetb/kubectx

kubens kube-system 

With addon like kubectl-use:

• https://github.com/kvaps/kubectl-use

# kubectl use prod
Switched to context "prod".

# kubectl use default
Switched to namespace "default".

# kubectl use stage kube-system
Switched to context "stage".
Switched to namespace "kube-system".

With kubie alternative to kubectl: [7]

• https://github.com/sbstp/kubie

kubie ctx ...

Advanced Commands

Persistent Volumes

Persistent Volumes
https://kubernetes.io/docs/concepts/storage/persistent-volumes/

Heirachy:

• Storage Class - global
• Persistent Volume - global
• Persistent Volumes Claims - tied to name space

Create Pod from Yaml

Sample Yaml: (php.yml)

apiVersion: v1
kind: Pod
metadata:
  name: nodejsapp-pod
  labels:
    app: nodejsapp
    type: front-end
spec:
  containers:
    - name: nodejsapp-erp
    image: bharathirajutut/erp:1.0

Create pod from Yaml:

kubectl apply -f php.yaml

ref [8]

---

Sample Yaml: (shell-demo.yaml)

apiVersion: v1
kind: Pod
metadata:
  name: shell-demo
spec:
  volumes:
  - name: shared-data
    emptyDir: {}
  containers:
  - name: nginx
    image: nginx
    volumeMounts:
    - name: shared-data
      mountPath: /usr/share/nginx/html
  hostNetwork: true
  dnsPolicy: Default

Create pod from Yaml:

kubectl apply -f https://k8s.io/examples/application/shell-demo.yaml

ref [9]

Get YAML from Object

kubectl get deployment,service,pod yourapp -o yaml --export

kubectl get deploy --all-namespaces -o yaml --export

kubectl get deploy deploymentname -o yaml

kubectl get deploy,sts,svc,configmap,secret -n default -o yaml --export > default.yaml

Get YAML for deployed Kubernetes services? - Stack Overflow
https://stackoverflow.com/questions/43941772/get-yaml-for-deployed-kubernetes-services

Rolling Deployment

kubectl rollout restart deployment -n test-namespace

ref: https://kubernetes.io/docs/reference/kubectl/generated/kubectl_rollout/kubectl_rollout_restart/

Sample Pod Configs

More Samples:

Pod - Kubernetes examples
https://k8s-examples.container-solutions.com/examples/Pod/Pod.html

Nginx

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80

kubectl apply -f https://k8s.io/examples/pods/simple-pod.yaml

ref: [10]

Pod Template

Prints one thing, then exits. Template for something bigger.

apiVersion: batch/v1
kind: Job
metadata:
  name: hello
spec:
  template:
    # This is the pod template
    spec:
      containers:
      - name: hello
        image: busybox:1.28
        command: ['sh', '-c', 'echo "Hello, Kubernetes!" && sleep infinity']
      restartPolicy: OnFailure
    # The pod template ends here

kshell.yaml

---
##
## NAMESPACE
##
apiVersion: v1
kind: Namespace
metadata:
  name: kenneth
  labels:
    name: kenneth
---
##
## APP DEPLOYMENT
##
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kshell
  namespace: kenneth
  labels:
    app: kshell
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kshell
  template:
    metadata:
      labels:
        app: kshell
    spec:
      containers:
      - name: kshell
        image: ubuntu
        command: ["sleep"]
        args: ["infinity"]
...

Busy Box Command

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: busybox
  name: busybox
spec:
  containers:
  - command:
    - /bin/sh
    - -c
    - |
      echo "running below scripts"
      i=0;
      while true;
      do
        echo "$i: $(date)";
        i=$((i+1));
        sleep 1;
      done
    name: busybox
    image: busybox

ref: [11]

Create Kubernetes Cluster

See Kubernetes/Cluster

Kill Pods Stuck in Terminating Due to Finalizers

Likely stuck due to Finalizer. Look for finalizer:

kubectl get pod [POD_NAMENAME] -o yaml

Example:

 finalizers:
 - actions.summerwind.dev/runner-pod

If so, remove the finalizer:

kubectl patch pod [POD_NAME] -p '{"metadata":{"finalizers":null}}'

Debugging Issues

k describe pod [POD NAME]

k logs [POD NAME]

k events -n [NAMESPACE]