Kubernetes/Cluster/Ingress-NGINX

Ingress with NGINX

Kubernetes Ingress with NGINX Ingress Controller Example
https://spacelift.io/blog/kubernetes-ingress

Installation Guide - Ingress-Nginx Controller
https://kubernetes.github.io/ingress-nginx/deploy/

kubernetes/ingress-nginx: Ingress-NGINX Controller for Kubernetes (GitHub)
https://github.com/kubernetes/ingress-nginx/

Install ingress manifest according to article #1:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.0/deploy/static/provider/cloud/deploy.yaml

Install ingress manifest according to article #2:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/cloud/deploy.yaml

Latest release v1.9.5 as of 2023.12.22:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.5/deploy/static/provider/cloud/deploy.yaml

v1.9.4 release:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.4/deploy/static/provider/cloud/deploy.yaml

Or latest code:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml

To remove:

kubectl delete -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.5/deploy/static/provider/cloud/deploy.yaml

Get ingress-nginx pods:

kubectl get pods --namespace ingress-nginx

# k get pods -A
NAMESPACE       NAME                                        READY   STATUS      RESTARTS      AGE
ingress-nginx   ingress-nginx-admission-create-5rwph        0/1     Completed   0             40s
ingress-nginx   ingress-nginx-admission-patch-vt8rt         0/1     Completed   1             40s
ingress-nginx   ingress-nginx-controller-7b498b6db5-2t8rv   1/1     Running     0             40s

Stuck waiting for external-ip

# kubectl get service ingress-nginx-controller --namespace=ingress-nginx
NAME                       TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller   LoadBalancer   10.107.58.156   <pending>     80:31044/TCP,443:30097/TCP   6m15s

k get service

k get service -A

kubectl rollout restart deployment ingress-nginx-controller -n ingress-nginx

---

k get pods -A

# k get pods -A
NAMESPACE       NAME                                        READY   STATUS              RESTARTS   AGE
ingress-nginx   ingress-nginx-admission-create-s9q5r        0/1     ContainerCreating   0          34m
ingress-nginx   ingress-nginx-admission-patch-4w2pp         0/1     ContainerCreating   0          34m
ingress-nginx   ingress-nginx-controller-7b498b6db5-fh5hr   0/1     ContainerCreating   0          34m
...

# k -n ingress-nginx describe pod ingress-nginx-admission-create-s9q5r

  Warning  FailedCreatePodSandBox  10m                    kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "effe0db2192b4ab7545e0cd28dee492c45caa433f71a201633015c6f0c2a1d8e" network for pod "ingress-nginx-admission-create-s9q5r": networkPlugin cni failed to set up pod "ingress-nginx-admission-create-s9q5r_ingress-nginx" network: plugin type="flannel" failed (add): failed to delegate add: failed to set bridge addr: "cni0" already has an IP address different from 10.244.3.1/24

SSL

Service and Ingress configuration:

---
##
## SERVICE
##
apiVersion: v1
kind: Service
metadata:
  name: dev-service
  namespace: dev
spec:
  selector:
    app: dev-nginx
  type: NodePort
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
---
##
## INGRESS WEB ACCESS
##
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  labels:
    app.aznot.com/instance: dev
    app.aznot.com/name: dev
  name: devex-ingress
  namespace: dev
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
    nginx.ingress.kubernetes.io/rewrite-target: "/"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/limit-rps: "20"
    nginx.ingress.kubernetes.io/client-max-body-size: "100m"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "300s"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300s"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      if ($host = "www.dev.aznot.com") {
          return 308 https://$host$request_uri;
      }
spec:
  # tls:
  # - hosts:
  #     - dev.aznot.com
  #   secretName: dev-ssl-certs
  rules:
  - host: dev.aznot.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: dev-service
            port:
              number: 80

When you are ready to deploy the SSL certificate, uncommend the tls: section.

Add cert to dev-ssl-certs:

# kubectl create secret tls ${CERT_NAME} --key ${KEY_FILE} --cert ${CERT_FILE}
kubectl -n dev create secret tls dev-ssl-certs --key dev.key --cert dev.crt
kubectl -n dev describe secret dev-ssl-certs
kubectl -n dev get secret dev-ssl-certs -o yaml

Note: the .crt/.cer/.pem file should have the key chain started with specific to least specific

# CN = dev.aznot.com
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
# C = US, ST = DE, L = Wilmington, O = Corporation Service Company, CN = Trusted Secure Certificate Authority DV
-----BEGIN CERTIFICATE-----
...

change IP of ingress

ingress-nginx-controller

apiVersion: v1
kind: Service
metadata:
  name: somename-lb
  namespace: namespace
  labels:
    app: someapp
spec:
  type: LoadBalancer
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
    name: http
  selector:
    app: someapp
  loadBalancerIP: xxx.xxx.xxx.xxx

Kubernetes/MetalLB - Is there a way to set an IP address for a service without individual address-pools? : kubernetes
https://www.reddit.com/r/kubernetes/comments/gy2evb/kubernetesmetallb_is_there_a_way_to_set_an_ip/

ingress-nginx-controller

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.9.4
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  externalTrafficPolicy: Local
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: http
    name: http
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: LoadBalancer

k get service ingress-nginx-controller -n ingress-nginx -o yaml > controller.yaml

...
status:
  loadBalancer:
    ingress:
    - ip: 192.168.108.80

Edit IP address... delete service

k delete service ingress-nginx-controller -n ingress-nginx

k apply -f controller.yaml

Back in business!

Alternative - NGINX Ingress Controller

nginxinc/kubernetes-ingress: NGINX and NGINX Plus Ingress Controllers for Kubernetes
https://github.com/nginxinc/kubernetes-ingress

NGINX Ingress Controller
https://docs.nginx.com/nginx-ingress-controller/

There are two Nginx Ingress Controllers for k8s. What? | by Grigor Khachatryan | Medium
https://grigorkh.medium.com/there-are-two-nginx-ingress-controllers-for-k8s-what-44c7b548e678

"There are two popular Kubernetes Ingress controllers that use NGINX — both are open source and hosted on GitHub. One is maintained by the Kubernetes open source community ( kubernetes/ingress-nginx on GitHub) and one is maintained by NGINX, Inc. ( nginxinc/kubernetes-ingress on GitHub)."

For the key difference between nginxinc/kubernetes-ingress and kubernetes/ingress-nginx Ingress controllers you can check out this table:

https://gist.github.com/grigorkh/f8e4fd73e99f0fde06a51e2ed7c2156c