Install Cilium networking on Kubernetes

This article continues after kubeadm has initialized the control plane. The node is expected to be visible through kubectl, but it is still NotReady because no pod network has been installed yet.

Cilium will provide the cluster network. Install the Cilium CLI first so the same tool can install Cilium, wait for rollout, and inspect the resulting network state. The command below is for amd64 Linux hosts.

curl -L --remote-name \
  https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz && \
sudo tar xzvfC cilium-linux-amd64.tar.gz /usr/local/bin && \
rm cilium-linux-amd64.tar.gz

cilium version

Cilium Gateway API support depends on compatible Gateway API CRDs. For this setup, pin the CRDs deliberately instead of applying whatever the latest Gateway API release happens to be. The tested setup below uses Gateway API v1.4.0 experimental CRDs.

Version note: Cilium 1.19.x is a Gateway API v1.4-era setup. Cilium 1.19.4 documentation lists Gateway API v1.4.1 support, while newer Cilium documentation moves to later Gateway API versions. Re-check this compatibility before upgrading Cilium or replacing the CRDs.

Remove Gateway API Upgrade Protection#

If a newer Gateway API release previously installed safe-upgrade protection policies, remove them before applying the pinned CRDs.

kubectl delete validatingadmissionpolicy \
  safe-upgrades.gateway.networking.k8s.io \
  --ignore-not-found

kubectl delete validatingadmissionpolicybinding \
  safe-upgrades.gateway.networking.k8s.io \
  --ignore-not-found

Install Compatible Gateway API CRDs#

kubectl apply -f \
  https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/experimental-install.yaml

In a fresh cluster, install these CRDs before Cilium so the Gateway API controller can start cleanly. If Cilium was already installed before the CRDs were applied, restart the Cilium operator after applying them.

Install Cilium with kube-proxy replacement enabled and Gateway API support turned on from the beginning. Enabling both options during the initial CNI install keeps the cluster networking path consistent before application workloads are added.

cilium install \
  --set kubeProxyReplacement=true \
  --set gatewayAPI.enabled=true

Wait for the Cilium rollout before troubleshooting anything else. A partially rolled-out CNI can make normal Kubernetes resources look broken even when the root cause is simply that networking is not ready yet.

cilium status --wait

kubectl get pods -n kube-system

kubectl get nodes

STATUS
Ready

Refresh Gateway API Reconciliation#

Restart the Cilium operator after the install or after changing Gateway API CRDs so it reconciles the Cilium GatewayClass with the currently installed resource definitions.

kubectl rollout restart deploy/cilium-operator -n kube-system

kubectl rollout status deploy/cilium-operator -n kube-system

Check The Cilium GatewayClass#

kubectl get gatewayclass

NAME     CONTROLLER                     ACCEPTED
cilium   io.cilium/gateway-controller   True

kubectl describe gatewayclass cilium

For a single-node lab cluster, remove the control-plane taint so normal workloads can run on the same node. Do not do this blindly in a multi-node production cluster; dedicated control-plane nodes should usually remain unscheduled for application workloads.

kubectl taint nodes --all node-role.kubernetes.io/control-plane-

kubectl describe node | grep Taints

Taints: <none>

Run a small nginx pod to confirm that the scheduler can place a workload and that the node can pull and start a container image through the configured runtime and network stack.

kubectl run nginx --image=nginx && \
kubectl get pods

nginx   Running

Install metrics-server so the cluster can report node and pod resource usage through the Kubernetes metrics API. In small lab environments, kubelet certificate trust is often not fully wired yet, so the insecure TLS flag is commonly added during early setup.

kubectl apply -f \
  https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Edit the metrics-server deployment and add the kubelet TLS flag to the container args.

kubectl edit deployment metrics-server -n kube-system

args:
  - --kubelet-insecure-tls

kubectl top nodes