Cilium Powered vClusters - Part 1: Using Cilium as CNI with Vind

Learn how to replace the default Flannel CNI and kube-proxy with Cilium's eBPF-powered networking inside a vind cluster - step by step, with every command explained.

If you've been following the vCluster ecosystem, you already know about vind, vCluster in Docker, a powerful alternative to kind for running Kubernetes clusters locally. In this post, we are going to use vind to run a local Kubernetes cluster and replace the default Flannel CNI and kube-proxy with Cilium, the modern eBPF-based networking solution.

This guide walks through every step, explains every concept, and highlights the most common mistakes so you can avoid them.

Why Cilium?

Every Kubernetes cluster needs to solve two networking problems:

JobTraditional toolWith CiliumPod networking - give pods IP addresses and let them talk to each otherFlannel / CalicoCilium CNIService routing - route traffic from service IPs to the right podkube-proxy (iptables)Cilium eBPF

Cilium handles both jobs using eBPF - a modern Linux kernel technology that bypasses iptables entirely. The result is faster networking, better observability, and a single component instead of two.

What is eBPF?

eBPF (extended Berkeley Packet Filter) lets programs run directly in the Linux kernel without modifying kernel source code. Cilium uses it to handle networking at the kernel level - much faster and more scalable than traditional iptables rules.

Prerequisites:

Make sure these are installed and running before starting:

# Check Docker is running
docker info | grep "Server Version"

# Check vCluster CLI version (needs v0.31+)
vcluster version

# Check Helm is installed
helm version --short

# Check kubectl is installed
kubectl version --client --short


If anything is missing, install it:

# Install vCluster CLI on Mac
brew install loft-sh/tap/vcluster

# Install Helm on Mac
brew install helm

# Install kubectl on Mac
brew install kubectl


Step 1 - Switch to Docker Driver

vcluster use driver docker


This tells the vCluster CLI to create clusters as Docker containers on your machine - activating vind mode. By default the CLI deploys vClusters into an existing Kubernetes cluster using Helm. Switching to the Docker driver changes this behaviour completely.

Optional: Start the vCluster Platform UI with vcluster platform start. This gives you a web interface to manage clusters visually - but it is completely optional. vind works perfectly without it.

Step 2 - Create the vind Configuration

Save this as vind-cilium.yaml:

experimental:
 docker:
   nodes:
     - name: "worker-1"
     - name: "worker-2"
deploy:
 kubeProxy:
   enabled: false   # Cilium will replace kube-proxy
 cni:
   flannel:
     enabled: false  # Cilium will replace Flannel


Let's break down what each setting does:

• experimental.docker.nodes - defines two extra worker node containers. vind also creates a control plane container automatically.
• deploy.kubeProxy.enabled: false - disables kube-proxy. kube-proxy normally handles service routing using iptables. Cilium replaces it using eBPF.
• deploy.cni.flannel.enabled: false - disables Flannel, the default CNI. Cilium will handle pod IP assignment and pod-to-pod networking instead.

Important: You cannot have two components doing the same networking job. If you leave Flannel or kube-proxy enabled alongside Cilium, they will conflict and cause broken pod connectivity or service routing failures.

Step 3 - Create the vind Cluster

vcluster create cilium-vind -f vind-cilium.yaml


This creates a Kubernetes cluster with three Docker containers - one control plane and two workers. Verify they are running:

docker ps | grep vcluster


Expected output

vcluster.cp.cilium-vind              <- control plane
vcluster.node.cilium-vind.worker-1   <- worker 1
vcluster.node.cilium-vind.worker-2   <- worker 2


Check node status:

kubectl get nodes


Expected output

NAME          STATUS     ROLES
cilium-vind   NotReady   control-plane,master
worker-1      NotReady   <none>
worker-2      NotReady   <none>


NotReady is expected! We disabled Flannel so there is no CNI running yet. Kubernetes nodes stay NotReady until a CNI plugin is installed to handle pod networking. Cilium will fix this in the next step.

Step 4 - Find the API Server IP

This is the most critical step - and the most commonly missed one.

kubectl get endpoints kubernetes -n default


Expected output

NAME         ENDPOINTS
kubernetes   172.20.0.2:8443   <- note this IP and port


⚡ The Chicken and Egg Problem: Cilium needs DNS to find the API server → DNS needs a CNI to work → Cilium IS the CNI → Cilium cannot start → stuck in Init:0/6 forever. The fix: bypass DNS entirely with a direct IP address. This is documented in the official Cilium kube-proxy-free docs - every example uses a direct IP, never a DNS name.

Step 5 - Install Cilium

1. Add the Cilium Helm repo

helm repo add cilium <https://helm.cilium.io>
helm repo update


2. Install Cilium - use the IP and port from Step 4

helm install cilium cilium/cilium \\
 --version 1.16.0 \\
 --namespace kube-system \\
 --set kubeProxyReplacement=true \\
 --set k8sServiceHost=172.20.0.2 \\
 --set k8sServicePort=8443 \\
 --set image.pullPolicy=IfNotPresent \\
 --set ipam.mode=kubernetes \\
 --set envoy.enabled=false


What each flag does:

FlagWhat it doeskubeProxyReplacement=trueCilium fully replaces kube-proxy -> all service routing via eBPF, no iptablesk8sServiceHost=172.20.0.2Direct IP of the API server -> bypasses DNS, works before CNI is runningk8sServicePort=8443Port of the API server - from the endpoints output in Step 4ipam.mode=kubernetesCilium uses Kubernetes pod CIDR ranges to assign pod IP addressesenvoy.enabled=falseDisables the Envoy L7 proxy sidecar -> not needed for this basic setupimage.pullPolicy=IfNotPresentReuses cached images -> saves time on repeated installs in a lab

• Now watch Cilium become ready and time it:

time kubectl -n kube-system rollout status ds/cilium


Expected output

Waiting for daemon set "cilium" rollout to finish: 0 of 3 updated pods are available...
Waiting for daemon set "cilium" rollout to finish: 1 of 3 updated pods are available...
Waiting for daemon set "cilium" rollout to finish: 2 of 3 updated pods are available...
daemon set "cilium" successfully rolled out

real    0m7.781s


Step 6 - Verify the Installation

1. Check all pods are healthy

kubectl get pods -n kube-system


Expected output

NAME                        READY   STATUS    RESTARTS
cilium-xxxxx                1/1     Running   0        <- Cilium agent node 1
cilium-yyyyy                1/1     Running   0        <- Cilium agent node 2
cilium-zzzzz                1/1     Running   0        <- Cilium agent node 3
cilium-operator-xxxxx       1/1     Running   0        <- Cilium operator
coredns-xxxxxxxxx           1/1     Running   0        <- cluster DNS


2. Check nodes are now Ready

kubectl get nodes


Expected output

NAME          STATUS   ROLES                  VERSION
cilium-vind   Ready    control-plane,master   v1.35.0
worker-1      Ready    <none>                 v1.35.0
worker-2      Ready    <none>                 v1.35.0


3. Verify kube-proxy is gone

kubectl get pods -n kube-system | grep kube-proxy
# Should return nothing because Cilium handles everything


4. Test DNS resolution

kubectl run dns-test --image=busybox:1.28 --restart=Never -- sleep 300
kubectl wait pod dns-test --for=condition=Ready --timeout=90s
kubectl exec dns-test -- nslookup kubernetes.default.svc.cluster.local


Expected output

Server:    10.109.x.x
Name:      kubernetes.default.svc.cluster.local
Address 1: 10.96.0.1


5. Test pod-to-pod networking

kubectl run nginx --image=nginx --restart=Never
kubectl wait pod nginx --for=condition=Ready --timeout=90s
kubectl get pod nginx -o wide


• Test connectivity (replace with actual pod IP)

kubectl exec dns-test -- wget -O- http://<NGINX_POD_IP> --timeout=5


Expected output

Connecting to 10.244.x.xxx (10.244.x.xxx:80)
Welcome to nginx!


• Cleanup test pods when done

kubectl delete pod dns-test nginx


Common Mistakes to Avoid

Mistake 1: Using a DNS name for k8sServiceHost: Using kubernetes.default.svc.cluster.local as the value for k8sServiceHost will cause Cilium to get stuck in Init:0/6 forever. DNS cannot resolve Kubernetes service names when no CNI is running yet. Always use the direct IP from kubectl get endpoints kubernetes -n default.

Mistake 2: Not setting k8sServiceHost at all: Leaving k8sServiceHost unset when kube-proxy is disabled causes the same Init:0/6 hang. Without kube-proxy, the KUBERNETES_SERVICE_HOST environment variable is never injected into pods, so Cilium has no way to auto-detect the API server.

Mistake 3: Forgetting to disable Flannel and kube-proxy: Running Cilium alongside Flannel and kube-proxy causes conflicts. Two CNIs fighting over pod networking and two components routing service traffic results in broken connectivity. Always set deploy.kubeProxy.enabled: false and deploy.cni.flannel.enabled: false in your vind config before installing Cilium.

Key Takeaway

When kube-proxy is disabled, you MUST set k8sServiceHost to the direct IP address of your API server. Find it with kubectl get endpoints kubernetes -n default and always use the IP, never a DNS name. This single setting is the difference between Cilium starting in under 8 seconds and being stuck in Init:0/6 forever.

References

• Official Cilium kube-proxy-free documentation
• vind official page - vCluster in Docker
• Replacing kind with vind - A Deep Dive
• Cilium Helm chart reference

‍