Cilium Powered vClusters: Part 1 - Using Cilium as CNI with vind
|
min Read
If you've been following the vCluster ecosystem, you already know about vind , vCluster in Docker , a powerful alternative to kind for running Kubernetes clusters locally.
In this post, we are going to use vind to run a local Kubernetes cluster and replace the default Flannel CNI and kube-proxy with Cilium, the modern eBPF-based networking solution.
This guide walks through every step, explains every concept, and highlights the most common mistakes so you can avoid them.
Why Cilium?
Every Kubernetes cluster needs to solve two networking problems:
JobTraditional ToolWith CiliumPod networking, give pods IP addresses and let them talk to each otherFlannel / CalicoCilium CNIService routing, route traffic from service IPs to the right podkube-proxy (iptables)Cilium eBPF
Cilium handles both jobs using eBPF, a modern Linux kernel technology that bypasses iptables entirely.
The result is:
- Faster networking
- Better observability
- A single component instead of two
What is eBPF?
eBPF (extended Berkeley Packet Filter) lets programs run directly in the Linux kernel without modifying kernel source code.
Cilium uses it to handle networking at the kernel level, which is much faster and more scalable than traditional iptables rules.
---|---|---|
| Pod networking , give pods IP addresses and let them talk to each other | Flannel / Calico | Cilium CNI |
| Service routing , route traffic from service IPs to the right pod | kube-proxy (iptables) | Cilium eBPF |
Cilium handles both jobs using eBPF, a modern Linux kernel technology that bypasses iptables entirely.
The result is:
- Faster networking
- Better observability
- A single component instead of two
What is eBPF?
eBPF (extended Berkeley Packet Filter) lets programs run directly in the Linux kernel without modifying kernel source code. Cilium uses it to handle networking at the kernel level, which is much faster and more scalable than traditional iptables rules.
Prerequisites
Make sure these are installed and running before starting:
# Check Docker is running
docker info | grep "Server Version"
# Check vCluster CLI version (needs v0.31+)
vcluster version
# Check Helm is installed
helm version --short
# Check kubectl is installed
kubectl version --client --short
If anything is missing:
# Install vCluster CLI on Mac
brew install loft-sh/tap/vcluster
# Install Helm on Mac
brew install helm
# Install kubectl on Mac
brew install kubectl
Step 1 , Switch to Docker Driver
vcluster use driver docker
This tells the vCluster CLI to create clusters as Docker containers on your machine, activating vind mode.
By default, the CLI deploys vClusters into an existing Kubernetes cluster using Helm. Switching to the Docker driver changes this behavior completely.
Optional:
Start the vCluster Platform UI with:
vcluster platform start
This gives you a web interface to manage clusters visually, but vind works perfectly without it.
Step 2 , Create the vind Configuration
Save this as vind-cilium.yaml:
experimental:
docker:
nodes:
- name: "worker-1"
- name: "worker-2"
deploy:
kubeProxy:
enabled: false
cni:
flannel:
enabled: false
What each setting does
experimental.docker.nodes- Defines two extra worker node containers
- vind also creates a control plane container automatically
deploy.kubeProxy.enabled: false- Disables kube-proxy
- Cilium replaces service routing using eBPF
deploy.cni.flannel.enabled: false- Disables Flannel, the default CNI
- Cilium handles pod networking instead
Important:
You cannot have two components doing the same networking job.
If you leave Flannel or kube-proxy enabled alongside Cilium, they will conflict and cause broken pod connectivity or service routing failures.
Step 3 , Create the vind Cluster
vcluster create cilium-vind -f vind-cilium.yaml
This creates a Kubernetes cluster with:
- One control plane
- Two worker containers
Verify they are running:
docker ps | grep vcluster
Expected output:
vcluster.cp.cilium-vind
vcluster.node.cilium-vind.worker-1
vcluster.node.cilium-vind.worker-2
Check node status:
kubectl get nodes
Expected output:
NAME STATUS ROLES
cilium-vind NotReady control-plane,master
worker-1 NotReady <none>
worker-2 NotReady <none>
NotReady is expected.
We disabled Flannel, so there is no CNI running yet.
Kubernetes nodes stay NotReady until a CNI plugin is installed.
Step 4 , Find the API Server IP
This is the most critical step and the most commonly missed one.
kubectl get endpoints kubernetes -n default
Expected output:
NAME ENDPOINTS
kubernetes 172.20.0.2:8443
When kube-proxy is disabled, Cilium cannot automatically find the Kubernetes API server.
Normally kube-proxy injects these environment variables into every pod:
KUBERNETES_SERVICE_HOSTKUBERNETES_SERVICE_PORT
With kube-proxy disabled, those variables do not exist.
Cilium needs to connect to the API server during startup.
If you give it a DNS name instead of a direct IP, it will fail because DNS itself requires a CNI to work.
The Chicken and Egg Problem
Cilium needs DNS to find the API server.
DNS needs a CNI to work.
Cilium is the CNI.
Result:
Init:0/6 forever
The fix:
Use a direct IP address instead of DNS.
Official docs:
Step 5 , Install Cilium
# Add the Cilium Helm repo
helm repo add cilium https://helm.cilium.io
helm repo update
# Install Cilium
helm install cilium cilium/cilium \
--version 1.16.0 \
--namespace kube-system \
--set kubeProxyReplacement=true \
--set k8sServiceHost=172.20.0.2 \
--set k8sServicePort=8443 \
--set image.pullPolicy=IfNotPresent \
--set ipam.mode=kubernetes \
--set envoy.enabled=false
Important flags
FlagWhat it doeskubeProxyReplacement=trueCilium fully replaces kube-proxyk8sServiceHost=172.20.0.2Direct IP of the API serverk8sServicePort=8443API server portipam.mode=kubernetesUses Kubernetes pod CIDRsenvoy.enabled=falseDisables Envoyimage.pullPolicy=IfNotPresentReuses cached images
Watch rollout status:
time kubectl -n kube-system rollout status ds/cilium
Expected output:
Waiting for daemon set "cilium" rollout to finish...
daemon set "cilium" successfully rolled out
Step 6 , Verify the Installation
1. Check all pods are healthy
kubectl get pods -n kube-system
Expected output:
cilium-xxxxx 1/1 Running
cilium-yyyyy 1/1 Running
cilium-zzzzz 1/1 Running
cilium-operator-xxxxx 1/1 Running
coredns-xxxxxxxxx 1/1 Running
2. Check nodes are now Ready
kubectl get nodes
Expected output:
NAME STATUS ROLES
cilium-vind Ready control-plane,master
worker-1 Ready
worker-2 Ready
3. Verify kube-proxy is gone
kubectl get pods -n kube-system | grep kube-proxy
Should return nothing.
4. Test DNS resolution
kubectl run dns-test \
--image=busybox:1.28 \
--restart=Never \
-- sleep 300
kubectl wait pod dns-test \
--for=condition=Ready \
--timeout=90s
kubectl exec dns-test -- \
nslookup kubernetes.default.svc.cluster.local
Expected output:
Server: 10.109.x.x
Name: kubernetes.default.svc.cluster.local
Address 1: 10.96.0.1
5. Test pod-to-pod networking
kubectl run nginx --image=nginx --restart=Never
kubectl wait pod nginx \
--for=condition=Ready \
--timeout=90s
kubectl get pod nginx -o wide
# Replace with actual pod IP
kubectl exec dns-test -- \
wget -O- http://<NGINX_POD_IP> --timeout=5
Expected output:
Connecting to 10.244.x.xxx
Welcome to nginx!
Cleanup:
kubectl delete pod dns-test nginx
Common Mistakes to Avoid
Mistake 1: Using a DNS name for k8sServiceHost
Using:
kubernetes.default.svc.cluster.local
for k8sServiceHost will cause Cilium to get stuck in:
Init:0/6
Always use the direct IP from:
kubectl get endpoints kubernetes -n default
Mistake 2: Not setting k8sServiceHost at all
Leaving k8sServiceHost unset when kube-proxy is disabled causes the same startup failure.
Without kube-proxy, the KUBERNETES_SERVICE_HOST environment variable is never injected into pods.
Mistake 3: Forgetting to disable Flannel and kube-proxy
Running Cilium alongside Flannel and kube-proxy causes conflicts.
Always set:
deploy:
kubeProxy:
enabled: false
cni:
flannel:
enabled: false
before installing Cilium.
Key Takeaway
When kube-proxy is disabled, you MUST set:
k8sServiceHost
to the direct IP address of your API server.
Find it with:
kubectl get endpoints kubernetes -n default
Always use the IP, never a DNS name.
This single setting is the difference between:
- Cilium starting in under 8 seconds
- Cilium stuck forever in
Init:0/6
References
- Official Cilium kube-proxy-free docs
- vind official page
- Replacing kind with vind , A Deep Dive
- Cilium Helm chart reference
No items found.
Share:
Ready to take vCluster for a spin?
Deploy your first virtual cluster today.
