Skip to main content

Exposing vcluster (ingress etc.)

By default, vcluster is only reachable via port-forwarding in remote clusters. However, this means that you need access to the host cluster, where the vcluster is running, in order to access it. To directly access vcluster without port-forwarding, you can use one of the following methods:

Local Kubernetes Clusters

If you are using a local Kubernetes cluster, such as docker-desktop, rancher-desktop, KinD or minikube, vcluster will automatically connect to it without the need of port-forwarding.

LoadBalancer service#

The easiest way is to use the flag --expose in vcluster create to tell vcluster to use a LoadBalancer service:

# Create a new vcluster with a LoadBalancer
vcluster create my-vcluster --expose
# Run any kube command in the vcluster
kubectl get ns

Thats it, your vcluster is now externally reachable through a LoadBalancer service.

Manual LoadBalancer service creation#

Instead of using the built-in flag --expose, you can also create the following load-balancer.yaml for a vcluster called my-vcluster in the namespace my-vcluster yourself:

apiVersion: v1
kind: Service
metadata:
name: vcluster-loadbalancer
namespace: my-vcluster
spec:
selector:
app: vcluster
release: my-vcluster
ports:
- name: https
port: 443
targetPort: 8443
protocol: TCP
type: LoadBalancer

Create the resource in the namespace via:

kubectl apply -f load-balancer.yaml

Find out the external ip via kubectl get svc vcluster-loadbalancer -n my-vcluster:

kubectl get svc vcluster-loadbalancer -n my-vcluster
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
vcluster-loadbalancer LoadBalancer 10.68.9.239 x.x.x.x 443:32678/TCP 7m15s

Now create a values.yaml to create the vcluster with:

syncer:
extraArgs:
- --tls-san=x.x.x.x

Create the virtual cluster with:

vcluster create my-vcluster -n my-vcluster --connect=false -f values.yaml

Update the current kube config via:

# Update the current kube config (or use --update-current=false to create a separate one)
vcluster connect my-vcluster -n my-vcluster --server=https://x.x.x.x

Access the vcluster:

# Run any kube context command
kubectl get ns

NodePort service#

You can also expose the vcluster via a NodePort service. Create the following nodeport.yaml for a vcluster called my-vcluster in the namespace my-vcluster:

apiVersion: v1
kind: Service
metadata:
name: vcluster-nodeport
namespace: my-vcluster
spec:
selector:
app: vcluster
release: my-vcluster
ports:
- name: https
port: 443
targetPort: 8443
protocol: TCP
type: NodePort

Create the resource in the namespace via:

kubectl apply -f nodeport.yaml

Find out the external port via kubectl get svc vcluster-nodeport -n my-vcluster:

kubectl get svc vcluster-nodeport -n my-vcluster
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
vcluster-nodeport NodePort 10.68.9.75 <none> 443:31992/TCP 85s

Find out the node ips via kubectl get nodes -o wide:

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
gke-cluster-1-default-pool-8f0bb8bb-p6wx Ready <none> 6d v1.20.6-gke.1000 10.156.0.14 x.x.x.x Container-Optimized OS from Google 5.4.104+ containerd://1.4.3
gke-cluster-1-default-pool-8f0bb8bb-vl79 Ready <none> 6d v1.20.6-gke.1000 10.156.0.15 y.y.y.y Container-Optimized OS from Google 5.4.104+ containerd://1.4.3
gke-cluster-1-default-pool-8f0bb8bb-wpkp Ready <none> 6d v1.20.6-gke.1000 10.156.0.16 z.z.z.z Container-Optimized OS from Google 5.4.104+ containerd://1.4.3

Now create a values.yaml to create the vcluster with:

syncer:
extraArgs:
- --tls-san=x.x.x.x,y.y.y.y,z.z.z.z

Create the virtual cluster with:

vcluster create my-vcluster -n my-vcluster --connect=false -f values.yaml

Retrieve the kube config via:

vcluster connect my-vcluster -n my-vcluster --update-current=false --server=https://x.x.x.x

Access the vcluster:

export KUBECONFIG=./kubeconfig.yaml
# Run any kube context command
kubectl get ns

Ingress#

Make sure you have an ingress controller installed into your cluster. Create the following ingress.yaml for a vcluster called my-vcluster in the namespace my-vcluster:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
# We need the ingress to pass through ssl traffic to the vcluster
# This only works for the nginx-ingress (enable via --enable-ssl-passthrough
# https://kubernetes.github.io/ingress-nginx/user-guide/tls/#ssl-passthrough )
# for other ingress controllers please check their respective documentation.
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
name: vcluster-ingress
namespace: my-vcluster
spec:
ingressClassName: nginx # use your ingress class name
rules:
- host: my-vcluster.example.com
http:
paths:
- backend:
service:
name: my-vcluster
port:
number: 443
path: /
pathType: ImplementationSpecific

Create the resource in the namespace via:

kubectl apply -f ingress.yaml
Enable SSL Passthrough Feature

If you are using the ingress nginx controller, please make sure you have enabled the SSL passthrough feature as it is disabled by default.

SSL Passthrough required

In order for this ingress to work correctly, you will need to enable SSL passthrough as TLS termination has to happen at the vcluster level and not ingress controller level. If you cannot do that, please take a look below for using an ingress without ssl passthrough.

Now create a values.yaml to create the vcluster with:

syncer:
extraArgs:
- --tls-san=my-vcluster.example.com

Create the virtual cluster with:

vcluster create my-vcluster -n my-vcluster --connect=false -f values.yaml

Retrieve the kube config via:

vcluster connect my-vcluster -n my-vcluster --update-current=false --server=https://my-vcluster.example.com

Access the vcluster:

export KUBECONFIG=./kubeconfig.yaml
# Run any kubectl command
kubectl get ns

Ingress without SSL-Passthrough#

If you cannot configure your ingress controller to use ssl-passthrough, you can also create an ingress similar to this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/ssl-redirect: "true"
name: vcluster-ingress
namespace: my-vcluster
spec:
ingressClassName: nginx # use your ingress class name
rules:
- host: my-vcluster.example.com
http:
paths:
- backend:
service:
name: my-vcluster
port:
number: 443
path: /
pathType: ImplementationSpecific
tls:
- hosts: my-vcluster.example.com

With this configuration you will need to use service account authentication in order to connect as the ingress controller won't be able to resolve the client-cert and client-key which is used by default as authentication method. To create a kube config that uses a service account, please run the following command:

vcluster connect my-vcluster -n my-vcluster --server=https://my-vcluster.example.com --service-account admin --cluster-role cluster-admin --insecure

Then access the vcluster:

# Run any kubectl command
kubectl get ns

In-Cluster#

In order to access the virtual cluster from within the host cluster, you can directly connect to the vcluster service. Make sure you can access that service and then create a kube config in the following form:

vcluster connect my-vcluster -n my-vcluster --server=my-vcluster.my-vcluster --insecure --update-current=false

Now access the virtual cluster with:

export KUBECONFIG=./kubeconfig.yaml
# Run any kubectl command
kubectl get ns