Skip to main content
Version: main 🚧

Private Nodes

Enterprise-Only Feature

This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.

Enabling the private nodes feature in vCluster will turn off syncing of resources and instead transform vCluster into a regular hosted control plane you can join regular worker nodes into. In this case, the control plane will still run as a pod in a host cluster, but actual worker nodes need to be joined into the vCluster for it to be able to schedule workloads to. As soon as nodes are joined into the vCluster, vCluster can automatically upgrade the nodes and take over the lifecycle of them.

To allow actual nodes to join the vCluster control plane, the vCluster can be exposed by a LoadBalancer and this endpoint is used for joining worker nodes. Worker nodes must be able to reach the exposed vCluster. vCluster does not need to reach the nodes directly and instead uses konnectivity to tunnel requests from the vCluster to the nodes (e.g. for kubectl logs etc.).


Overview

Comparison Private Nodes vs Resource Syncing​

For an overview of what is possible with private nodes mode vs regular resource syncing, see the following table:


Private NodesResource Syncing
Control Plane as PodsYesYes
Isolated Control PlaneYesYes
Install Custom CNIYesNo
Install Custom CSI DriversYesNo
Separate NetworkYesNo
Completely Isolated NodesYesNo
Reuse Host NodesNoYes
Reuse Host ControllersNoYes
See vCluster Resources in Host ClusterNoYes

Create vCluster with Private Nodes​

To start vCluster in private nodes mode use the following configuration:

# This enables private node mode
privateNodes:
  enabled: true

# Control Plane options
controlPlane:
  distro:
    k8s:
      image:
        tag: v1.31.2 # Kubernetes version you want to use
  service:
    spec:
      type: LoadBalancer # If you want to expose vCluster via LoadBalancer (recommended option)

# Networking configuration
networking:
  # Specify the pod cidr
  podCIDR: 10.64.0.0/16
  # Specify the service cidr
  serviceCIDR: 10.128.0.0/16

Enabling this mode will currently disable the following other configurations:

  • All syncing of resources to the host cluster will not work anymore
  • The scheduler is enabled by default
  • K3s distro is not supported for this mode
  • All integrations in the vcluster.yaml are disabled
  • Embedded CoreDNS is not supported
  • Replicated services is not supported
Deploy without the Platform

Follow this guide to create an access key and setup the vCluster for deployment via Helm or other tools.

Joining Worker Nodes into the vCluster​

vCluster currently supports two modes of joining nodes into it:

Join Worker Node via Kubeadm​

Before you can join actual nodes into the vCluster, first you need to create a token for the nodes to join. You can either use a single token for all nodes or create a token per node, this is up to you.

To create a new token, simply run the following command while connected to the vCluster:

vcluster token create

This will print a command similar to:

curl -sfLk https://<vcluster-endpoint>/node/join?token=<token> | sh -

Simply run the command to join a node into the vCluster:

Preparing node for Kubernetes installation...
Kubernetes version: v1.30.1
Installing Kubernetes binaries...
Enabling containerd and kubelet...
Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /etc/systemd/system/containerd.service.
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /etc/systemd/system/kubelet.service.
Starting containerd and kubelet...
Installation successful!
Joining node into cluster...
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
Using vanilla kubeadm

If you want to see the raw kubeadm join command you can use the --kubeadm flag instead.

Join Worker Nodes via Cluster API​

For automatically joining nodes via Cluster API, please setup the cluster via the following command to install the vCluster Cluster API provider:

clusterctl init --infrastructure vcluster

As soon as the cluster-api provider is running, you can install a provider to do the actual node creation, e.g. KubeVirt.

VCLUSTER_VERSION=v0.26.0-alpha.9

clusterctl generate cluster test --kubernetes-version v1.31.2 -n test --from https://raw.githubusercontent.com/loft-sh/cluster-api-provider-vcluster/refs/heads/main/hack/kubevirt/template.yaml

Then apply the manifests to the cluster and Cluster API should create the vCluster and join the worker nodes via the KubeVirt provider.

vCluster Control Plane upgrades​

Kubernetes Version Skew Policy applies

Keep in mind to not upgrade multiple minor versions at the same time for the control plane as outlined in the Kubernetes Version Skew Policy. Instead always upgrade a single minor version, wait until the cluster becomes healthy and then upgrade to the next minor version. E.g. v1.26 -> v1.27 -> v1.28

Upgrading the vCluster control plane is super simple and only requires exchanging the Kubernetes init container version, vCluster will then automatically upgrade the control plane and if configured automatically upgrade all the worker nodes. To exchange the Kubernetes version, specify the following in your vcluster.yaml:

...
controlPlane:
  statefulSet:
    image:
      tag: v1.31.1 # or any other kubernetes version
...

vCluster Worker upgrades​

Kubernetes Version Skew Policy applies

Keep in mind to follow the Kubernetes Version Skew Policy for worker upgrades.

There is two modes of how worker upgrades can be done:

  • Automatically by vCluster when the control-plane was updated (recommended)
  • Manually via vCluster CLI or kubeadm

Automatic Upgrades by vCluster​

Automatic worker node upgrades are enabled by default and vCluster will try to upgrade nodes as soon as it detects a mismatch between control-plane and worker nodes. By default vCluster will upgrade a single node at a time.

The upgrade pod will run directly on the node and do the following steps on the node:

  1. Download the Kubernetes binary bundle from the vCluster control plane
  2. Exchange kubeadm binary
  3. Run kubeadm upgrade node
  4. Cordon the node
  5. Exchange other binaries such as containerd, kubelet etc.
  6. Restart containerd and kubelet if needed
  7. Uncordon the node

You can also skip specific nodes from the automatic update by adding the label vcluster.loft.sh/skip-auto-upgrade=true on the node or specifying a nodeSelector via the vCluster config.

autoUpgrade required object pro​

AutoUpgrade holds configuration for auto upgrade.

enabled required boolean true pro​

Enabled defines if auto upgrade should be enabled.

image required string pro​

Image is the image for the auto upgrade. If empty defaults to the controlPlane.statefulSet.image.

imagePullPolicy required string pro​

ImagePullPolicy is the policy how to pull the image.

nodeSelector required object pro​

NodeSelector is the node selector for the auto upgrade. If empty will select all worker nodes.

bundleRepository required string pro​

BundleRepository is the repository to use for downloading the Kubernetes bundle. Defaults to https://github.com/loft-sh/kubernetes/releases/download

binariesPath required string pro​

BinariesPath is the base path for the kubeadm binaries. Defaults to /usr/local/bin

cniBinariesPath required string pro​

CNIBinariesPath is the base path for the CNI binaries. Defaults to /opt/cni/bin

concurrency required integer 1 pro​

Concurrency is the number of nodes that can be upgraded at the same time.

warning

Node upgrades usually shouldn't restart any pods, but depending on the Kubernetes version upgrade it might be possible.

Manual Upgrades​

You can either upgrade a node via vCluster CLI by running the following command:

vcluster node upgrade my-node

Or by manually using the Kubeadm Kubernetes guide from the official documentation.

Addons​

CNI​

vCluster will install Flannel by default, if you want to install your own custom CNI, you can disable that via:

privateNodes:
  cni:
    flannel:
      enabled: false

CoreDNS​

vCluster will install CoreDNS by default to allow cluster dns resolving of services and pods.

Konnectivity​

vCluster uses Konnectivity to establish a connection between control-plane and worker nodes for commands like kubectl logs, kubectl exec, kubectl port-forward etc. The Konnectivity server runs as part of the vCluster control plane, while an agent is deployed inside the vCluster. If not needed or wanted, you can disable Konnectivity in the vCluster config.

Kube Proxy​

vCluster installs Kube Proxy by default to ensure that services are configured on the nodes. Some CNI implement their own kube-proxy functionality, so you can optionally disable kube-proxy deployment in the vCluster config.

Local Path Provisioner​

vCluster installs Local Path Provisioner by default to allow stateful workloads within the vCluster.

Load Docker Image into a Node​

To load a local docker image into a worker node you can use the following command:

vcluster node load-image my-node --image nginx:latest

Config reference​

privateNodes required object pro​

PrivateNodes holds configuration for vCluster private nodes mode.

enabled required boolean false pro​

Enabled defines if dedicated nodes should be enabled.

kubeProxy required object pro​

KubeProxy holds dedicated kube proxy configuration.

enabled required boolean true pro​

Enabled defines if the kube proxy should be enabled.

image required string pro​

Image is the image for the kube-proxy.

imagePullPolicy required string pro​

ImagePullPolicy is the policy how to pull the image.

nodeSelector required object {} pro​

NodeSelector is the node selector for the kube-proxy.

priorityClassName required string pro​

PriorityClassName is the priority class name for the kube-proxy.

tolerations required object[] [] pro​

Tolerations is the tolerations for the kube-proxy.

extraEnv required object[] [] pro​

ExtraEnv is the extra environment variables for the kube-proxy.

extraArgs required string[] [] pro​

ExtraArgs are additional arguments to pass to the kube-proxy.

kubelet required object pro​

Kubelet holds dedicated kubelet configuration.

cgroupDriver required string pro​

CgroupDriver defines the cgroup driver to use for the kubelet.

cni required object pro​

CNI holds dedicated CNI configuration.

flannel required object pro​

Flannel holds dedicated Flannel configuration.

enabled required boolean true pro​

Enabled defines if Flannel should be enabled.

localPathProvisioner required object pro​

LocalPathProvisioner holds dedicated local path provisioner configuration.

enabled required boolean true pro​

Enabled defines if LocalPathProvisioner should be enabled.

autoUpgrade required object pro​

AutoUpgrade holds configuration for auto upgrade.

enabled required boolean true pro​

Enabled defines if auto upgrade should be enabled.

image required string pro​

Image is the image for the auto upgrade. If empty defaults to the controlPlane.statefulSet.image.

imagePullPolicy required string pro​

ImagePullPolicy is the policy how to pull the image.

nodeSelector required object pro​

NodeSelector is the node selector for the auto upgrade. If empty will select all worker nodes.

bundleRepository required string pro​

BundleRepository is the repository to use for downloading the Kubernetes bundle. Defaults to https://github.com/loft-sh/kubernetes/releases/download

binariesPath required string pro​

BinariesPath is the base path for the kubeadm binaries. Defaults to /usr/local/bin

cniBinariesPath required string pro​

CNIBinariesPath is the base path for the CNI binaries. Defaults to /opt/cni/bin

concurrency required integer 1 pro​

Concurrency is the number of nodes that can be upgraded at the same time.

networking required object pro​

Networking options related to the virtual cluster.

serviceCIDR required string pro​

ServiceCIDR holds the service cidr for the virtual cluster. This should only be set if privateNodes.enabled is true or vCluster cannot detect the host service cidr.

podCIDR required string 10.244.0.0/16 pro​

PodCIDR holds the pod cidr for the virtual cluster. This should only be set if privateNodes.enabled is true.

replicateServices required object pro​

ReplicateServices allows replicating services from the host within the virtual cluster or the other way around.

toHost required object[] pro​

ToHost defines the services that should get synced from virtual cluster to the host cluster. If services are synced to a different namespace than the virtual cluster is in, additional permissions for the other namespace are required.

from required string pro​

From is the service that should get synced. Can be either in the form name or namespace/name.

to required string pro​

To is the target service that it should get synced to. Can be either in the form name or namespace/name.

fromHost required object[] pro​

FromHost defines the services that should get synced from the host to the virtual cluster.

from required string pro​

From is the service that should get synced. Can be either in the form name or namespace/name.

to required string pro​

To is the target service that it should get synced to. Can be either in the form name or namespace/name.

resolveDNS required object[] pro​

ResolveDNS allows to define extra DNS rules. This only works if embedded coredns is configured.

hostname required string pro​

Hostname is the hostname within the vCluster that should be resolved from.

service required string pro​

Service is the virtual cluster service that should be resolved from.

namespace required string pro​

Namespace is the virtual cluster namespace that should be resolved from.

target required object pro​

Target is the DNS target that should get mapped to

hostname required string pro​

Hostname to use as a DNS target

ip required string pro​

IP to use as a DNS target

hostService required string pro​

HostService to target, format is hostNamespace/hostService

hostNamespace required string pro​

HostNamespace to target

vClusterService required string pro​

VClusterService format is hostNamespace/vClusterName/vClusterNamespace/vClusterService

advanced required object pro​

Advanced holds advanced network options.

clusterDomain required string cluster.local pro​

ClusterDomain is the Kubernetes cluster domain to use within the virtual cluster.

fallbackHostCluster required boolean false pro​

FallbackHostCluster allows to fallback dns to the host cluster. This is useful if you want to reach host services without any other modification. You will need to provide a namespace for the service, e.g. my-other-service.my-other-namespace

proxyKubelets required object pro​

ProxyKubelets allows rewriting certain metrics and stats from the Kubelet to "fake" this for applications such as prometheus or other node exporters.

byHostname required boolean true pro​

ByHostname will add a special vCluster hostname to the nodes where the node can be reached at. This doesn't work for all applications, e.g. Prometheus requires a node IP.

byIP required boolean true pro​

ByIP will create a separate service in the host cluster for every node that will point to virtual cluster and will be used to route traffic.

controlPlane required object pro​

Configure vCluster's control plane components and deployment.

distro required object pro​

Distro holds virtual cluster related distro options. A distro cannot be changed after vCluster is deployed.

k8s required object pro​

K8S holds K8s relevant configuration.

enabled required boolean false pro​

Enabled specifies if the K8s distro should be enabled. Only one distro can be enabled at the same time.

version required string pro​

[Deprecated] Version field is deprecated. Use controlPlane.distro.k8s.image.tag to specify the Kubernetes version instead.

apiServer required object pro​

APIServer holds configuration specific to starting the api server.

enabled required boolean true pro​

Enabled signals this container should be enabled.

command required string[] [] pro​

Command is the command to start the distro binary. This will override the existing command.

extraArgs required string[] [] pro​

ExtraArgs are additional arguments to pass to the distro binary.

controllerManager required object pro​

ControllerManager holds configuration specific to starting the controller manager.

enabled required boolean true pro​

Enabled signals this container should be enabled.

command required string[] [] pro​

Command is the command to start the distro binary. This will override the existing command.

extraArgs required string[] [] pro​

ExtraArgs are additional arguments to pass to the distro binary.

scheduler required object pro​

Scheduler holds configuration specific to starting the scheduler. Enable this via controlPlane.advanced.virtualScheduler.enabled

command required string[] [] pro​

Command is the command to start the distro binary. This will override the existing command.

extraArgs required string[] [] pro​

ExtraArgs are additional arguments to pass to the distro binary.

image required object pro​

Image is the distro image

registry required string ghcr.io pro​

Registry is the registry of the container image, e.g. my-registry.com or ghcr.io. This setting can be globally overridden via the controlPlane.advanced.defaultImageRegistry option. Empty means docker hub.

repository required string loft-sh/kubernetes pro​

Repository is the repository of the container image, e.g. my-repo/my-image

tag required string v1.32.1 pro​

Tag is the tag of the container image, e.g. latest. If set to the default, it will use the host Kubernetes version.

imagePullPolicy required string pro​

ImagePullPolicy is the pull policy for the distro image

env required object[] [] pro​

Env are extra environment variables to use for the main container and NOT the init container.

resources required object map[limits:map[cpu:100m memory:256Mi] requests:map[cpu:40m memory:64Mi]] pro​

Resources for the distro init container

securityContext required object {} pro​

Security options can be used for the distro init container

k3s required object pro​

[Deprecated] K3S holds K3s relevant configuration.

enabled required boolean false pro​

Enabled specifies if the K3s distro should be enabled. Only one distro can be enabled at the same time.

token required string pro​

Token is the K3s token to use. If empty, vCluster will choose one.

image required object pro​

Image is the distro image

registry required string pro​

Registry is the registry of the container image, e.g. my-registry.com or ghcr.io. This setting can be globally overridden via the controlPlane.advanced.defaultImageRegistry option. Empty means docker hub.

repository required string rancher/k3s pro​

Repository is the repository of the container image, e.g. my-repo/my-image

tag required string v1.32.1-k3s1 pro​

Tag is the tag of the container image, e.g. latest. If set to the default, it will use the host Kubernetes version.

imagePullPolicy required string pro​

ImagePullPolicy is the pull policy for the distro image

env required object[] pro​

Env are extra environment variables to use for the main container and NOT the init container.

resources required object map[limits:map[cpu:100m memory:256Mi] requests:map[cpu:40m memory:64Mi]] pro​

Resources for the distro init container

securityContext required object {} pro​

Security options can be used for the distro init container

command required string[] [] pro​

Command is the command to start the distro binary. This will override the existing command.

extraArgs required string[] [] pro​

ExtraArgs are additional arguments to pass to the distro binary.

backingStore required object pro​

BackingStore defines which backing store to use for virtual cluster. If not defined will use embedded database as a default backing store.

etcd required object pro​

Etcd defines that etcd should be used as the backend for the virtual cluster

embedded required object pro​

Embedded defines to use embedded etcd as a storage backend for the virtual cluster

enabled required boolean false pro​

Enabled defines if the embedded etcd should be used.

migrateFromDeployedEtcd required boolean false pro​

MigrateFromDeployedEtcd signals that vCluster should migrate from the deployed external etcd to embedded etcd.

snapshotCount required integer pro​

SnapshotCount defines the number of snapshots to keep for the embedded etcd. Defaults to 10000 if less than 1.

extraArgs required string[] [] pro​

ExtraArgs are additional arguments to pass to the embedded etcd.

deploy required object pro​

Deploy defines to use an external etcd that is deployed by the helm chart

enabled required boolean false pro​

Enabled defines that an external etcd should be deployed.

statefulSet required object pro​

StatefulSet holds options for the external etcd statefulSet.

enabled required boolean true pro​

Enabled defines if the statefulSet should be deployed

EnableServiceLinks for the StatefulSet pod

image required object pro​

Image is the image to use for the external etcd statefulSet

registry required string registry.k8s.io pro​

Registry is the registry of the container image, e.g. my-registry.com or ghcr.io. This setting can be globally overridden via the controlPlane.advanced.defaultImageRegistry option. Empty means docker hub.

repository required string etcd pro​

Repository is the repository of the container image, e.g. my-repo/my-image

tag required string 3.5.21-0 pro​

Tag is the tag of the container image, e.g. latest. If set to the default, it will use the host Kubernetes version.

imagePullPolicy required string pro​

ImagePullPolicy is the pull policy for the external etcd image

env required object[] [] pro​

Env are extra environment variables

extraArgs required string[] [] pro​

ExtraArgs are appended to the etcd command.

resources required object pro​

Resources the etcd can consume

limits required object pro​

Limits are resource limits for the container

requests required object map[cpu:20m memory:150Mi] pro​

Requests are minimal resources that will be consumed by the container

pods required object pro​

Pods defines extra metadata for the etcd pods.

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

highAvailability required object pro​

HighAvailability are high availability options

replicas required integer 1 pro​

Replicas are the amount of pods to use.

scheduling required object pro​

Scheduling options for the etcd pods.

nodeSelector required object {} pro​

NodeSelector is the node selector to apply to the pod.

affinity required object {} pro​

Affinity is the affinity to apply to the pod.

tolerations required object[] [] pro​

Tolerations are the tolerations to apply to the pod.

priorityClassName required string pro​

PriorityClassName is the priority class name for the the pod.

podManagementPolicy required string Parallel pro​

PodManagementPolicy is the statefulSet pod management policy.

topologySpreadConstraints required object[] [] pro​

TopologySpreadConstraints are the topology spread constraints for the pod.

security required object pro​

Security options for the etcd pods.

podSecurityContext required object {} pro​

PodSecurityContext specifies security context options on the pod level.

containerSecurityContext required object {} pro​

ContainerSecurityContext specifies security context options on the container level.

persistence required object pro​

Persistence options for the etcd pods.

volumeClaim required object pro​

VolumeClaim can be used to configure the persistent volume claim.

enabled required boolean true pro​

Enabled enables deploying a persistent volume claim.

accessModes required string[] [ReadWriteOnce] pro​

AccessModes are the persistent volume claim access modes.

retentionPolicy required string Retain pro​

RetentionPolicy is the persistent volume claim retention policy.

size required string 5Gi pro​

Size is the persistent volume claim storage size.

storageClass required string pro​

StorageClass is the persistent volume claim storage class.

volumeClaimTemplates required object[] [] pro​

VolumeClaimTemplates defines the volumeClaimTemplates for the statefulSet

addVolumes required object[] [] pro​

AddVolumes defines extra volumes for the pod

addVolumeMounts required object[] pro​

AddVolumeMounts defines extra volume mounts for the container

name required string pro​

This must match the Name of a Volume.

readOnly required boolean pro​

Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.

mountPath required string pro​

Path within the container at which the volume should be mounted. Must not contain ':'.

subPath required string pro​

Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).

mountPropagation required string pro​

mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.

subPathExpr required string pro​

Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive.

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

service required object pro​

Service holds options for the external etcd service.

enabled required boolean true pro​

Enabled defines if the etcd service should be deployed

annotations required object {} pro​

Annotations are extra annotations for the external etcd service

headlessService required object pro​

HeadlessService holds options for the external etcd headless service.

annotations required object {} pro​

Annotations are extra annotations for the external etcd headless service

external required object pro​

External defines to use a self-hosted external etcd that is not deployed by the helm chart

enabled required boolean false pro​

Enabled defines if the external etcd should be used.

endpoint required string pro​

Endpoint holds the endpoint of the external etcd server, e.g. my-example-service:2379

tls required object pro​

TLS defines the tls configuration for the external etcd server

caFile required string pro​

CaFile is the path to the ca file

certFile required string pro​

CertFile is the path to the cert file

keyFile required string pro​

KeyFile is the path to the key file

database required object pro​

Database defines that a database backend should be used as the backend for the virtual cluster. This uses a project called kine under the hood which is a shim for bridging Kubernetes and relational databases.

embedded required object pro​

Embedded defines that an embedded database (sqlite) should be used as the backend for the virtual cluster

enabled required boolean false pro​

Enabled defines if the database should be used.

dataSource required string pro​

DataSource is the kine dataSource to use for the database. This depends on the database format. This is optional for the embedded database. Examples:

  • mysql: mysql://username:password@tcp(hostname:3306)/k3s
  • postgres: postgres://username:password@hostname:5432/k3s
keyFile required string pro​

KeyFile is the key file to use for the database. This is optional.

certFile required string pro​

CertFile is the cert file to use for the database. This is optional.

caFile required string pro​

CaFile is the ca file to use for the database. This is optional.

extraArgs required string[] [] pro​

ExtraArgs are additional arguments to pass to Kine.

external required object pro​

External defines that an external database should be used as the backend for the virtual cluster

enabled required boolean false pro​

Enabled defines if the database should be used.

dataSource required string pro​

DataSource is the kine dataSource to use for the database. This depends on the database format. This is optional for the embedded database. Examples:

  • mysql: mysql://username:password@tcp(hostname:3306)/k3s
  • postgres: postgres://username:password@hostname:5432/k3s
keyFile required string pro​

KeyFile is the key file to use for the database. This is optional.

certFile required string pro​

CertFile is the cert file to use for the database. This is optional.

caFile required string pro​

CaFile is the ca file to use for the database. This is optional.

extraArgs required string[] [] pro​

ExtraArgs are additional arguments to pass to Kine.

connector required string pro​

Connector specifies a secret located in a connected vCluster Platform that contains database server connection information to be used by Platform to create a database and database user for the vCluster. and non-privileged user. A kine endpoint should be created using the database and user on Platform registration. This is optional.

coredns required object pro​

CoreDNS defines everything related to the coredns that is deployed and used within the vCluster.

enabled required boolean true pro​

Enabled defines if coredns is enabled

embedded required boolean false pro​

Embedded defines if vCluster will start the embedded coredns service within the control-plane and not as a separate deployment. This is a PRO feature.

security required object pro​

Security defines pod or container security context.

podSecurityContext required object {} pro​

PodSecurityContext specifies security context options on the pod level.

containerSecurityContext required object {} pro​

ContainerSecurityContext specifies security context options on the container level.

service required object pro​

Service holds extra options for the coredns service deployed within the virtual cluster

spec required object map[type:ClusterIP] pro​

Spec holds extra options for the coredns service

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

deployment required object pro​

Deployment holds extra options for the coredns deployment deployed within the virtual cluster

image required string pro​

Image is the coredns image to use

replicas required integer 1 pro​

Replicas is the amount of coredns pods to run.

nodeSelector required object {} pro​

NodeSelector is the node selector to use for coredns.

affinity required object {} pro​

Affinity is the affinity to apply to the pod.

tolerations required object[] [] pro​

Tolerations are the tolerations to apply to the pod.

resources required object pro​

Resources are the desired resources for coredns.

limits required object map[cpu:1000m memory:170Mi] pro​

Limits are resource limits for the container

requests required object map[cpu:20m memory:64Mi] pro​

Requests are minimal resources that will be consumed by the container

pods required object pro​

Pods is additional metadata for the coredns pods.

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

topologySpreadConstraints required object[] [map[labelSelector:map[matchLabels:map[k8s-app:vcluster-kube-dns]] maxSkew:1 topologyKey:kubernetes.io/hostname whenUnsatisfiable:DoNotSchedule]] pro​

TopologySpreadConstraints are the topology spread constraints for the CoreDNS pod.

overwriteConfig required string pro​

OverwriteConfig can be used to overwrite the coredns config

overwriteManifests required string pro​

OverwriteManifests can be used to overwrite the coredns manifests used to deploy coredns

priorityClassName required string pro​

PriorityClassName specifies the priority class name for the CoreDNS pods.

proxy required object pro​

Proxy defines options for the virtual cluster control plane proxy that is used to do authentication and intercept requests.

bindAddress required string 0.0.0.0 pro​

BindAddress under which vCluster will expose the proxy.

port required integer 8443 pro​

Port under which vCluster will expose the proxy. Changing port is currently not supported.

extraSANs required string[] [] pro​

ExtraSANs are extra hostnames to sign the vCluster proxy certificate for.

hostPathMapper required object pro​

HostPathMapper defines if vCluster should rewrite host paths.

enabled required boolean pro​

Enabled specifies if the host path mapper will be used

central required boolean pro​

Central specifies if the central host path mapper will be used

ingress required object pro​

Ingress defines options for vCluster ingress deployed by Helm.

enabled required boolean false pro​

Enabled defines if the control plane ingress should be enabled

host required string my-host.com pro​

Host is the host where vCluster will be reachable

pathType required string ImplementationSpecific pro​

PathType is the path type of the ingress

spec required object map[tls:[]] pro​

Spec allows you to configure extra ingress options.

annotations required object map[nginx.ingress.kubernetes.io/backend-protocol:HTTPS nginx.ingress.kubernetes.io/ssl-passthrough:true nginx.ingress.kubernetes.io/ssl-redirect:true] pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

service required object pro​

Service defines options for vCluster service deployed by Helm.

enabled required boolean true pro​

Enabled defines if the control plane service should be enabled

spec required object map[type:ClusterIP] pro​

Spec allows you to configure extra service options.

kubeletNodePort required integer 0 pro​

KubeletNodePort is the node port where the fake kubelet is exposed. Defaults to 0.

httpsNodePort required integer 0 pro​

HTTPSNodePort is the node port where https is exposed. Defaults to 0.

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

statefulSet required object pro​

StatefulSet defines options for vCluster statefulSet deployed by Helm.

highAvailability required object pro​

HighAvailability holds options related to high availability.

replicas required integer 1 pro​

Replicas is the amount of replicas to use for the statefulSet.

leaseDuration required integer 60 pro​

LeaseDuration is the time to lease for the leader.

renewDeadline required integer 40 pro​

RenewDeadline is the deadline to renew a lease for the leader.

retryPeriod required integer 15 pro​

RetryPeriod is the time until a replica will retry to get a lease.

resources required object pro​

Resources are the resource requests and limits for the statefulSet container.

limits required object map[ephemeral-storage:8Gi memory:2Gi] pro​

Limits are resource limits for the container

requests required object map[cpu:200m ephemeral-storage:400Mi memory:256Mi] pro​

Requests are minimal resources that will be consumed by the container

scheduling required object pro​

Scheduling holds options related to scheduling.

nodeSelector required object {} pro​

NodeSelector is the node selector to apply to the pod.

affinity required object {} pro​

Affinity is the affinity to apply to the pod.

tolerations required object[] [] pro​

Tolerations are the tolerations to apply to the pod.

priorityClassName required string pro​

PriorityClassName is the priority class name for the the pod.

podManagementPolicy required string Parallel pro​

PodManagementPolicy is the statefulSet pod management policy.

topologySpreadConstraints required object[] [] pro​

TopologySpreadConstraints are the topology spread constraints for the pod.

security required object pro​

Security defines pod or container security context.

podSecurityContext required object {} pro​

PodSecurityContext specifies security context options on the pod level.

containerSecurityContext required object map[allowPrivilegeEscalation:false runAsGroup:0 runAsUser:0] pro​

ContainerSecurityContext specifies security context options on the container level.

probes required object pro​

Probes enables or disables the main container probes.

livenessProbe required object pro​

LivenessProbe specifies if the liveness probe for the container should be enabled

enabled required boolean true pro​

Enabled defines if this option should be enabled.

readinessProbe required object pro​

ReadinessProbe specifies if the readiness probe for the container should be enabled

enabled required boolean true pro​

Enabled defines if this option should be enabled.

startupProbe required object pro​

StartupProbe specifies if the startup probe for the container should be enabled

enabled required boolean true pro​

Enabled defines if this option should be enabled.

persistence required object pro​

Persistence defines options around persistence for the statefulSet.

volumeClaim required object pro​

VolumeClaim can be used to configure the persistent volume claim.

enabled required string|boolean auto pro​

Enabled enables deploying a persistent volume claim. If auto, vCluster will automatically determine based on the chosen distro and other options if this is required.

accessModes required string[] [ReadWriteOnce] pro​

AccessModes are the persistent volume claim access modes.

retentionPolicy required string Retain pro​

RetentionPolicy is the persistent volume claim retention policy.

size required string 5Gi pro​

Size is the persistent volume claim storage size.

storageClass required string pro​

StorageClass is the persistent volume claim storage class.

volumeClaimTemplates required object[] [] pro​

VolumeClaimTemplates defines the volumeClaimTemplates for the statefulSet

dataVolume required object[] [] pro​

Allows you to override the dataVolume. Only works correctly if volumeClaim.enabled=false.

binariesVolume required object[] [map[emptyDir:map[] name:binaries]] pro​

BinariesVolume defines a binaries volume that is used to retrieve distro specific executables to be run by the syncer controller. This volume doesn't need to be persistent.

addVolumes required object[] [] pro​

AddVolumes defines extra volumes for the pod

addVolumeMounts required object[] pro​

AddVolumeMounts defines extra volume mounts for the container

name required string pro​

This must match the Name of a Volume.

readOnly required boolean pro​

Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.

mountPath required string pro​

Path within the container at which the volume should be mounted. Must not contain ':'.

subPath required string pro​

Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).

mountPropagation required string pro​

mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.

subPathExpr required string pro​

Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive.

EnableServiceLinks for the StatefulSet pod

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

pods required object pro​

Additional labels or annotations for the statefulSet pods.

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

image required object pro​

Image is the image for the controlPlane statefulSet container

registry required string ghcr.io pro​

Configure the registry of the container image, e.g. my-registry.com or ghcr.io It defaults to ghcr.io and can be overriding either by using this field or controlPlane.advanced.defaultImageRegistry

repository required string loft-sh/vcluster-pro pro​

Configure the repository of the container image, e.g. my-repo/my-image. It defaults to the vCluster pro repository that includes the optional pro modules that are turned off by default. If you still want to use the pure OSS build, use 'loft-sh/vcluster-oss' instead.

tag required string pro​

Tag is the tag of the container image, e.g. latest

imagePullPolicy required string pro​

ImagePullPolicy is the policy how to pull the image.

workingDir required string pro​

WorkingDir specifies in what folder the main process should get started.

command required string[] [] pro​

Command allows you to override the main command.

args required string[] [] pro​

Args allows you to override the main arguments.

env required object[] [] pro​

Env are additional environment variables for the statefulSet container.

dnsPolicy required string pro​

Set DNS policy for the pod.

dnsConfig required object pro​

Specifies the DNS parameters of a pod.

nameservers required string[] pro​

A list of DNS name server IP addresses. This will be appended to the base nameservers generated from DNSPolicy. Duplicated nameservers will be removed.

searches required string[] pro​

A list of DNS search domains for host-name lookup. This will be appended to the base search paths generated from DNSPolicy. Duplicated search paths will be removed.

options required object[] pro​

A list of DNS resolver options. This will be merged with the base options generated from DNSPolicy. Duplicated entries will be removed. Resolution options given in Options will override those that appear in the base DNSPolicy.

name required string pro​

Required.

value required string pro​

serviceMonitor required object pro​

ServiceMonitor can be used to automatically create a service monitor for vCluster deployment itself.

enabled required boolean false pro​

Enabled configures if Helm should create the service monitor.

labels required object {} pro​

Labels are the extra labels to add to the service monitor.

annotations required object {} pro​

Annotations are the extra annotations to add to the service monitor.

advanced required object pro​

Advanced holds additional configuration for the vCluster control plane.

defaultImageRegistry required string pro​

DefaultImageRegistry will be used as a prefix for all internal images deployed by vCluster or Helm. This makes it easy to upload all required vCluster images to a single private repository and set this value. Workload images are not affected by this.

virtualScheduler required object pro​

VirtualScheduler defines if a scheduler should be used within the virtual cluster or the scheduling decision for workloads will be made by the host cluster.

enabled required boolean false pro​

Enabled defines if this option should be enabled.

serviceAccount required object pro​

ServiceAccount specifies options for the vCluster control plane service account.

enabled required boolean true pro​

Enabled specifies if the service account should get deployed.

name required string pro​

Name specifies what name to use for the service account.

imagePullSecrets required object[] pro​

ImagePullSecrets defines extra image pull secrets for the service account.

name required string pro​

Name of the image pull secret to use.

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

workloadServiceAccount required object pro​

WorkloadServiceAccount specifies options for the service account that will be used for the workloads that run within the virtual cluster.

enabled required boolean true pro​

Enabled specifies if the service account for the workloads should get deployed.

name required string pro​

Name specifies what name to use for the service account for the virtual cluster workloads.

imagePullSecrets required object[] pro​

ImagePullSecrets defines extra image pull secrets for the workload service account.

name required string pro​

Name of the image pull secret to use.

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

headlessService required object pro​

HeadlessService specifies options for the headless service used for the vCluster StatefulSet.

annotations required object {} pro​

Annotations are extra annotations for this resource.

labels required object {} pro​

Labels are extra labels for this resource.

konnectivity required object pro​

Konnectivity holds dedicated konnectivity configuration. This is only available when privateNodes.enabled is true.

server required object pro​

Server holds configuration for the konnectivity server.

enabled required boolean true pro​

Enabled defines if the konnectivity server should be enabled.

extraArgs required string[] [] pro​

ExtraArgs are additional arguments to pass to the konnectivity server.

agent required object pro​

Agent holds configuration for the konnectivity agent.

enabled required boolean true pro​

Enabled defines if the konnectivity agent should be enabled.

replicas required integer 1 pro​

Replicas is the number of replicas for the konnectivity agent.

image required string pro​

Image is the image for the konnectivity agent.

imagePullPolicy required string pro​

ImagePullPolicy is the policy how to pull the image.

nodeSelector required object {} pro​

NodeSelector is the node selector for the konnectivity agent.

priorityClassName required string pro​

PriorityClassName is the priority class name for the konnectivity agent.

tolerations required object[] [] pro​

Tolerations is the tolerations for the konnectivity agent.

extraEnv required object[] [] pro​

ExtraEnv is the extra environment variables for the konnectivity agent.

extraArgs required string[] [] pro​

ExtraArgs are additional arguments to pass to the konnectivity agent.

globalMetadata required object pro​

GlobalMetadata is metadata that will be added to all resources deployed by Helm.

annotations required object {} pro​

Annotations are extra annotations for this resource.