Version: main 🚧

Private Nodes

Limited vCluster Tenancy Configuration Support

This feature is only available when using the following worker node types:

Private Nodes

Enterprise-Only Feature

This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.

Using private nodes is a tenancy model for vCluster where, instead of sharing the host cluster’s worker nodes, you assign individual worker nodes to a vCluster instance. These private nodes act as the vCluster’s worker nodes and are treated as worker nodes for the vCluster.

Because these nodes are real Kubernetes nodes, vCluster does not sync any resources to the host cluster as no host cluster worker nodes are used. All workloads run directly on the attached nodes as if they were native to the virtual cluster.

This approach can be understood as follows: the control plane creates a virtual Kubernetes context, while the workloads can run on separate physical nodes that exist entirely within that virtual context.

One key benefit of using private nodes is that vCluster can automatically manage the lifecycle of the worker nodes.

To allow nodes to join the virtual cluster, the cluster must be exposed and accessible. One option is to expose a LoadBalancer service and use its endpoint for the worker nodes. vCluster does not need direct access to the nodes; instead, it uses Konnectivity to tunnel requests—such as kubectl logs—from the control plane to the nodes.

vcluster.yaml configuration

To create a vCluster with private nodes, you'll need to enable the feature as well as set some other configuration options in preparation for worker nodes to join.

Enable private nodes
# Enable private nodes
privateNodes:
  enabled: true

# vCluster control plane options
controlPlane:
  distro:
    k8s:
      image:
        tag: v1.31.2 # Kubernetes version you want to use
  service:
    spec:
      type: LoadBalancer # If you want to expose vCluster using LoadBalancer (Recommended option)

# Networking configuration
networking:
  # Specify the pod CIDR
  podCIDR: 10.64.0.0/16
  # Specify the service CIDR
  serviceCIDR: 10.128.0.0/16

Scheduling enabled by default

When private nodes are enabled, scheduling is handled by the virtual cluster and the virtual scheduler is automatically enabled. You do not need to manually configure this in your vcluster.yaml.

Virtual scheduler automatically enabled with private nodes
# Enabling private nodes enables the virtual scheduler
# Disabling the virtual scheduler is not possible
controlPlane:
  advanced:
    virtualScheduler:
      enabled: true

Other vCluster feature limitations

When private nodes are enabled, certain vCluster features are automatically disabled or unavailable. If you include these options in your vcluster.yaml, they are ignored or might cause configuration errors.

The following features are not available when using private nodes:

sync.* - No resource syncing between virtual and host clusters
integrations.* - Integrations depend on syncing functionality
networking.replicateServices - Services are not replicated to host
controlPlane.distro.k3s - Only standard Kubernetes (k8s) is supported
controlPlane.coredns.embedded: true - Embedded CoreDNS conflicts with custom CNI
controlPlane.advanced.virtualScheduler.enabled: false - Virtual scheduler cannot be disabled
sleepMode.* - No ability to sleep workloads or control plane

Unsupported vcluster.yaml options with private nodes
# These configurations are NOT supported with private nodes

# Resource syncing between virtual and host clusters is disabled
sync:
  services:
    enabled: false  # Services cannot be synced to host cluster
  secrets:
    enabled: false  # Secrets cannot be synced to host cluster
                    # All other sync options (pods, configmaps, etc.) are also disabled

# Platform integrations require syncing functionality
integrations:
  metricsServer:
    enabled: false  # Metrics server integration not supported
                    # All other integrations are disabled due to sync dependency

# Service replication to host cluster is not available
networking:
  replicateServices:
    enabled: false  # Services run entirely within virtual cluster

# Distribution restrictions
controlPlane:
  distro:
    k3s:
      enabled: false  # k3s distribution not supported
    k8s:
      enabled: true   # Only standard Kubernetes works

  # DNS configuration limitations
  coredns:
    embedded: false   # Embedded CoreDNS conflicts with custom CNI options
  advanced:
    # Virtual scheduler is required for workload placement
    virtualScheduler:
      enabled: true   # Always enabled (cannot be disabled)
  # Host Path Mapper is not supported
  hostPathMapper:
    enabled: false

# Sleep mode is not available
sleepMode:
  enabled: false

# Isolated Control Plane mode is not supported
experimental:
  isolatedControlPlane:
    enabled: false

Configure addons

vCluster supports addons that extend the functionality of your virtual cluster. You can configure these addons during deployment to adjust networking, observability, and other features for your environment and requirements.

CNI

vCluster installs Flannel by default. You can install your own CNI with the following.

Disable default Flannel CNI
privateNodes:
  cni:
    flannel:
      enabled: false

CoreDNS

vCluster installs CoreDNS by default to allow cluster DNS resolving of services and pods.

Konnectivity

vCluster uses Konnectivity to establish a connection between the control plane and worker nodes for commands such as kubectl logs, kubectl exec, kubectl port-forward. The Konnectivity server runs as part of the vCluster control plane, while an agent is deployed inside the vCluster. If it is not needed or wanted, you can disable Konnectivity in the vCluster config.

Kube proxy

vCluster installs Kube Proxy by default to ensure that services are configured on the nodes. Some CNI implement their own kube-proxy functionality. Optionally, you can disable kube-proxy deployment in the vCluster config.

Local Path Provisioner

vCluster installs the Local Path Provisioner by default to allow stateful workloads within the vCluster.

Config reference

`privateNodes` required object

PrivateNodes holds configuration for vCluster private nodes mode.

`enabled` required boolean false

Enabled defines if dedicated nodes should be enabled.

`importNodeBinaries` required boolean true

ImportNodeBinaries defines to use the loft-sh/kubernetes:VERSION-full image to also copy the node binaries to the control plane. This allows upgrades and joining new nodes into the cluster without having to download the binaries from the internet.

`kubelet` required object

Kubelet holds kubelet configuration that is used for all nodes.

`cgroupDriver` required string

CgroupDriver defines the cgroup driver to use for the kubelet.

`autoUpgrade` required object

AutoUpgrade holds configuration for auto upgrade.

`enabled` required boolean true

Enabled defines if auto upgrade should be enabled.

`image` required string

Image is the image for the auto upgrade pod started by vCluster. If empty defaults to the controlPlane.statefulSet.image.

`imagePullPolicy` required string

ImagePullPolicy is the policy how to pull the image.

`nodeSelector` required object

NodeSelector is the node selector for the auto upgrade. If empty will select all worker nodes.

`bundleRepository` required string

BundleRepository is the repository to use for downloading the Kubernetes bundle. Defaults to https://github.com/loft-sh/kubernetes/releases/download

`binariesPath` required string

BinariesPath is the base path for the kubeadm binaries. Defaults to /usr/local/bin

`cniBinariesPath` required string

CNIBinariesPath is the base path for the CNI binaries. Defaults to /opt/cni/bin

`concurrency` required integer 1

Concurrency is the number of nodes that can be upgraded at the same time.

`joinNode` required object

JoinNode holds configuration specifically used during joining the node (see "kubeadm join").

`preJoinCommands` required string[]

PreJoinCommands are commands that will be executed before the join process starts.

`postJoinCommands` required string[]

PostJoinCommands are commands that will be executed after the join process starts.

`containerd` required object

Containerd holds configuration for the containerd join process.

`enabled` required boolean true

Enabled defines if containerd should be installed and configured by vCluster.

`registry` required object

Registry holds configuration for how containerd should be configured to use a registries.

`configPath` required string

ConfigPath is the path to the containerd registry config.

`mirrors` required {key: object}

Mirrors holds configuration for the containerd registry mirrors. E.g. myregistry.io:5000 or docker.io. See https://github.com/containerd/containerd/blob/main/docs/hosts.md for more details.

`server` required string

Server is the fallback server to use for the containerd registry mirror. E.g. https://registry-1.docker.io. See https://github.com/containerd/containerd/blob/main/docs/hosts.md for more details.

`caCert` required string[]

CACert are paths to CA certificates to use for the containerd registry mirror.

`skipVerify` required boolean

SkipVerify is a boolean to skip the certificate verification for the containerd registry mirror and allows http connections.

`capabilities` required string[]

Capabilities is a list of capabilities to enable for the containerd registry mirror. If empty, will use pull and resolve capabilities.

`hosts` required object[]

Hosts holds configuration for the containerd registry mirror hosts. See https://github.com/containerd/containerd/blob/main/docs/hosts.md for more details.

`server` required string

Server is the server to use for the containerd registry mirror host. E.g. http://192.168.31.250:5000.

`caCert` required string[]

CACert are paths to CA certificates to use for the containerd registry mirror host.

`skipVerify` required boolean

SkipVerify is a boolean to skip the certificate verification for the containerd registry mirror and allows http connections.

`capabilities` required string[]

Capabilities is a list of capabilities to enable for the containerd registry mirror. If empty, will use pull and resolve capabilities.

`importImages` required string[]

ImportImages is a list of images to import into the containerd registry from local files. If the path is a folder, all files that end with .tar or .tar.gz in the folder will be imported.

`pauseImage` required string

PauseImage is the image for the pause container.

`caCertPath` required string

CACertPath is the path to the SSL certificate authority used to secure communications between node and control-plane. Defaults to "/etc/kubernetes/pki/ca.crt".

`skipPhases` required string[]

SkipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the "kubeadm join --help" command.

`nodeRegistration` required object

NodeRegistration holds configuration for the node registration similar to the kubeadm node registration.

`criSocket` required string

CRI socket is the socket for the CRI.

`kubeletExtraArgs` required object[]

KubeletExtraArgs passes through extra arguments to the kubelet. The arguments here are passed to the kubelet command line via the environment file kubeadm writes at runtime for the kubelet to source. This overrides the generic base-level configuration in the kubelet-config ConfigMap Flags have higher priority when parsing. These values are local and specific to the node kubeadm is executing on. An argument name in this list is the flag name as it appears on the command line except without leading dash(es). Extra arguments will override existing default arguments. Duplicate extra arguments are allowed.

`name` required string

Name is the name of the argument.

`value` required string

Value is the value of the argument.

`taints` required object[]

Taints are additional taints to set for the kubelet.

`key` required string

Required. The taint key to be applied to a node.

`value` required string

The taint value corresponding to the taint key.

`effect` required string

Required. The effect of the taint on pods that do not tolerate the taint. Valid effects are NoSchedule, PreferNoSchedule and NoExecute.

`ignorePreflightErrors` required string[]

IgnorePreflightErrors provides a slice of pre-flight errors to be ignored when the current node is registered, e.g. 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.

`imagePullPolicy` required string

ImagePullPolicy specifies the policy for image pulling during kubeadm "init" and "join" operations. The value of this field must be one of "Always", "IfNotPresent" or "Never". If this field is unset kubeadm will default it to "IfNotPresent", or pull the required images if not present on the host.

vcluster.yaml configuration​

Scheduling enabled by default​

Other vCluster feature limitations​

Configure addons​

CNI​

CoreDNS​

Konnectivity​

Kube proxy​

Local Path Provisioner​

Config reference​

privateNodes required object ​

enabled required boolean false ​

importNodeBinaries required boolean true ​

kubelet required object ​

cgroupDriver required string ​

autoUpgrade required object ​

enabled required boolean true ​

image required string ​

imagePullPolicy required string ​

nodeSelector required object ​

bundleRepository required string ​

binariesPath required string ​

cniBinariesPath required string ​

concurrency required integer 1 ​

joinNode required object ​

preJoinCommands required string[] ​

postJoinCommands required string[] ​

containerd required object ​

enabled required boolean true ​

registry required object ​

configPath required string ​

mirrors required {key: object} ​

server required string ​

caCert required string[] ​

skipVerify required boolean ​

capabilities required string[] ​

hosts required object[] ​

server required string ​

caCert required string[] ​

skipVerify required boolean ​

capabilities required string[] ​

importImages required string[] ​

pauseImage required string ​

caCertPath required string ​

skipPhases required string[] ​

nodeRegistration required object ​

criSocket required string ​

kubeletExtraArgs required object[] ​

name required string ​

value required string ​

taints required object[] ​

key required string ​

value required string ​

effect required string ​

ignorePreflightErrors required string[] ​

imagePullPolicy required string ​

vcluster.yaml configuration

Scheduling enabled by default

Other vCluster feature limitations

Configure addons

CNI

CoreDNS

Konnectivity

Kube proxy

Local Path Provisioner

Config reference

`privateNodes` required object

`enabled` required boolean false

`importNodeBinaries` required boolean true

`kubelet` required object

`cgroupDriver` required string

`autoUpgrade` required object

`enabled` required boolean true

`image` required string

`imagePullPolicy` required string

`nodeSelector` required object

`bundleRepository` required string

`binariesPath` required string

`cniBinariesPath` required string

`concurrency` required integer 1

`joinNode` required object

`preJoinCommands` required string[]

`postJoinCommands` required string[]

`containerd` required object

`enabled` required boolean true

`registry` required object

`configPath` required string

`mirrors` required {key: object}

`server` required string

`caCert` required string[]

`skipVerify` required boolean

`capabilities` required string[]

`hosts` required object[]

`server` required string

`caCert` required string[]

`skipVerify` required boolean

`capabilities` required string[]

`importImages` required string[]

`pauseImage` required string

`caCertPath` required string

`skipPhases` required string[]

`nodeRegistration` required object

`criSocket` required string

`kubeletExtraArgs` required object[]

`name` required string

`value` required string

`taints` required object[]

`key` required string

`value` required string

`effect` required string

`ignorePreflightErrors` required string[]

`imagePullPolicy` required string