Skip to main content
Version: main 🚧

privateNodes

The privateNodes key configures private nodes, including kubelet settings, node joining, Auto Nodes, automatic upgrades, and VPN. For how to configure it, see Private nodes.

privateNodes object ​

PrivateNodes holds configuration for vCluster private nodes mode.

enabled boolean false ​

Enabled defines if dedicated nodes should be enabled.

kubelet object ​

Kubelet holds kubelet configuration that is used for all nodes.

config object {} ​

Config is the config for the kubelet that will be merged into the default kubelet config. More information can be found here: https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/#kubelet-config-k8s-io-v1beta1-KubeletConfiguration

autoUpgrade object ​

AutoUpgrade holds configuration for auto upgrade.

enabled boolean true ​

Enabled defines if auto upgrade should be enabled.

image string ​

Image is the image for the auto upgrade pod started by vCluster. If empty defaults to the controlPlane.statefulSet.image.

imagePullPolicy string ​

ImagePullPolicy is the policy how to pull the image.

nodeSelector object ​

NodeSelector is the node selector for the auto upgrade. If empty will select all worker nodes.

binariesPath string ​

BinariesPath is the base path for the kubeadm binaries. Defaults to /usr/local/bin

cniBinariesPath string ​

CNIBinariesPath is the base path for the CNI binaries. Defaults to /opt/cni/bin

concurrency integer 1 ​

Concurrency is the number of nodes that can be upgraded at the same time.

podSecurityContext object {} ​

PodSecurityContext specifies security context options on the pod level for the upgrade pod.

containerSecurityContext object {} ​

ContainerSecurityContext specifies security context options on the container level for the upgrade container.

joinNode object ​

JoinNode holds configuration specifically used during joining the node (see "kubeadm join").

preInstallCommands string[] ​

PreInstallCommands are commands that will be executed before containerd, kubelet etc. is installed.

preJoinCommands string[] ​

PreJoinCommands are commands that will be executed before kubeadm join is executed.

postJoinCommands string[] ​

PostJoinCommands are commands that will be executed after kubeadm join is executed.

containerd object ​

Containerd holds configuration for the containerd join process.

enabled boolean true ​

Enabled defines if containerd should be installed and configured by vCluster.

registry object ​

Registry holds configuration for how containerd should be configured to use a registries.

configPath string ​

ConfigPath is the path to the containerd registry config.

mirrors {key: object} ​

Mirrors holds configuration for the containerd registry mirrors. E.g. myregistry.io:5000 or docker.io. See https://github.com/containerd/containerd/blob/main/docs/hosts.md for more details.

server string ​

Server is the fallback server to use for the containerd registry mirror. E.g. https://registry-1.docker.io. See https://github.com/containerd/containerd/blob/main/docs/hosts.md for more details.

caCert string[] ​

CACert are paths to CA certificates to use for the containerd registry mirror.

skipVerify boolean ​

SkipVerify is a boolean to skip the certificate verification for the containerd registry mirror and allows http connections.

capabilities string[] ​

Capabilities is a list of capabilities to enable for the containerd registry mirror. If empty, will use pull and resolve capabilities.

overridePath boolean ​

OverridePath is a boolean to override the path for the containerd registry mirror.

hosts object[] ​

Hosts holds configuration for the containerd registry mirror hosts. See https://github.com/containerd/containerd/blob/main/docs/hosts.md for more details.

server string ​

Server is the server to use for the containerd registry mirror host. E.g. http://192.168.31.250:5000.

caCert string[] ​

CACert are paths to CA certificates to use for the containerd registry mirror host.

skipVerify boolean ​

SkipVerify is a boolean to skip the certificate verification for the containerd registry mirror and allows http connections.

capabilities string[] ​

Capabilities is a list of capabilities to enable for the containerd registry mirror. If empty, will use pull and resolve capabilities.

overridePath boolean ​

OverridePath is a boolean to override the path for the containerd registry mirror.

auth {key: object} ​

Auth holds configuration for the containerd registry auth. See https://github.com/containerd/containerd/blob/main/docs/cri/registry.md#configure-registry-credentials for more details.

username string ​

Username is the username for the containerd registry.

password string ​

Password is the password for the containerd registry.

identityToken string ​

IdentityToken is the token for the containerd registry.

auth string ​

Auth is the auth config for the containerd registry.

pauseImage string ​

PauseImage is the image for the pause container.

caCertPath string ​

CACertPath is the path to the SSL certificate authority used to secure communications between node and control-plane. Defaults to "/etc/kubernetes/pki/ca.crt".

skipPhases string[] ​

SkipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the "kubeadm join --help" command.

nodeRegistration object ​

NodeRegistration holds configuration for the node registration similar to the kubeadm node registration.

criSocket string ​

CRI socket is the socket for the CRI.

kubeletExtraArgs object[] ​

KubeletExtraArgs passes through extra arguments to the kubelet. The arguments here are passed to the kubelet command line via the environment file kubeadm writes at runtime for the kubelet to source. This overrides the generic base-level configuration in the kubelet-config ConfigMap Flags have higher priority when parsing. These values are local and specific to the node kubeadm is executing on. An argument name in this list is the flag name as it appears on the command line except without leading dash(es). Extra arguments will override existing default arguments. Duplicate extra arguments are allowed.

name string ​

Name is the name of the argument.

value string ​

Value is the value of the argument.

taints object[] ​

Taints are additional taints to set for the kubelet.

key string ​

Required. The taint key to be applied to a node.

value string ​

The taint value corresponding to the taint key.

effect string ​

Required. The effect of the taint on pods that do not tolerate the taint. Valid effects are NoSchedule, PreferNoSchedule and NoExecute.

ignorePreflightErrors string[] ​

IgnorePreflightErrors provides a slice of pre-flight errors to be ignored when the current node is registered, e.g. 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.

imagePullPolicy string ​

ImagePullPolicy specifies the policy for image pulling during kubeadm "init" and "join" operations. The value of this field must be one of "Always", "IfNotPresent" or "Never". If this field is unset kubeadm will default it to "IfNotPresent", or pull the required images if not present on the host.

autoNodes object[] ​

AutoNodes stores auto nodes configuration.

provider required string ​

Provider is the node provider of the nodes in this pool.

properties object ​

Properties are the node provider properties. This is a simple key value map and can contain things like region, subscription, etc. that is then used by the node provider to create the nodes and node environment.

static object[] ​

Static defines static node pools. Static node pools have a fixed size and are not scaled automatically.

name required string ​

Name is the name of this static nodePool

properties object ​

Properties are extra properties that will be set directly on the node claim.

nodeTypeSelector object[] ​

NodeTypeSelector filters the types of nodes that can be provisioned by this pool. All requirements must be met for a node type to be eligible.

property required string ​

Property is the property on the node type to select.

operator string ​

Operator is the comparison operator, such as "In", "NotIn", "Exists". If empty, defaults to "In".

values string[] ​

Values is the list of values to use for comparison. This is mutually exclusive with value.

value string ​

Value is the value to use for comparison. This is mutually exclusive with values.

taints object[] ​

Taints are the taints to apply to the nodes in this pool.

key string ​

Required. The taint key to be applied to a node.

value string ​

The taint value corresponding to the taint key.

effect string ​

Required. The effect of the taint on pods that do not tolerate the taint. Valid effects are NoSchedule, PreferNoSchedule and NoExecute.

nodeLabels object ​

NodeLabels are the labels to apply to the nodes in this pool.

terminationGracePeriod string ​

TerminationGracePeriod is the maximum duration the controller will wait before forcefully deleting the pods on a node, measured from when deletion is first initiated.

Warning: this feature takes precedence over a Pod's terminationGracePeriodSeconds value, and bypasses any blocked PDBs or the karpenter.sh/do-not-disrupt annotation.

This field is intended to be used by cluster administrators to enforce that nodes can be cycled within a given time period. When set, drifted nodes will begin draining even if there are pods blocking eviction. Draining will respect PDBs and the do-not-disrupt annotation until the TGP is reached.

Karpenter will preemptively delete pods so their terminationGracePeriodSeconds align with the node's terminationGracePeriod. If a pod would be terminated without being granted its full terminationGracePeriodSeconds prior to the node timeout, that pod will be deleted at T = node timeout - pod terminationGracePeriodSeconds.

The feature can also be used to allow maximum time limits for long-running jobs which can delay node termination with preStop hooks. Defaults to 30s. Set to Never to wait indefinitely for pods to be drained.

quantity required integer ​

Quantity is the number of desired nodes in this pool.

profile string ​

Profile references a NodeProfile applied to nodes created by this pool.

dynamic object[] ​

Dynamic defines dynamic node pools. Dynamic node pools are scaled automatically based on the requirements within the cluster. Karpenter is used under the hood to handle the scheduling of the nodes.

name required string ​

Name is the name of this NodePool

properties object ​

Properties are extra properties that will be set directly on the node claim.

nodeTypeSelector object[] ​

NodeTypeSelector filters the types of nodes that can be provisioned by this pool. All requirements must be met for a node type to be eligible.

property required string ​

Property is the property on the node type to select.

operator string ​

Operator is the comparison operator, such as "In", "NotIn", "Exists". If empty, defaults to "In".

values string[] ​

Values is the list of values to use for comparison. This is mutually exclusive with value.

value string ​

Value is the value to use for comparison. This is mutually exclusive with values.

taints object[] ​

Taints are the taints to apply to the nodes in this pool.

key string ​

Required. The taint key to be applied to a node.

value string ​

The taint value corresponding to the taint key.

effect string ​

Required. The effect of the taint on pods that do not tolerate the taint. Valid effects are NoSchedule, PreferNoSchedule and NoExecute.

nodeLabels object ​

NodeLabels are the labels to apply to the nodes in this pool.

limits object ​

Limits specify the maximum resources that can be provisioned by this node pool, mapping to the 'limits' field in Karpenter's NodePool API.

disruption object ​

Disruption contains the parameters that relate to Karpenter's disruption logic

consolidateAfter string ​

ConsolidateAfter is the duration the controller will wait before attempting to terminate nodes that are underutilized. Refer to ConsolidationPolicy for how underutilization is considered.

consolidationPolicy string ​

ConsolidationPolicy describes which nodes Karpenter can disrupt through its consolidation algorithm. This policy defaults to "WhenEmptyOrUnderutilized" if not specified

budgets object[] ​

Budgets is a list of Budgets. If there are multiple active budgets, Karpenter uses the most restrictive value. If left undefined, this will default to one budget with a value to 10%.

nodes string ​

Nodes dictates the maximum number of NodeClaims owned by this NodePool that can be terminating at once. This is calculated by counting nodes that have a deletion timestamp set, or are actively being deleted by Karpenter. This field is required when specifying a budget.

schedule string ​

Schedule specifies when a budget begins being active, following the upstream cronjob syntax. If omitted, the budget is always active. Timezones are not supported.

duration string ​

Duration determines how long a Budget is active since each Schedule hit. Only minutes and hours are accepted, as cron does not work in seconds. If omitted, the budget is always active. This is required if Schedule is set.

terminationGracePeriod string ​

TerminationGracePeriod is the maximum duration the controller will wait before forcefully deleting the pods on a node, measured from when deletion is first initiated.

Warning: this feature takes precedence over a Pod's terminationGracePeriodSeconds value, and bypasses any blocked PDBs or the karpenter.sh/do-not-disrupt annotation.

This field is intended to be used by cluster administrators to enforce that nodes can be cycled within a given time period. When set, drifted nodes will begin draining even if there are pods blocking eviction. Draining will respect PDBs and the do-not-disrupt annotation until the TGP is reached.

Karpenter will preemptively delete pods so their terminationGracePeriodSeconds align with the node's terminationGracePeriod. If a pod would be terminated without being granted its full terminationGracePeriodSeconds prior to the node timeout, that pod will be deleted at T = node timeout - pod terminationGracePeriodSeconds.

The feature can also be used to allow maximum time limits for long-running jobs which can delay node termination with preStop hooks. Defaults to 30s. Set to Never to wait indefinitely for pods to be drained.

expireAfter string ​

The amount of time a Node can live on the cluster before being removed

weight integer ​

Weight is the weight of this node pool.

profile string ​

Profile references a NodeProfile applied to nodes created by this pool.

vpn object ​

VPN holds configuration for the private nodes vpn. This can be used to connect the private nodes to the control plane or connect the private nodes to each other if they are not running in the same network. Platform connection is required for the vpn to work.

enabled boolean false ​

Enabled defines if the private nodes vpn should be enabled.

nodeToNode object ​

NodeToNode holds configuration for the node to node vpn. This can be used to connect the private nodes to each other if they are not running in the same network.

enabled boolean false ​

Enabled defines if the node to node vpn should be enabled.

daemon object ​

Daemon holds configuration for the private nodes daemon that is deployed on the nodes.

enabled boolean false ​

Enabled defines if the private nodes daemon should be enabled.

controlPlaneLoadBalancer object ​

ControlPlaneLoadBalancer holds configuration for the control plane load balancer. This is used to load balance the control plane traffic on the node to the control plane nodes. This is useful to achieve true high availability for the control plane without having to deploy a separate load balancer.

enabled boolean false ​

Enabled defines if the control plane load balancer should be enabled. The control plane load balancer is used to load balance the control plane traffic on the node to the control plane nodes.

kubeProxy boolean true ​

KubeProxy defines if the kube proxy should be proxied through the control plane load balancer as well.

port integer 11343 ​

Port defines the port for the control plane load balancer.

defaultProfile string ​

DefaultProfile is the name of a NodeProfile applied to nodes joining this vCluster that do not select one explicitly. Overridden by per-pool or per-node selection. Requires a platform connection to resolve.