Pods

Limited vCluster Tenancy Configuration Support

This feature is only available when using the following worker node types:

Host Nodes

By default, this is enabled.

Sync all Pod resources, including ephemeral containers, from the virtual cluster to the host cluster.

Automatically apply tolerations to all pods synced by vCluster

Kubernetes has a concept of Taints and Tolerations, which is used for controlling scheduling. If you have a use case that requires vCluster to sync all pods and automatically set a toleration on those pods, then you can achieve this with the enforceTolerations option. You can pass multiple toleration expressions, and the syncer adds them to every new pod that vCluster syncs.

sync:
  toHost:
    pods:
      enabled: true
      enforceTolerations:
        - example-key=example-value:NoSchedule
        - another-key:PreferNoSchedule

info

vCluster does not support setting the tolerationSeconds field of a toleration. If your use case requires this, open an issue in the vCluster repo on GitHub.

Replace container images when pods are synced

If certain images used within the virtual cluster are not accessible in the host cluster due to registry restrictions or security policies, translateImage can map these to equivalent, permitted images in the host cluster's registry.

sync:
  toHost:
    pods:
      enabled: true
      translateImage:
        "virtualcluster/image:tag": "hostcluster/alternative-image:tag"

Patches

Enterprise-Only Feature

This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.

Patches override the default resource syncing rules in your vCluster yaml configurations.

By default, vCluster syncs specific resources between virtual and host clusters. To modify the sync behavior, you can use patches to specify which fields to edit, exclude, or override during syncing.

For example, vCluster can mirror resources from the virtual cluster to the host cluster—any changes made in the virtual cluster are automatically applied to the host cluster. The same applies in the other direction if syncing is set up from host to virtual cluster.

vCluster supports two patch types:

• JavaScript expression patch: Uses JavaScript ES6 expressions to dynamically modify fields during syncing. You can define how a field changes when syncing from a virtual cluster to a host cluster, or from a host cluster to a virtual cluster.
• Reference patch: Modifies specific fields within a resource to point to different resources. If the referenced resource exists in the host cluster, vCluster automatically imports it into the virtual cluster. If the referenced resource exists in the virtual cluster and syncing is configured, vCluster can import it into the host cluster.

Using wildcards in patches

You can apply a wildcard, using an asterisk (*) in a specified path, to modify all elements of an array or object. For instance, spec.containers[*].name selects the name field of every container in the spec.containers array, and spec.containers[*].volumeMounts[*] selects all volumeMounts for each container. When using the asterisk (*) notation, vCluster applies changes individually to every element that matches the path.

JavaScript expression patch

A JavaScript expression patch allows you to use JavaScript ES6 expressions to change specific fields when syncing between virtual and host clusters. This is useful when modifying resource configurations to align with differing environments or naming conventions between clusters. If your clusters use different container name prefixes, a JavaScript expression patch can automatically update them.

JavaScript Runtime

vCluster uses Goja, a pure Go implementation of ECMAScript, to evaluate JavaScript expressions. This provides a secure, sandboxed environment for executing your patch expressions. Refer to Goja's documentation for details on supported JavaScript features and limitations.

You can define a path for spec.containers[*].name field in vcluster.yaml using the following configuration:

sync:
  toHost:
    pods:
      enabled: true
      patches:
      - path: spec.containers[*].name
        expression: '"my-prefix-"+value'
        # optional reverseExpression to reverse the change from the host cluster
        # reverseExpression: 'value.slice("my-prefix".length)'

In the example:

• The path targets the spec.containers[*].name field to override when syncing to the host cluster. The * wildcard applies the patch to each container individually.
• "'my-prefix-' + value" defines a JavaScript expression that prepends "my-prefix-" to the spec.containers[*].name field in the host cluster.

Reverse sync

You can use the reverseExpression field to define how to revert changes when syncing from the host cluster back to the virtual cluster. For example, add reverseExpression: {"value.slice('my-prefix'.length)"} to vcluster.yaml to remove the "my-prefix-" prefix when syncing back from the host cluster to the virtual cluster in the previous example.

To replace value with a hardcoded one, put the desired value in the quotation marks:

sync:
  toHost:
    pods:
      enabled: true
      patches:
      - path: spec.containers[*].name
        expression: '"my-value"'

Adding new keys in maps like annotations or labels to the resource is also possible with expressions. For example, to add a label to the resource in the host cluster, you can use:

sync:
  toHost:
    pods:
      enabled: true
      patches:
      - path: metadata.label["my-new-label"]
        expression: 'valueExists ? value + "-changed" : "new-label-value"'

The value variable in the case where the map entry does not exist is null. To distinguish between an entry that does not exist and an entry that exists with a null value, you can use the valueExists variable in your expression. The valueExists variable is true if the entry exists in the map, even if its value is null.

The keys can also be conditionally added based on the value returned by the expression. If the expression evaluates to null, the key is not added to the map. Here is an example of conditionally adding a label based on the existence of another label:

sync:
  toHost:
    pods:
      enabled: true
      patches:
      - path: metadata.labels["conditional-label"]
        expression: "context.virtualObject.metadata?.label?.['my-label'] ? 'my-value' : null"

The above example checks if the my-label label exists in the virtual object. If it does, it adds the conditional-label with the value my-value. If it does not exist, it does not add the label.

JavaScript expressions

The JavaScript expression patch supports a limited set of JavaScript features. For example, it does not support import statements or other Node.js-specific features. You can use standard JavaScript expressions, functions, and operators. The expression is evaluated in a sandboxed environment, so it cannot access external resources or the host system. Also, the JavaScript expression must be a valid ES6 expression. This means it should not contain any statements, only expressions that return a value.

A common use is to use ternary operators to conditionally modify values based on their existence or other criteria. Another common use is the optional chaining operator like in: value.myfield?.anotherfield || "default-value".

Additional functions available:

• atob(encodedData): Decodes a base64-encoded string and returns the decoded string. Ex: atob("bXktdmFsdWU=") returns my-value.
• btoa(stringToEncode): Encodes a string in base64 and returns the encoded string. Ex: btoa("my-value") returns bXktdmFsdWU=.
• virtualToHostDNS(virtualSvc): Translates a virtual service to its corresponding host FQDN. The domain must follow the format <name>.<namespace>.svc[.<cluster-domain>]. Ex: virtualToHostDNS(nginx.default.svc.cluster.local) returns nginx-x-default-x-vcluster.vcluster-ns.svc.cluster.local.

These functions are useful for encoding/decoding values or translating services when syncing between clusters:

sync:
  toHost:
    pods:
      enabled: true
      patches:
      - path: metadata.annotations["encoded-value"]
        expression: 'valueExists ? btoa(value) : null'
      - path: metadata.annotations["decoded-value"]
        expression: 'valueExists ? atob(value) : null'
      - path: metadata.annotations["referenced-service"]
        expression: 'valueExists ? virtualToHostDNS(value) : null'

Context variable

The context variable is an object supported in JavaScript expression patches, that provides access to virtual cluster data during syncing. The context object includes the following properties:

• context.vcluster.name: Name of the virtual cluster.
• context.vcluster.namespace: Namespace of the virtual cluster.
• context.vcluster.config: Configuration of the virtual cluster, basically vcluster.yaml merged with the defaults.
• context.hostObject: Host object (null if not available).
• context.virtualObject: Virtual object (null if not available).
• context.path: Matched path on the object, useful when using wildcard path selectors (*).

Reference patch

A reference patch links a field in one resource to another resource. During syncing, vCluster updates the reference and imports the linked resource from the virtual cluster to the host cluster or from the host cluster to the virtual cluster, depending on the sync direction and whether the resource exists.

You can use reference patches to share resources, such as Secrets or ConfigMaps, between clusters without manually recreating or duplicating them.

For example, if the host cluster contains a secret named "my-example-secret", vCluster automatically imports it into the virtual cluster.

Workloads in the virtual cluster can then use the secret without manual syncing.

You can sync between the virtual cluster and the host cluster by mapping spec.secretName to a secret in the host cluster:

sync:
  toHost:
    pods:
      enabled: true
      patches:
      - path: metadata.annotations["my-secret-ref"]
        reference:
          apiVersion: v1
          kind: Secret

In the example:

• The code uses a patch to add metadata.annotations["my-secret-ref"]
• It references a Secret in the host cluster using the patch and ensures pods in the host cluster links to the correct Secret.

::: Reference Patches with Namespace Syncing If you have enabled syncing namespaces, reference patches are only required if the namespace is part of the patch. You can use the namespacePath option to specify the path of the namespace of the reference. :::

Use secrets for ServiceAccount tokens

A host Pod requires a ServiceAccount token to communicate with the virtual clusters API. If you don't want to create these secrets in the host cluster, disable this option. vCluster then adds annotations to the pods.

Rewrite hosts

Applies only to Pods that have the field spec.subdomain set. In such Pods, the fqdn hostname (hostname -f) is constructed based on the host cluster namespace the vCluster runs in. This is usually not what applications expect as they are unaware of the host cluster.

If this option is enabled, vCluster injects an initContainer to override the Pod's /etc/hosts file to change the fqdn hostname to match the expected domain.

Configuration change

In vCluster version 0.27.0, the rewriteHosts.initContainer.image configuration changed from a string to an object format. This change provides granular control over the registry, repository, and tag of the image.

For more information on the new object structure, see the configuration reference.

Config reference

Do Not Disable

Disabling the syncing of this resource could cause the vCluster to not work properly.

pods required object

Pods defines if pods created within the virtual cluster should get synced to the host cluster.

enabled required boolean true

Enabled defines if pod syncing should be enabled.

translateImage required object {}

TranslateImage maps an image to another image that should be used instead. For example this can be used to rewrite a certain image that is used within the virtual cluster to be another image on the host cluster

enforceTolerations required string[] []

EnforceTolerations will add the specified tolerations to all pods synced by the virtual cluster.

useSecretsForSATokens required boolean false

UseSecretsForSATokens will use secrets to save the generated service account tokens by virtual cluster instead of using a pod annotation.

runtimeClassName required string

RuntimeClassName is the runtime class to set for synced pods.

priorityClassName required string

PriorityClassName is the priority class to set for synced pods.

rewriteHosts required object

RewriteHosts is a special option needed to rewrite statefulset containers to allow the correct FQDN. virtual cluster will add a small container to each stateful set pod that will initially rewrite the /etc/hosts file to match the FQDN expected by the virtual cluster.

enabled required boolean true

Enabled specifies if rewriting stateful set pods should be enabled.

initContainer required object

InitContainer holds extra options for the init container used by vCluster to rewrite the FQDN for stateful set pods.

image required object

Image is the image virtual cluster should use to rewrite this FQDN.

registry required string mirror.gcr.io

Registry is the registry of the container image, e.g. my-registry.com or ghcr.io. This setting can be globally overridden via the controlPlane.advanced.defaultImageRegistry option. Empty means docker hub.

repository required string library/alpine

Repository is the repository of the container image, e.g. my-repo/my-image

tag required string 3.20

Tag is the tag of the container image, and is the default version.

resources required object

Resources are the resources that should be assigned to the init container for each stateful set init container.

limits required object map[cpu:30m memory:64Mi]

Limits are resource limits for the container

requests required object map[cpu:30m memory:64Mi]

Requests are minimal resources that will be consumed by the container

patches required object[]

Patches patch the resource according to the provided specification.

path required string

Path is the path within the patch to target. If the path is not found within the patch, the patch is not applied.

expression required string

Expression transforms the value according to the given JavaScript expression.

reverseExpression required string

ReverseExpression transforms the value according to the given JavaScript expression.

reference required object

Reference treats the path value as a reference to another object and will rewrite it based on the chosen mode automatically. In single-namespace mode this will translate the name to "vxxxxxxxxx" to avoid conflicts with other names, in multi-namespace mode this will not translate the name.

apiVersion required string

APIVersion is the apiVersion of the referenced object.

apiVersionPath required string

APIVersionPath is optional relative path to use to determine the kind. If APIVersionPath is not found, will fallback to apiVersion.

kind required string

Kind is the kind of the referenced object.

kindPath required string

KindPath is the optional relative path to use to determine the kind. If KindPath is not found, will fallback to kind.

namePath required string

NamePath is the optional relative path to the reference name within the object.

namespacePath required string

NamespacePath is the optional relative path to the reference namespace within the object. If omitted or not found, namespacePath equals to the metadata.namespace path of the object.

labels required object

Labels treats the path value as a labels selector.

hybridScheduling required object

HybridScheduling is used to enable and configure hybrid scheduling for pods in the virtual cluster.

enabled required boolean false

Enabled specifies if hybrid scheduling is enabled.

hostSchedulers required string[] []

HostSchedulers is a list of schedulers that are deployed on the host cluster.