Skip to main content
Version: main 🚧

Priority classes

By default, this is disabled.

PriorityClass resources set which pods get scheduled first when resources are limited. High-priority pods can kick out low-priority pods to get resources.

Typically, each virtual cluster needs its own copy of every PriorityClass. This means creating the same PriorityClass multiple times-once per virtual cluster. If anything changes in your PriorityClass, you must update each copy in each virtual cluster. PriorityClass syncing eliminates this duplication and maintenance by making host cluster PriorityClass resources available to virtual clusters. You define priority classes once in the host cluster, and vCluster automatically makes them accessible to virtual clusters.

You can either sync all PriorityClass resources or configure syncing using label selectors. When a virtual cluster pod references a priorityClassName, vCluster validates that the corresponding PriorityClass exists in the virtual cluster through the syncing process. If the referenced PriorityClass is not synced (either because it does not exist in the host cluster or does not match your label selector), the pod is not scheduled on the host cluster.

Enable syncing​

When enabled, vCluster takes control of all PriorityClass resources in the virtual cluster. It only allows PriorityClass resources that are synced from the host cluster to exist.

This example configuration enables syncing all PriorityClass resources from the host cluster to the virtual cluster:

sync:
  fromHost:
    priorityClasses:
      enabled: true

If you try to create the PriorityClass resource directly in the virtual cluster, using for example kubectl create or kubectl apply, vCluster detects it and deletes it immediately. This prevents conflicts between locally-created PriorityClass resources and those synced from the host cluster, ensuring that only PriorityClass resources from the host cluster exist in the virtual cluster.

PriorityClass resource limitation

When enabled, PriorityClass resource creation can only occur in the host cluster. Any PriorityClass resources created in the virtual cluster will be deleted.

How syncing works​

When PriorityClass syncing is enabled, vCluster can use a label selector to control which PriorityClass and Pod resources are synchronized between the host and virtual clusters. This affects the resource types with separate unidirectional sync flows:

  • PriorityClass resources (host → virtual): vCluster syncs PriorityClass resources from the host cluster to the virtual cluster. You cannot create PriorityClass resources directly in the virtual cluster. If a selector is defined, only PriorityClass resources matching the selector will sync. If no selector is defined, all PriorityClass resources are synced.

  • Pod resources (virtual → host): vCluster syncs Pod resources from the virtual cluster to the host cluster only if any of the following is true:

    • No selector is defined for PriorityClass syncing, which allows all Pod resources to sync regardless of the priorityClassName.
    • The Pod's priorityClassName references a PriorityClass resource that matches the selector and is synced from the host cluster.
    • The Pod has an empty priorityClassName field.

The same selector controls both sync flows. The selector determines which PriorityClass resources are imported from the host cluster and which Pod resources can sync to the host cluster based on the referenced PriorityClass resource.

Use selectors to filter​

Selectors provide precise control over which PriorityClass resources get synced from the host cluster and which Pod resources gets synced to the host cluster. vCluster supports two types of selector criteria that follow standard Kubernetes label selector syntax.

Filter with matchLabels​

The matchLabels selector defines exact label key-value pairs that must be present on an PriorityClass for it to be synced. This provides straightforward filtering based on specific labels.

The following example syncs only PriorityClass resources with a environment: development label:

Example with matchLabels
sync:
  fromHost:
    priorityClasses:
      enabled: true
      selector:
        matchLabels:
          environment: development

Filter with matchExpressions​

The matchExpressions selector allows more flexible, set-based filtering with support for multiple operators:

  • In: Select all resources that has a specific key and the value is in the set of values
  • NotIn: Select all resources that has a specific key and the value is not in the set of values
  • Exists: Select all resources including a label with the key; no values are checked
  • DoesNotExist: Select all resources without a label with the key; no values are checked

The following example syncs PriorityClass resources where the key of the label is kubernetes.io/priority.class and the value of that label is either low or medium:

Example with matchExpressions
sync:
  fromHost:
    priorityClasses:
      enabled: true
      selector:
        matchExpressions:
          - key: kubernetes.io/priority.class
            operator: In
            values:
              - low
              - medium

Combined filter criteria​

Label conditions are additive meaning all specified label conditions must match for the PriorityClass resource to be selected to be synced. This means that if you define multiple matchLabels and matchExpressions, the PriorityClass resource must satisfy all criteria to be synced to the virtual cluster.

The following example syncs PriorityClass resources that match both label and expression criteria:

Example with matchLabels and matchExpressions
sync:
  fromHost:
    priorityClasses:
      enabled: true
      selector:
        matchLabels:
          environment: development
        matchExpressions:
          - key: kubernetes.io/priority.class
            operator: In
            values:
              - low
              - medium

In this example, the PriorityClass must have the following labels to sync to the virtual cluster:

  • environment: development
  • Either kubernetes.io/priority.class:low or kubernetes.io/priority.class:medium

Sync behavior considerations​

Resource lifecycle​

Synced PriorityClass resources function like any other Kubernetes resource in the virtual cluster. You can view them with kubectl get priorityclass and reference them in your Pod resource specifications with the priorityClassName field.

When you modify the PriorityClass resource in the host cluster, vCluster re-evaluates whether it still matches the selector criteria. If the PriorityClass continues to match, vCluster updates the corresponding resource in the virtual cluster to reflect the changes. If the PriorityClass no longer matches the selector criteria, vCluster removes it from the virtual cluster.

The selector system acts as both a resource filter and a validation mechanism. As a resource filter, it ensures that vCluster creates only PriorityClass resources matching the defined selector criteria. The selector also functions as a creation validation mechanism - when you create Pod resources in the virtual cluster, the resources can only reference PriorityClass resources that exist in the virtual cluster.

Orphaned resources​

When vCluster removes a synced PriorityClass from the virtual cluster due to selector changes or deletion from the host cluster, any Pod resources that reference it remain in the virtual and host clusters. These orphaned Pod resources stop receiving updates but vCluster does not automatically delete them to prevent unintended data loss. To remove these orphaned resources, you must delete them manually in the virtual and host clusters. This manual approach ensures that you maintain full control over resource cleanup and can verify that deletions are intentional.

Error handling and troubleshooting​

When the PriorityClass resource doesn't match the selector criteria during evaluation, vCluster logs a warning in the syncer pod's output to help with troubleshooting and monitoring. This logging provides visibility into which resources are being filtered out and why.

When you creates the Pod resource that references a PriorityClass not matching the selector criteria, several things occur to provide clear feedback:

  • The Pod fails to sync to the host cluster
  • vCluster records an event on the Pod resource in the virtual cluster
  • The event indicates that the specified PriorityClass is not available according to the current selector configuration

The error output appears as a Kubernetes event that you can view using kubectl describe. The event message clearly states that the Pod resource was not synced because the referenced PriorityClass does not match the defined selector criteria.

The error output looks like this:

Example error handling output
vcluster-virtual-cluster-1:~$ kubectl describe pod my-pod
Name:                       my-pod
Namespace:                  default
PriorityClass:     medium
Events:
  Type     Reason            Age   From                   Message
  ----     ------            ----  ----                    -------
  Warning  SyncWarning       10s   pod-syncer      did not sync pod "my-pod" to host because it does not match the selector under 'sync.fromHost.priorityClasses.selector'

Config reference​

priorityClasses required object pro​

PriorityClasses defines if priority classes classes should get synced from the host cluster to the virtual cluster, but not back.

enabled required boolean false pro​

Enabled defines if this option should be enabled.

patches required object[] pro​

Patches patch the resource according to the provided specification.

path required string pro​

Path is the path within the patch to target. If the path is not found within the patch, the patch is not applied.

expression required string pro​

Expression transforms the value according to the given JavaScript expression.

reverseExpression required string pro​

ReverseExpression transforms the value according to the given JavaScript expression.

reference required object pro​

Reference treats the path value as a reference to another object and will rewrite it based on the chosen mode automatically. In single-namespace mode this will translate the name to "vxxxxxxxxx" to avoid conflicts with other names, in multi-namespace mode this will not translate the name.

apiVersion required string pro​

APIVersion is the apiVersion of the referenced object.

apiVersionPath required string pro​

APIVersionPath is optional relative path to use to determine the kind. If APIVersionPath is not found, will fallback to apiVersion.

kind required string pro​

Kind is the kind of the referenced object.

kindPath required string pro​

KindPath is the optional relative path to use to determine the kind. If KindPath is not found, will fallback to kind.

namePath required string pro​

NamePath is the optional relative path to the reference name within the object.

namespacePath required string pro​

NamespacePath is the optional relative path to the reference namespace within the object. If omitted or not found, namespacePath equals to the metadata.namespace path of the object.

labels required object pro​

Labels treats the path value as a labels selector.

selector required object pro​

Selector defines the selector to use for the resource. If not set, all resources of that type will be synced.

matchLabels required object pro​

matchExpressions required object[] pro​

key required string pro​
operator required string pro​
values required string[] pro​