Skip to main content

Storage

vcluster Persistent Volume Provisioning
vcluster - Persistent Volume Provisioning

Since the vcluster's syncer synchronizes pods to the underlying host cluster to schedule them, vcluster users can (by default, this can be configured in advanced options) use the storage classes of the underlying host cluster to create persistent volume claims and to mount persistent volumes.

vcluster provides helm values to adjust this behavior during vcluster installation or upgrade. Find out more below.

Sync Persistent Volumes#

By default, creating persistent volumes in the vcluster will have no effect, as vcluster runs without any cluster scoped access in the host cluster. However, if you enable persistentvolumes sync via helm values, the appropriate ClusterRole will be created in the host cluster and the syncer will be started with a flag that enables persistent volume synchronization from vcluster down to the underlying host cluster.

Create a vcluster with persistent volume sync#

To enable the synchronization of the PersistentVolume and StorageClass resources add the following to your values.yaml:

sync:
persistentvolumes:
enabled: true
# If you want to create custom storage classes
# inside the vcluster.
storageclasses:
enabled: true

then create or upgrade the vcluster with:

vcluster create my-vcluster -n my-vcluster-namespace --upgrade -f values.yaml

How does it work?#

When you enable persistent volume sync, vcluster will create persistent volumes that are created in vcluster itself in the host cluster in the form of vcluster-PERSISTENT_VOLUME_NAME-x-VCLUSTER_NAMESPACE-x-VCLUSTER_NAME to avoid any conflicts with already existing persistent volumes or other vclusters that sync persistent volumes. vcluster will then rewrite persistent volume claims with those new names so that it seems that the virtual name was bound.

This means that when you create a PVC in the form of:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: persistent-volume-claim
spec:
storageClassName: my-storage-class
volumeName: my-persistent-volume
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi

vcluster will rewrite this PVC into the following in the host cluster to prevent any conflicts with already existing storage classes or persistent volumes:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: persistent-volume-claim-x-default-x-VCLUSTER_NAME
spec:
storageClassName: vcluster-my-storage-class-VCLUSTER_NAMESPACE-x-VCLUSTER_NAME
volumeName: vcluster-my-persistent-volume-VCLUSTER_NAMESPACE-x-VCLUSTER_NAME
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi

This only happens if persistent volume sync is enabled in the vcluster. There might be cases where you want to disable this automatic rewriting of PVCs (for example if you want to mount an already existing PV of the host cluster to a PVC in the vcluster), for that case you can set the annotation called vcluster.loft.sh/skip-translate to true, which will tell vcluster to not rewrite the PVC volumeName, storageClass, selectors or dataSource.

Sync Volume Snapshots#

Kubernetes VolumeSnapshot resource represents a snapshot of a volume on a storage system. You can read more about volume snapshots on the official Kubernetes documentation page of this feature.

By default, VolumeSnapshot syncing is disabled, and creating a VolumeSnapshot custom resource in the vcluster will have no effect. Following chapters describe how to enable this feature in the vcluster.

Tech-preview feature

This vcluster feature is still in the early state of maturity, so it is highly recommended to perform testing with your specific storage configurations and use cases.

We offer this feature to the community as a tech-preview to gather feedback and information about compatibility with various CSI implementations and Kubernetes distributions. Please share your experience with us via #vcluster channel in our Slack instance or via GitHub issues.

Host prerequisites#

Vcluster relies fully on the volume snapshot capabilities of the host cluster, which has to fullfil certain criteria.

Host cluster must have all relevant snapshot CRDs installed, whitout which the vcluster will fail to start when volume snapshots sync is enabled.

Host cluster should have a common snapshot controller installed, as well as a compatible CSI driver. Without these the volume snapshots will not be created in the storage backend.

It is also recommended for the host cluster to have the volume snapshots validating webhook installed.

Create a vcluster with volume snapshots sync#

To enable synchronization of all resources relevant for the volume snapshotting, and automatically create the necessary RBAC permissions, add the following to your values.yaml:

sync:
volumesnapshots:
enabled: true

then create or upgrade the vcluster with:

vcluster create my-vcluster -n my-vcluster-namespace --upgrade -f values.yaml
info

It is recommend to install the volume snapshots validating webhook in your vcluster instance.

How does it work?#

When you enable volume snapshot sync, vcluster will start watching VolumeSnapshot, VolumeSnapshotContent, and VolumeSnapshotClass CRs and syncing them between vcluster and host cluster. These resource types are synced in the following ways:

VolumeSnapshot resources created in the vcluster will be synced to the host cluster with the name in form of vcluster-VOLUME_SNAPSHOT_NAME-x-VOLUME_SNAPSHOT_NAMESPACE-x-VCLUSTER_NAME. The status and finalizers of this resource will be synced back into vcluster. The .spec.source field of the VolumeSnapshot resource in the host cluster will be rewritten to reference the expected PersistentVolumeClaim or VolumeSnapshotContent resource.

VolumeSnapshotContent resources created in the vcluster will be synced to the host cluster with the name in form of vcluster-VOLUME_SNAPSHOT_NAME-x-VCLUSTER_NAMESPACE-x-VCLUSTER_NAME. VolumeSnapshotContent resources created in the host cluster and referencing VolumeSnapshot from the vcluster will be synced into vcluster. The status and finalizers of the resource in host cluster will be synced into its vcluster representation. The .spec.volumeSnapshotRef field of the VolumeSnapshotContent resource will be rewritten to reference the expected VolumeSnapshot resource.

VolumeSnapshotClass resources will be synced from the host cluster into vcluster only.