Skip to main content

Persisting vCluster data

vCluster uses k3s as Kubernetes distribution for the virtual control plane. The ability to run Kubernetes using a datastore other than etcd sets K3s apart from other Kubernetes distributions. This feature provides flexibility to vCluster operators. The available datastore options allow you to select a datastore that best fits your use case.

K3s supports the following datastore options:

For more information, please take a look at the k3s documentation directly.

Embedded SQLite without Persistent Volume

By default vCluster will deploy k3s to use a persistent volume claim to store the data in. You can also instead use an emptyDir to store the virtual cluster data.

In order to use an emptyDir to store the data instead of a persistent volume, please create a values.yaml with the following contents:

storage:
persistence: false

Then upgrade or recreate the vCluster with:

vcluster create my-vcluster -n my-vcluster --upgrade -f values.yaml
Potential Data Loss

This method should only be used for testing purposes, as data will be lost upon pod recreation. If you are having problems with k3s in general, please consider using another Kubernetes distribution such as k0s or vanilla k8s

Datastore Options

If you want to use an external datastore such as PostgreSQL, MySQL, or etcd you must set the K3S_DATASTORE_ENDPOINT environment variable of the vCluster container so that K3s knows how to connect to it. You may also specify environment variables to configure the authentication and encryption of the connection. The following environment variables are available:

  • K3S_DATASTORE_ENDPOINT: Specify a PostgreSQL, MySQL, or etcd connection string. This is a string used to describe the connection to the datastore. The structure of this string is specific to each backend and is detailed below.
  • K3S_DATASTORE_CAFILE: TLS Certificate Authority (CA) file used to help secure communication with the datastore. If your datastore serves requests over TLS using a certificate signed by a custom certificate authority, you can specify that CA using this parameter so that the K3s client can properly verify the certificate.
  • K3S_DATASTORE_CERTFILE: TLS certificate file used for client certificate based authentication to your datastore. To use this feature, your datastore must be configured to support client certificate based authentication. If you specify this parameter, you must also specify the K3S_DATASTORE_KEYFILE parameter.
  • K3S_DATASTORE_KEYFILE: TLS key file used for client certificate based authentication to your datastore. See the previous K3S_DATASTORE_CERTFILE parameter for more details.

Configuration & Examples

As mentioned, the format of the value passed to the K3S_DATASTORE_ENDPOINT environment variable is dependent upon the datastore backend.

In its most common form, the K3S_DATASTORE_ENDPOINT environment variable for etcd has the following format:

https://etcd-host-1:2379,https://etcd-host-2:2379,https://etcd-host-3:2379 

The above assumes a typical three node etcd cluster. The environment variable can accept one more comma separated etcd URLs.

Postgres Example

The following example could be used to launch a vCluster that connects to a PostgresSQL database named k3s. Create a values.yaml with:

vcluster:
env:
- name: K3S_DATASTORE_ENDPOINT
value: postgres://username:password@hostname:5432/k3s

Create the vCluster with:

vcluster create my-vcluster -n my-vcluster -f values.yaml

MySQL Example

The following example could be used to connect the vCluster to a MySQL database using client certificate authentication. Create a values.yaml with:

vcluster:
env:
- name: K3S_DATASTORE_ENDPOINT
value: 'mysql://username:password@tcp(hostname:3306)/k3s'
- name: K3S_DATASTORE_CERTFILE
value: '/path/to/client.crt'
- name: K3S_DATASTORE_KEYFILE
value: '/path/to/client.key'
volumeMounts:
- mountPath: /data
name: data
- mountPath: /path/to
name: datastore-tls
volumes:
- name: datastore-tls
secret:
secretName: my-datastore-secret
items:
- key: tls.key
path: client.key
- key: tls.crt
path: client.crt

Create the vCluster with:

vcluster create my-vcluster -n my-vcluster -f values.yaml