Skip to main content

Overview

warning

This is currently still an alpha feature

Plugins are a feature to extend the capabilities of vcluster. They allow you to add custom functionality, such as:

  1. Syncing specific resources from or to the virtual clusters, including cluster scoped resources like cluster roles
  2. Syncing custom resources from or to the virtual cluster
  3. Deploying resources on virtual cluster startup, such as CRDs, applications, etc.
  4. Manage resources and applications inside the host or virtual cluster
  5. Enforcing certain restrictions on synced resources or extending the existing syncers of vcluster
  6. Any other operator use case that could benefit from having access to the virtual cluster and the host cluster simultaneously.

A plugin in its purest form is a Kubernetes operator that will have access to both the virtual cluster and the host cluster simultaneously. This is the main difference between a vcluster plugin and a regular Kubernetes operator that you would just install inside the vcluster itself. Given this dual access, the plugin is able to translate resources between both clusters, which is the basic building block of how vcluster works.

Recommended Reads

In order to better understand how vcluster plugins work, it is recommended to read about Kubernetes operators as well as controllers.

Architecture#

Each plugin will run as a sidecar container inside the vcluster pod. This is done to allow easier communication between vcluster and the plugins as well as provide certain capabilities such as high-availability out of the box. The plugin itself will contact the vcluster pod during startup to obtain the access credentials to the virtual and host cluster. The plugin controllers are started with these credentials, similar to how vcluster itself starts its resource syncers.

Plugin Controllers#

Resource syncing is the heart of vcluster which enables the virtual cluster to behave like an actual Kubernetes cluster. A Kubernetes controller that is responsible for resource syncing in vcluster is called a syncer. This controller reacts on changes to objects within the virtual cluster and on changes to objects within the host cluster. The syncer tries to map each virtual object to a physical object in the host cluster and then compares those. After it discovers a change, the syncer ensures that the virtual cluster object and the physical cluster object are aligned in the desired state, and if not, the syncer changes either one of those objects to reflect the desired state.

Each plugin can define several of those resource syncers that would work exactly like the built-in syncers of vcluster. However, you'll not need to sync every Kubernetes resource to the host cluster, as some can stay purely virtual. Only resources that influence the workloads need to be synced, for example, pods, services, and endpoints, while others such as deployments, replicasets, namespaces etc. are only relevant to the Kubernetes control plane and hence are not needed in the host cluster.

Plugin SDK#

Recommended Reads

If you want to start developing your own vcluster plugins, it is recommended that you read about controller-runtime as well as kube builder that uses the controller runtime internally.

vcluster provides an SDK for writing plugin controllers that abstracts a lot of the syncer complexity away from the user, but still gives you access to the underlying structures if you need it. Internally, the vcluster SDK uses the popular controller-runtime project, that is used by vcluster itself to create the controllers. The vcluster SDK lets you write custom plugin controllers with just a few lines of code.

Since the plugin SDK interfaces are mostly compatible with the vcluster syncers, you can also take a look at how those are implemented in the vcluster itself, which work in most cases the same way as if those would be implemented in a plugin. It would be even possible to reimplement all vcluster syncers in a separate plugin.

Loading and Installing Plugins to vcluster#

Since the most common distribution method of vcluster is helm, plugins are also configured via helm values. If you develop a plugin of your own, we recommend creating a plugin.yaml (the name has no special functionality, you could also name it my-plugin.yaml or extra-values.yaml) in the following format:

# Plugin Definition below. This is essentially a valid helm values file that will be merged
# with the other vcluster values during vcluster create or helm install.
plugin:
myPlugin:
image: plugin-image
# Other optional sidecar values below
# command: ...
# env: ...
# Configure Extra RBAC Rules like this
#rbac:
# role:
# extraRules:
# - apiGroups: ["example.loft.sh"]
# ...
# clusterRole:
# extraRules:
# - apiGroups: ["apiextensions.k8s.io"]
# ...

The plugin.yaml is a valid helm values file used to define the plugin's sidecar configuration and additional RBAC rules needed to function properly. If you want to distribute that plugin for others, it's also possible to install a plugin through an URL:

# Install a plugin with a local plugin.yaml
vcluster create my-vcluster -n my-vcluster -f plugin.yaml -f other-values.yaml
# Install a plugin with a remote URL
vcluster create my-vcluster -n my-vcluster -f https://github.com/my-org/my-plugin/plugin.yaml -f other-values.yaml
# Install a plugin with helm with a remote URL
helm install my-vcluster vcluster -n my-vcluster --repo https://charts.loft.sh -f https://github.com/my-org/my-plugin/plugin.yaml -f other-values.yaml
Examples

You can take a look at the vcluster-sdk repo for some working examples.

Don't install untrusted plugins

A plugin runs with the same permissions as vcluster itself does in the Kubernetes cluster and can also define additional permissions through its plugin.yaml, so make sure you only install plugins you trust.