Set up the resource proxy
This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.
This feature requires vCluster Platform. Both the client and target tenant clusters must be managed as VirtualClusterInstance within the platform.
The resource proxy lets a client tenant clusterTenant clusterA fully isolated Kubernetes environment provisioned for a single tenant. Each tenant cluster has its own API server, controller manager, and resource namespace, backed by a virtualized control plane hosted on a control plane cluster. From the tenant's perspective it behaves exactly like a standard Kubernetes cluster. transparently forward custom resource requests to a target tenant cluster, which stores the objects and can run controllers against them. Set it up to centralize management of selected custom resources across tenant clusters, expose target-hosted custom resources across projects, or build cross-cluster automation.
This page walks through setting it up for common scenarios. For the mechanism, see How the resource proxy works. For the vcluster.yaml fields themselves, see Configure the resource proxy.
Platform RBAC requirements​
Platform checks access to the target at two points, using two different principals:
- When someone creates or updates the client VirtualClusterInstance, the user or automation submitting that request must have
useaccess to the target VirtualClusterInstance. - When the client tenant cluster registers itself, the client's Platform identity must have the same
useaccess. Its identity has the formatloft:vcluster:<client-project-namespace>:<name>.
These checks require RBAC on the platform's management cluster, in addition to the resource RBAC you configure inside the target tenant cluster.
Platform RBAC configuration​
Create a Role and RoleBinding in the target tenant cluster's project namespace on the platform's management cluster. With the default project namespace prefix, this namespace is p-<target-project-name>. Installations with a customized prefix, including installations upgraded from older versions, can use a different project namespace. The following RoleBinding grants access to the client identity used during registration:
# Platform RBAC for resource proxy
# This grants the client tenant cluster identity access during registration.
# The identity creating or updating the client also needs use access.
# Apply to the platform's management cluster in the target project namespace.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: vcluster-proxy-target-access
namespace: p-default
rules:
- apiGroups: ["management.loft.sh"]
resources: ["virtualclusterinstances"]
resourceNames: ["target"] # Target vCluster name
verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: client-vcluster-proxy-access
namespace: p-default
subjects:
- kind: User
name: "loft:vcluster:p-default:client" # Client vCluster identity
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: vcluster-proxy-target-access
apiGroup: rbac.authorization.k8s.io
Apply this configuration to the platform's management cluster (not the tenant clusters):
kubectl apply -f platform-proxy-rbac.yaml --context <platform-context>
The user or automation that creates or updates the client may already receive use access through its Platform project role. Check the submitting identity separately:
kubectl auth can-i use virtualclusterinstances.management.loft.sh/target \
-n p-default \
--context <platform-context>
If the command returns no, a Platform administrator must bind that user, group, or service account to the vcluster-proxy-target-access Role, or grant equivalent use access. This must happen before it creates or updates the client.
Examples​
Basic proxy setup​
This example demonstrates a simple two-cluster setup where a client tenant cluster proxies MyResource resources to a target tenant cluster.
Create the target tenant cluster.
Create a tenant cluster to serve as the target. The target doesn't need any proxy configuration - it just stores the resources and enforces RBAC:
Create target tenant clustervcluster create target --driver platform --project defaultInstall the CustomResourceDefinition in the target tenant cluster.
The CustomResourceDefinition must exist in the target tenant cluster:
myresource-crd.yamlapiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:name: myresources.example.comspec:group: example.comnames:kind: MyResourcelistKind: MyResourceListplural: myresourcessingular: myresourcescope: Namespacedversions:- name: v1served: truestorage: trueschema:openAPIV3Schema:type: objectproperties:spec:type: objectproperties:name:type: stringpriority:type: stringApply the CustomResourceDefinition to the target tenant cluster:
Apply CustomResourceDefinition to targetvcluster connect target --driver platform --project default -- kubectl apply -f myresource-crd.yamlConfigure RBAC in the target tenant cluster.
Create RBAC rules to allow the client tenant cluster to access resources. The client tenant cluster authenticates using its vClustervClusterAn open-source software product that creates and manages tenant clusters within Kubernetes infrastructure. vCluster provides tenant isolation capabilities while reducing infrastructure costs. Platform identity in the format
loft:vcluster:<client-project-namespace>:<name>. For example, a client namedclientwhose project namespace isp-defaultusesloft:vcluster:p-default:client:target-rbac.yamlapiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata:name: vcluster-proxy-clientrules:- apiGroups: ["example.com"]resources: ["myresources"]verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]- apiGroups: ["example.com"]resources: ["myresources/status"]verbs: ["get", "update", "patch"]- apiGroups: [""]resources: ["namespaces"]verbs: ["get", "list", "watch", "create"]---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata:name: vcluster-proxy-clientroleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: vcluster-proxy-clientsubjects:- kind: User# vCluster identity format: loft:vcluster:<client-project-namespace>:<name>name: "loft:vcluster:p-default:client"apiGroup: rbac.authorization.k8s.ioApply RBAC to the target tenant cluster:
Apply RBAC to targetvcluster connect target --driver platform --project default -- kubectl apply -f target-rbac.yamlKeep the
getverb when grantingupdate,patch, ordeleteinownedmode. The proxy performs a GET before those named mutations to verify ownership. Withoutget, it masks the forbidden ownership check as a 404.Configure platform RBAC.
Grant the client identity permission to access the target when the tenant cluster registers with vCluster Platform. Apply this to the platform's management cluster:
platform-rbac.yaml# Platform RBAC for resource proxy# This grants the client tenant cluster identity access during registration.# The identity creating or updating the client also needs use access.# Apply to the platform's management cluster in the target project namespace.apiVersion: rbac.authorization.k8s.io/v1kind: Rolemetadata:name: vcluster-proxy-target-accessnamespace: p-defaultrules:- apiGroups: ["management.loft.sh"]resources: ["virtualclusterinstances"]resourceNames: ["target"] # Target vCluster nameverbs: ["use"]---apiVersion: rbac.authorization.k8s.io/v1kind: RoleBindingmetadata:name: client-vcluster-proxy-accessnamespace: p-defaultsubjects:- kind: Username: "loft:vcluster:p-default:client" # Client vCluster identityapiGroup: rbac.authorization.k8s.ioroleRef:kind: Rolename: vcluster-proxy-target-accessapiGroup: rbac.authorization.k8s.ioApply to the platform's management cluster:
Apply platform RBACkubectl apply -f platform-rbac.yaml --context <platform-context>Before creating the client in the next step, also verify that your current Platform identity can use the target:
Check creator access to targetkubectl auth can-i use virtualclusterinstances.management.loft.sh/target \-n p-default \--context <platform-context>Create the client tenant cluster with proxy configuration.
Configure the client tenant cluster to proxy MyResource resources to the target:
client-vcluster.yamlexperimental:proxy:customResources:myresources.example.com/v1:enabled: truetargetVirtualCluster:name: "target"Deploy the client tenant cluster:
Deploy client tenant clustervcluster create client --driver platform --project default -f client-vcluster.yamlTest the proxy.
Create a MyResource in the client tenant cluster:
Create MyResource in clientvcluster connect client --driver platform --project default -- kubectl apply -f - <<EOFapiVersion: example.com/v1kind: MyResourcemetadata:name: test-resourcenamespace: defaultspec:name: "Test Resource"priority: "high"EOFVerify the resource exists in both tenant clusters:
Verify resource in both clusters# Check in client via proxyvcluster connect client --driver platform --project default -- kubectl get myresources# Check in target where resources are storedvcluster connect target --driver platform --project default -- kubectl get myresources
Multi-target proxy​
A single tenant cluster can proxy different API groups and versions to different target tenant clusters. Routing is group/version-wide, so sibling resources in the same group and version can't use different targets.
# Multi-target Resource Proxy configuration
# Proxies different API groups and versions to different target tenant clusters
experimental:
proxy:
customResources:
# Route example.com/v1 to target-a. These explicit resource names are
# advertised by synthetic fallback discovery.
myresources.example.com/v1:
enabled: true
targetVirtualCluster:
name: "target-a"
secondaryresources.example.com/v1:
enabled: true
targetVirtualCluster:
name: "target-a"
# Route test.io/v2 to target-b. These explicit resource names are
# advertised by synthetic fallback discovery.
otherresources.test.io/v2:
enabled: true
targetVirtualCluster:
name: "target-b"
additionalresources.test.io/v2:
enabled: true
targetVirtualCluster:
name: "target-b"
In this configuration:
target-aservesexample.com/v1. The explicit entries cause fallback discovery to advertise MyResource and SecondaryResource.target-bservestest.io/v2. The explicit entries cause fallback discovery to advertise OtherResource and AdditionalResource.
While either target is reachable, its live discovery can advertise other sibling resources in the same group and version, and the proxy can route requests to them. Use target RBAC to restrict which resource kinds the client can access.
Wildcard group and version proxy​
Use a wildcard key to express that an entire API group and version belongs to the same target without naming individual resources.
# Wildcard Resource Proxy configuration
# Enables group/version-wide proxying for example.com/v1 to a target tenant cluster.
# Unlike explicit entries, a wildcard can't advertise resource names in fallback discovery.
experimental:
proxy:
customResources:
"*.example.com/v1":
enabled: true
targetVirtualCluster:
name: "target"
While the target is reachable, any resource kind it advertises for example.com/v1 can be proxied. Explicit entries have the same live routing scope, but discovery degrades differently. A wildcard produces an empty fallback resource list when the target is unavailable, whereas explicit entries advertise the configured resource names. See Wildcard group and version entry for details.
Cross-project proxy​
By default, the target tenant cluster is assumed to be in the same project as the client. You can proxy to a tenant cluster in a different project by specifying the project field. This works across different control planeControl PlaneThe container orchestration layer that exposes the API and interfaces to define, deploy, and manage the lifecycle of containers. In vCluster, each tenant cluster has its own control plane components. clusters connected to the same vCluster Platform.
For a cross-project proxy, create the Platform Role and RoleBindings in the target project's namespace. Grant access both to the user or automation that creates or updates the client and to the client identity used during registration. For example, consider a client named client in project namespace p-team-a that proxies to a target in project namespace p-central. Bind loft:vcluster:p-team-a:client to the Role in p-central, and also bind the submitting principal to that Role.
# Cross-project Resource Proxy configuration
# Proxies resources to a target tenant cluster in a different project
experimental:
proxy:
customResources:
myresources.example.com/v1:
enabled: true
targetVirtualCluster:
name: "target"
project: "other-project" # Target is in a different project
This is useful for scenarios where:
- A shared resource storage cluster exists in a centralized project
- Teams in different projects need to access common resources
- Tenant clusters across different control plane clusters need to access custom resources hosted by the same target
- CI/CD environments need access to centralized resource management
Multi-client isolation​
This example demonstrates how multiple client tenant clusters can share a target while maintaining isolation. See Multi-client isolation for how the isolation mechanism works, and Access modes for the accessResources field this relies on.
Configure client tenant clusters.
Both clients proxy to the same target:
team-a-vcluster.yamlexperimental:proxy:customResources:myresources.example.com/v1:enabled: truetargetVirtualCluster:name: "orchestrator"# Uses default accessResources: ownedteam-b-vcluster.yamlexperimental:proxy:customResources:myresources.example.com/v1:enabled: truetargetVirtualCluster:name: "orchestrator"# Uses default accessResources: ownedConfigure target RBAC for multiple clients.
Configure RBAC in the target for both clients:
multi-client-target-rbac.yamlapiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata:name: vcluster-proxy-clientrules:- apiGroups: ["example.com"]resources: ["myresources"]verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]- apiGroups: ["example.com"]resources: ["myresources/status"]verbs: ["get", "update", "patch"]- apiGroups: [""]resources: ["namespaces"]verbs: ["get", "list", "watch", "create"]---# Bind for team-aapiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata:name: vcluster-proxy-team-aroleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: vcluster-proxy-clientsubjects:- kind: Username: "loft:vcluster:p-default:team-a"apiGroup: rbac.authorization.k8s.io---# Bind for team-bapiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata:name: vcluster-proxy-team-broleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: vcluster-proxy-clientsubjects:- kind: Username: "loft:vcluster:p-default:team-b"apiGroup: rbac.authorization.k8s.ioApply to the target tenant cluster:
Apply target RBACvcluster connect orchestrator --driver platform --project default -- kubectl apply -f multi-client-target-rbac.yamlConfigure platform RBAC for multiple clients.
Grant both client tenant clusters permission to access the target through vCluster Platform:
multi-client-platform-rbac.yamlapiVersion: rbac.authorization.k8s.io/v1kind: Rolemetadata:name: vcluster-proxy-target-accessnamespace: p-defaultrules:- apiGroups: ["management.loft.sh"]resources: ["virtualclusterinstances"]resourceNames: ["orchestrator"]verbs: ["use"]---apiVersion: rbac.authorization.k8s.io/v1kind: RoleBindingmetadata:name: team-a-vcluster-proxy-accessnamespace: p-defaultsubjects:- kind: Username: "loft:vcluster:p-default:team-a"apiGroup: rbac.authorization.k8s.ioroleRef:kind: Rolename: vcluster-proxy-target-accessapiGroup: rbac.authorization.k8s.io---apiVersion: rbac.authorization.k8s.io/v1kind: RoleBindingmetadata:name: team-b-vcluster-proxy-accessnamespace: p-defaultsubjects:- kind: Username: "loft:vcluster:p-default:team-b"apiGroup: rbac.authorization.k8s.ioroleRef:kind: Rolename: vcluster-proxy-target-accessapiGroup: rbac.authorization.k8s.ioApply to the platform's management cluster:
Apply platform RBACkubectl apply -f multi-client-platform-rbac.yaml --context <platform-context>Test isolation.
Test multi-client isolation# Team A creates a resourcevcluster connect team-a --driver platform --project default -- kubectl apply -f - <<EOFapiVersion: example.com/v1kind: MyResourcemetadata:name: team-a-resourcespec:name: "Team A Data Resource"EOF# Team B creates a resourcevcluster connect team-b --driver platform --project default -- kubectl apply -f - <<EOFapiVersion: example.com/v1kind: MyResourcemetadata:name: team-b-resourcespec:name: "Team B ML Resource"EOF# Team A only sees their resourcevcluster connect team-a --driver platform --project default -- kubectl get myresources# NAME AGE# team-a-resource 1m# Team B only sees their resourcevcluster connect team-b --driver platform --project default -- kubectl get myresources# NAME AGE# team-b-resource 1m# Target orchestrator sees bothvcluster connect orchestrator --driver platform --project default -- kubectl get myresources# NAME AGE# team-a-resource 2m# team-b-resource 1m
Next steps​
Once the proxy is running:
- Keep proxy traffic off the DERP relay and set up monitoring. See Operate the resource proxy.
- If something isn't working, start with the quick triage. See Troubleshoot the resource proxy.