AI platforms as a service
Offer your tenants a working AI platform instead of a bare cluster. A tenant requests a cluster, and vCluster Platform installs NVIDIA Run:ai or NVIDIA Dynamo into it from a tested Stack. You can also run the platform once on the control plane cluster and serve it to tenants yourself.
This page shows how the pieces fit together and the order to build them in. Each step links to the guide that owns the procedure.
What the service provides
- A tested AI platform. vCluster Platform bundles certified Stacks for NVIDIA Run:ai and NVIDIA Dynamo. Each one installs every component in dependency order and reports health per component.
- Self-service delivery. Tenants get the platform by creating a cluster from a template that includes it, or, with the right permissions, by installing it into a cluster they already have.
- A choice of where it runs. Install the platform inside each tenant's cluster, or once on the control plane cluster for many tenants.
- Your own platforms too. The same Stack API that delivers the certified platforms delivers any custom Stack you build, such as a model serving stack.
How it works
A Stack deploys several applications as one dependency-aware unit. The certified AI platforms are Stacks that ship with vCluster Platform, so you install them the same way you install your own.
| Layer | What it does | Where it's documented |
|---|---|---|
| StackTemplate | A reusable, parameterized graph of tasks. Each task installs one App or Argo CD Application, and later tasks can wait on earlier ones and read their outputs. | What are Stacks |
| Certified Stack | A StackTemplate that vCluster Platform bundles and tests. The UI marks it as certified and read-only, and each Platform release pins its own copy. | Certified Stacks |
| StackInstance | One installation of a StackTemplate into a destination, either a cluster or a control plane cluster. | Use a Stack |
| Cluster template | A cluster template can declare Stacks, so every cluster created from it gets the platform automatically. | Use a Stack |
| Access | A project's allowed templates decide which StackTemplates and cluster templates its members can use. Kubernetes role-based access control (RBAC) decides who can install and manage Stacks. | Stack permissions, Allowed templates |
vCluster Platform bundles these certified AI platforms.
| Platform | Where it runs | What it gives tenants |
|---|---|---|
| NVIDIA Run:ai | A dedicated control plane inside each cluster, or one central control plane on the control plane cluster that each cluster registers with. | GPU scheduling and workload management for AI and machine learning teams. |
| NVIDIA Dynamo | One runtime inside a cluster, or one runtime on the control plane cluster. | A runtime that serves AI models across GPU nodes. |
Build the service
- Install vCluster Platform on the cluster that will run the management plane. The certified Stacks use App tasks, and Apps are included on every plan. See Install.
- Decide how tenants run workloads. If tenants get Kubernetes access to deploy their own models or jobs, the worker node model decides how isolated they are. See Decide whether tenants share hardware, and read where the service ends.
- Prepare GPU capacity. Both platforms need GPU nodes. The NVIDIA Run:ai Stacks install the NVIDIA GPU Operator unless your provider already manages it. NVIDIA Dynamo expects the NVIDIA device plugin to be in place already. Decide which component owns the NVIDIA drivers before you install anything. The certified Stack prerequisites cover ingress, storage, and registry credentials.
- Choose a platform and a model. For NVIDIA Run:ai, choose between a dedicated and a central control plane, as described in NVIDIA Run:ai. For NVIDIA Dynamo, choose whether each cluster runs its own runtime or you run one on the control plane cluster. See the Certified Stacks overview.
- Install the platform. For the NVIDIA Run:ai central model, install the shared host foundation once on the control plane cluster. The dedicated model needs nothing up front, because its bundled cluster template installs everything when a cluster is created. See Manage Certified Stacks. For NVIDIA Dynamo, install the
nvidia-dynamotemplate on the control plane cluster, or add it to a cluster template so each cluster gets its own runtime. See Use a Stack. - Give tenants access. Allow each project the StackTemplates it may install and the cluster templates that include them. Projects created in the UI allow every StackTemplate by default, so narrow the list where a tenant shouldn't get a platform. The built-in Project Admin role can install Stacks, and other members need a custom project role. See Allowed templates and Stack permissions.
- Test the service. Create one cluster from a tenant project, and wait for every task in its StackInstance to report
Healthy. Then run a workload that uses a GPU. Each certified Stack page lists the checks that prove the platform is ready, beyond a healthy Helm release.
Build your own platform
The certified Stacks cover two platforms. To offer something else, such as a model serving stack with vLLM or KServe, build your own StackTemplate.
- Start from a certified Stack. Copy its YAML, give it a new name, and remove the
vcluster.com/certifiedannotation. Your copy doesn't receive later bundle updates. See Customize a certified stack. - Start from scratch. Define tasks, dependencies, parameters, and outputs, beginning with the starter stack. See Create a Stack template.
- Start from a community Stack. The vcluster-stacks repository has community Stacks from customers, partners, community members, and vCluster Labs. vCluster Platform doesn't bundle or certify them.
For a model serving stack's components and the order to install them in, see Inference serving stacks.
Operate the service
- Watch each installation. A StackInstance reports an aggregate phase and the status of every task. See Monitor a stack.
- Upgrade deliberately. Each Platform release pins its certified bundle, and the bundled Apps pin their chart versions. Changing a version parameter on NVIDIA Run:ai doesn't upgrade its charts. NVIDIA Dynamo upgrades through its
versionparameter. - Remove in order. Both platforms need resources removed in a set order, or tenant registrations and model deployments can get stuck. Follow the removal steps on each certified Stack page.
- Troubleshoot installations. See Troubleshoot Stacks.
Know where the service ends
vCluster Platform owns the Stack lifecycle. That covers installing each component in order, reporting its health, and removing it. The AI platform owns everything it manages after that, including models, job scheduling, and GPU allocation. Your GPU nodes and their NVIDIA drivers sit underneath both.
Where the platform runs decides who you're trusting. Both bundled NVIDIA Run:ai cluster templates use shared control plane cluster nodes by default. The central model separates tenants' GPU nodes with node labels, which are scheduling policy, not a security boundary. When tenants deploy their own models into an NVIDIA Dynamo runtime in their cluster, they have Kubernetes access and run their own workloads.
Shared nodes are a supported, common model for trusted tenants. They give each tenant control-plane, API, and namespace isolation, but tenant workloads share the same kernel and physical nodes. They aren't a security boundary for untrusted tenants with Kubernetes access or arbitrary workload execution. Good fits are development, testing, CI/CD, and internal engineering teams.
Use private nodes for external, resale, regulated, or otherwise untrusted tenant offerings, optionally with vNode for runtime isolation.
NetworkPolicy is an added isolation layer worth enabling even for trusted tenants. vCluster can create the policies for you through policies.networkPolicy, and your control plane cluster's CNI enforces them. Confirm your CNI supports enforcement, since some accept NetworkPolicy resources without acting on them. See the security baseline.
If you accept shared nodes for a trusted-tenant use case, follow the shared-node hardening guide as the required next step.
If you run NVIDIA Dynamo on the control plane cluster and serve your own models through an API, tenants never get Kubernetes access. That's an application-level offering, and it can serve external customers.
Related
- Inference provider shows how model serving combines with clusters and GPU capacity in a complete platform.
- AI cloud provider and Enterprise AI factory show where AI platforms fit alongside the other services.
- Kubernetes as a service covers the clusters these platforms install into.