GPU as a Service on Kubernetes Without a Cloud Provider (The Bare Metal Architecture)

Summary

• Running large-scale GPU workloads on managed Kubernetes often leads to high costs and slow performance, with pod startups taking over 6 minutes. Building a GPU as a Service (GaaS) platform on bare metal provides superior economics and control by eliminating the hypervisor tax.
• A successful GaaS architecture is built in three phases: zero-touch provisioning to automatically prepare servers, orchestration for tenant isolation to provide isolated Kubernetes clusters without VM overhead, and cloud-like operationalization for self-service and autoscaling.
• This integrated approach can significantly accelerate time-to-market, with GPU cloud providers launching production platforms in as little as 90 days. The vCluster Platform combined with vMetal provides a proven, integrated path to build your own bare metal AI cloud.

You've been down this road before. You spun up a managed GPU Kubernetes cluster on AKS, EKS, or GKE. The scheduler works. The abstractions feel familiar. Then the bill arrives — and so does the latency, the cold-start penalty, and the quiet realization that you're building your AI infrastructure on top of someone else's control plane.

For teams running a handful of GPU workloads experimentally, managed cloud Kubernetes is a reasonable trade. For neoclouds, inference providers, and enterprises running GPU at any serious scale, those trade-offs compound fast. Virtualization can complicate deployment and increase operational overhead. Pod startup on new GPU nodes routinely takes 6–7 minutes or more, and the cost of scaling with multiple GPU pods is often prohibitively high. NUMA misalignment quietly degrades throughput. The preference in the field is clear: avoid unnecessary virtualization layers wherever possible.

That's exactly why the most performance-sensitive organizations are buying their own racks and building a GPU as a Service (GaaS) layer on bare metal Kubernetes. They get back hardware control, eliminate the hypervisor tax, maintain NUMA alignment, and — critically — own the control plane.

This article walks through the architecture for doing that. It's structured around three phases: provisioning (getting bare servers to GPU-ready nodes automatically), orchestration (running a Kubernetes fabric with tenant isolation directly on metal), and operationalization (running the platform like a real cloud provider). We'll look at how vMetal and vCluster Platform function as an integrated path through all three.

Phase 1: Provisioning — Zero-Touch, From Rack to Ready

The first problem in building a bare metal GPU cloud isn't Kubernetes. It's the physical servers sitting in a rack, waiting to become nodes.

In most environments, this process involves manual coordination: engineers rack the server, configure BIOS, install an OS via USB or IPMI console, configure networking, install the GPU driver stack, and finally register the machine into a cluster. That pipeline is fine for a dozen servers. At hundreds of GPU nodes, it's a full-time engineering burden — and a source of configuration drift, inconsistency, and human error.

The solution is zero-touch provisioning built around PXE (Preboot Execution Environment) boot. When a new GPU server powers on and is connected to the provisioning network, it broadcasts a DHCP request. A PXE server responds with a boot image, which loads a lightweight agent that executes a fully automated provisioning sequence: OS installation, driver configuration, network policy application, and Kubernetes node registration — all without a human in the loop.

vMetal is built exactly for this. It handles the entire bare metal lifecycle: PXE boot, OS installation, machine registration, VLAN/VXLAN network automation (via Netris integration), and cluster assignment. When a new GPU server is racked and powered on, vMetal discovers it, provisions it, and registers it as a GPU node in the target cluster — all automatically.

The architectural picture looks like this: a provisioning plane (vMetal) sits alongside your physical network, watching for new machines on the provisioning VLAN. When a machine appears, vMetal boots it via PXE, runs the OS installer, applies network configuration, installs the GPU driver stack, and registers the node. From that point, the machine is live Kubernetes capacity — typically within minutes of being powered on.

This zero-touch model means your infrastructure team can focus on racking hardware, not operating it. New GPU capacity becomes available at machine-boot speed, not human-workflow speed.

Phase 2: Orchestration — Multi-Tenant Kubernetes Without VM Overhead

With a pool of provisioned GPU nodes available, the next challenge is building the Kubernetes layer itself — specifically, a fabric that provides tenant isolation without the performance penalties that make engineers want to avoid virtualization entirely.

Running Kubernetes Directly on Metal

The conventional path to bare metal Kubernetes involves an intermediate bootstrapping tool: kubeadm, k3s, or k0s. These work, but they add another layer of tooling, dependency management, and potential failure modes between your hardware and your cluster.

vMetal ships with vCluster Standalone — a certified Kubernetes distribution that runs as a lightweight binary directly on Linux, with no external Kubernetes dependency required as a base layer. You get a CNCF-certified cluster directly on the bare metal host, without the intermediate scaffolding. That simplicity matters at scale: fewer moving parts mean fewer failure modes, and a reduced attack surface.

Isolated Tenant Clusters in Seconds

With a host cluster running directly on your GPU nodes, the next problem is tenant isolation. You have many tenants — internal teams, customers, research groups — and they all need their own isolated Kubernetes environments. How do you give them genuine isolation without provisioning a separate physical cluster per tenant (expensive, slow) or collapsing everything into namespace-level separation (weak blast radius, shared RBAC)?

This is where vCluster Platform provides the critical architectural layer. Rather than either of those extremes, vCluster virtualizes the Kubernetes control plane itself. Each tenant gets a fully functional, CNCF-certified Kubernetes cluster — complete with its own API server, etcd, controller manager, and RBAC — running as a lightweight pod inside the host cluster.

The architectural model is layered: the host cluster runs on bare metal, managed by vMetal. Inside the host cluster, vCluster Platform orchestrates dozens or hundreds of tenant clusters. Each tenant cluster is a pod containing a full K8s control plane. Those tenant clusters schedule their actual workloads onto the underlying GPU worker nodes, which remain shared at the hardware level but logically partitioned at the control plane level.

The result is a system where tenant clusters spin up in seconds rather than hours, each tenant has cluster-admin level access to their own environment (including custom CRDs and RBAC policies), and the host cluster's GPU nodes are efficiently shared without tenants interfering with each other's control planes.

Workload Isolation Without the Hypervisor Tax

This architecture deliberately avoids traditional VM-based isolation precisely because of the performance problems the community has documented: slow Boot times due to GPU BARs, PCIe bottlenecks from data transfer overhead, and NUMA misalignment when GPU and vCPU resources land in different memory domains.

For teams that require stronger workload isolation — particularly GPU cloud operators running untrusted workloads in isolated tenant environments — vNode (currently in private beta) provides kernel-native isolation using seccomp, cgroups, Linux namespaces, and AppArmor, without any hypervisor. Container breakout protection is enforced at the kernel level without the VM overhead that kills GPU throughput. Bare metal GPU performance is preserved.

Phase 3: Operationalization — Running GaaS Like a Cloud Provider

Infrastructure that provisions and orchestrates correctly is necessary but not sufficient. The final phase is about operating the platform at scale — with dynamic capacity, self-service access, governance controls, and production-grade AI environments — so that it behaves like a cloud, not a collection of servers.

Dynamic Capacity: Bare Metal Karpenter

One of the deepest cost complaints in the community is the tension between GPU utilization and scaling responsiveness. Pre-provisioning GPU nodes wastes expensive hardware during idle periods. Waiting for nodes to provision on-demand creates unacceptable cold-start delays, especially when pod startup on a new node already takes 6+ minutes.

vMetal addresses this directly with Auto Nodes, a bare metal implementation of Karpenter-style autoscaling. When a tenant cluster has unschedulable pods — workloads waiting for GPU capacity that doesn't exist yet — Auto Nodes automatically provisions new bare metal GPU servers via Terraform, boots them through the same PXE pipeline from Phase 1, and registers them as live nodes. When capacity is no longer needed, nodes can be decommissioned and returned to the available pool.

This just-in-time provisioning eliminates the binary choice between "overpay for idle GPU capacity" and "wait 10 minutes during traffic spikes." New hardware joins the cluster at machine-boot speed, without manual intervention.

Self-Service: Giving Tenants the Cloud Experience

Reducing operator burden requires that tenants can provision and manage their own environments without filing tickets. vCluster Platform ships with a fleet management UI, CLI, and API, along with a self-service tenant portal that gives end-users an EKS/GKE-like experience — they can spin up their own isolated Kubernetes cluster, configure it, and manage it without involving the platform team.

This dramatically changes the operational model. Instead of every tenant request flowing through a central operations queue, tenants operate autonomously within the boundaries the platform operator sets. The platform team's job shifts from manual provisioning to policy definition.

Governance: Quotas and Cost Attribution

Self-service without guardrails becomes a cost and compliance problem quickly. vCluster Platform supports per-tenant resource quotas that are enforced at the control plane level, preventing any single tenant from consuming disproportionate GPU capacity. Quotas can be templated and applied consistently across new tenant clusters at creation time.

Cost attribution — the ability to report which tenant consumed which resources — is equally important for chargeback in enterprise environments and margin management in neocloud settings. The platform's tenant isolation model creates clean per-tenant resource boundaries that integrate with standard cost management tooling.

Certified Stacks: From Kubernetes to Production AI in Minutes

Bare Kubernetes is not an AI platform. Production AI workloads require integrated schedulers, distributed training frameworks, notebook environments, and often Slurm compatibility for teams migrating from HPC environments.

Certified Stacks are pre-validated AI environment configurations that deploy directly into tenant clusters. Integrations include Run:AI for GPU scheduling, Ray for distributed training and inference, Jupyter for interactive workloads, and Slurm (via the Slinky integration) for teams running hybrid Slurm/Kubernetes environments. These stacks go through validation against vCluster's tenant isolation model, so you're not debugging compatibility issues between your AI platform and your tenant isolation layer.

The practical outcome: a tenant asking for "a GPU cluster with Ray and Jupyter" can have a fully configured, isolated environment in minutes, not days.

From Rack to Revenue: The 90-Day Benchmark

The integrated workflow across these three phases looks like this:

vMetal handles Phase 1 — new hardware is racked, powered on, and automatically provisioned into GPU-ready Kubernetes nodes without human intervention. vCluster Standalone (embedded in vMetal) handles Phase 1.5 — a CNCF-certified host cluster runs directly on those nodes without bootstrapping dependencies. vCluster Platform handles Phase 2 — isolated tenant clusters spin up in seconds inside that host cluster, each with a dedicated control plane, RBAC, and CRDs. Auto Nodes, the self-service portal, per-tenant quotas, and Certified Stacks handle Phase 3 — the platform operates like a cloud.

The question is how long this actually takes to deploy in production. Lintasarta, Indonesia's leading GPU cloud provider, launched their full platform in 90 days, running 170+ tenant clusters on this architecture. That's a GPU cloud — with tenant isolation, self-service, and production AI workloads — built from scratch in three months.

vCluster Labs is also named in the NVIDIA DGX SuperPOD reference architecture.

Owning Your AI Infrastructure

The case for building GPU as a Service on bare metal Kubernetes isn't ideological — it's economic and technical. Cloud-managed Kubernetes solves convenience problems but creates cost problems, latency problems, and control problems that compound as scale increases. The engineers who actually run GPU workloads want to avoid virtualization overhead, they need NUMA alignment, they can't absorb 7-minute pod startup times, and they can't scale when GPU pods cost what they cost on public cloud.

The three-phase architecture described here — zero-touch provisioning, virtualized control plane tenant isolation, and cloud-like operationalization — is a concrete and proven path forward. The vMetal + vCluster Platform stack provides the integrated tooling to de-risk that path, with production benchmarks and reference architectures that demonstrate it works at scale.

If you're building an AI cloud, an internal GPU factory, or an inference platform on bare metal, this is the architecture worth understanding — and the 90-day deployment timeline is worth taking seriously. To see how vCluster Platform and vMetal can accelerate your AI cloud deployment, request a demo.

Frequently Asked Questions

What is a bare metal GPU as a Service (GaaS) platform?

A bare metal GPU as a Service (GaaS) platform is an infrastructure layer that provides on-demand access to GPU resources running directly on physical hardware, without a hypervisor. It combines automated server provisioning, Kubernetes orchestration for tenant isolation, and cloud-like operational tools to give users a self-service experience for running AI/ML workloads at scale.

Why build a GaaS platform on bare metal instead of using a managed cloud service?

Building on bare metal gives you direct control over the hardware, eliminating the performance penalties and high costs associated with virtualization in managed cloud services. This approach avoids the "hypervisor tax," reduces pod startup latency (often 6+ minutes in the cloud), ensures NUMA alignment for better throughput, and provides a more cost-effective model for scaling GPU infrastructure.

How does this architecture achieve tenant isolation without using traditional VMs?

This architecture achieves tenant isolation by virtualizing the Kubernetes control plane itself using vCluster Platform, rather than virtualizing the hardware with VMs. Each tenant receives their own isolated Kubernetes cluster (API server, etcd, etc.) running as a lightweight pod on the shared bare metal host cluster, providing strong control plane isolation without the performance overhead of a hypervisor.

What is the "hypervisor tax" for GPU workloads?

The "hypervisor tax" refers to the performance degradation and increased operational overhead caused by running GPU workloads on virtual machines. This includes slower boot times, PCIe data transfer bottlenecks between the GPU and the VM, and potential NUMA misalignment, all of which can quietly degrade throughput and increase latency.

How does bare metal Kubernetes handle autoscaling for GPU nodes?

This architecture uses a Karpenter-style autoscaler called Auto Nodes, designed specifically for bare metal. When a workload requires more GPU capacity than is available, Auto Nodes automatically provisions new physical servers, installs the OS and drivers via a PXE boot process, and joins them to the Kubernetes cluster, making new capacity available in minutes.

What is the difference between vMetal and vCluster Platform?

vMetal and vCluster Platform work together to manage the full infrastructure lifecycle. vMetal is the provisioning layer (Phase 1) that automates the process of turning racked servers into GPU-ready Kubernetes nodes. vCluster Platform is the orchestration and operationalization layer (Phases 2 & 3) that runs on those nodes to create and manage isolated tenant clusters.

Can I run different AI frameworks like Ray or Slurm on this platform?

Yes, the platform supports pre-validated configurations called Certified Stacks for deploying production AI environments into tenant clusters. These stacks include integrations for popular frameworks like Run:AI for scheduling, Ray for distributed training, Jupyter for interactive notebooks, and even Slurm for teams migrating from traditional HPC environments.

How fast can a production-ready GaaS platform be deployed?

A production-ready GaaS platform using this architecture can be deployed in approximately 90 days. This timeline is based on real-world deployments, such as the Indonesian GPU cloud provider Lintasarta, which launched its full platform with over 170 tenant clusters in three months.