DIY GPU Sharing in Kubernetes

Summary

• Kubernetes' default scheduling underutilizes expensive GPU hardware—e.g., an 8xH100 server costs over $250,000—by assigning one pod per GPU.
• Time-slicing increases workload density but lacks the memory, fault, and security isolation required for production multi-tenant environments.
• NVIDIA MIG offers true hardware-enforced isolation and predictable performance but is less flexible and supports a maximum of seven instances per GPU.
• A complete solution for tenant isolation requires both hardware partitioning and control plane isolation, which platforms like vCluster provide by giving each tenant a dedicated tenant cluster.

An 8xH100 server costs upwards of $250,000. For GPU cloud operators running platforms with tenant isolation, leaving that hardware underutilized isn't just inefficient — it's financially untenable. Yet that's exactly what happens when you rely on Kubernetes' default GPU scheduling behavior.

Out of the box, Kubernetes treats GPUs like indivisible units. A pod requests a GPU, it gets the whole thing — and no other pod touches that device until the first one is done. For many AI/ML workloads that only need a fraction of a modern GPU's compute, this means paying for an H100 while using 10% of it.

GPU sharing is the answer. But as practitioners quickly discover, the phrase covers a spectrum of approaches — from simple time-slicing configurations to hardware-level partitioning with NVIDIA MIG — each with its own trade-offs around isolation, performance, and operational complexity. And even once you've nailed the hardware layer, there's a second, harder problem: the Kubernetes control plane layer above it.

This article walks through the two primary DIY approaches to GPU sharing in Kubernetes — time-slicing and MIG — how to combine them for tiered service offerings, and why hardware-level sharing alone isn't sufficient for true tenant isolation. We'll also cover what fills that remaining gap.

The Kubernetes GPU-Sharing Problem

Kubernetes schedules GPUs through device plugins. The NVIDIA device plugin exposes GPUs as a schedulable resource (nvidia.com/gpu), and pods claim them via resource limits:

apiVersion: v1
kind: Pod
metadata:
 name: cuda-vector-add
spec:
 restartPolicy: OnFailure
 containers:
   - name: cuda-vector-add
     image: "nvidia/cuda:11.0-base"
     resources:
       limits:
         nvidia.com/gpu: 1 # requesting 1 full GPU


This model has a fundamental limitation: only whole GPU instances can be assigned. A pod requesting nvidia.com/gpu: 1 monopolizes an entire physical device regardless of its actual utilization. For a single-tenant cluster running large training jobs, this is fine. For a GPU cloud operator serving dozens of customers with varied workloads — inference services, fine-tuning runs, Jupyter notebooks, managed Kubernetes environments — it's a recipe for expensive idle capacity.

The two primary solutions are time-slicing and MIG. Each attacks the problem differently.

Approach 1: Time-Slicing with the NVIDIA GPU Operator

Time-slicing is the simpler of the two approaches. The NVIDIA GPU Operator enables a single GPU to be logically divided into multiple schedulable replicas. The GPU driver context-switches between workloads — similar to how a CPU handles multiple threads — giving each pod a slice of execution time.

Setting Up Time-Slicing

First, ensure the NVIDIA GPU Operator is installed. Once installed, you enable time-slicing via a ConfigMap in the gpu-operator namespace:

apiVersion: v1
kind: ConfigMap
metadata:
 name: time-slicing-config
 namespace: gpu-operator
data:
 config.yaml: |
   version: v1
   sharing:
     timeSlicing:
       resources:
       - name: nvidia.com/gpu
         replicas: 4


This configuration tells the operator to advertise 4 schedulable GPU resources for every 1 physical GPU. To apply this config to specific nodes, label the target nodes:

kubectl label node <node-name> nvidia.com/time-slicing.config=time-slicing-config


After applying, a node with 2 physical GPUs will expose 8 schedulable nvidia.com/gpu resources. Pods requesting nvidia.com/gpu: 1 can now be co-scheduled on the same physical device.

Pros

• Simple to configure — a single ConfigMap and a node label
• High density — high replica counts let you pack many small workloads onto a single GPU
• Flexible — different configurations can be applied per-node or per-node-pool

Cons

The Kubernetes community is increasingly skeptical of time-slicing for anything beyond development workloads, and for good reason.

No memory isolation. Time-slicing does not provide private GPU memory. All pods sharing the device see the same memory space. This means one misbehaving workload can cause OOM (Out of Memory) errors that crash every other workload on that GPU, a common pain point for operators.

No fault isolation. A GPU-level fault in one pod propagates to all pods sharing that device. There's no blast radius containment.

Security exposure. In environments with tenant isolation, the lack of memory isolation creates potential data exposure between tenants. The security risks are a documented concern, not an edge case.

Performance unpredictability. Context-switching overhead introduces latency. Workloads with strict latency requirements — production inference endpoints, for example — will see degraded and variable performance.

For these reasons, many consider time-slicing unsuitable for production use cases. While there are valid uses for managed Kubernetes environments and batch jobs with loose SLAs, the limitations for production GPU infrastructure with tenant isolation are significant and can catch operators off guard.

Approach 2: NVIDIA MIG (Multi-Instance GPU)

MIG — Multi-Instance GPU — is a fundamentally different approach. Instead of software-level time-slicing, MIG physically partitions a single GPU at the hardware level into up to seven fully independent GPU Instances (GIs).

Each MIG instance gets its own:

• Dedicated high-bandwidth memory (HBM)
• L2 cache
• Streaming multiprocessors (SMs)
• Memory controllers

This is hardware-enforced isolation, not a scheduling abstraction. One tenant's MIG instance cannot affect another's — not through memory pressure, not through a GPU fault, not through any side channel. MIG is available on NVIDIA's A100, H100, and newer data center GPUs.

Setting Up MIG

MIG is managed through the NVIDIA GPU Operator. You configure the migStrategy in the operator settings — either single (all instances on a node use the same MIG profile) or mixed (different profiles on the same node). You can refer to NVIDIA's MIG documentation for the full catalog of available profiles and their compute/memory allocations.

Once configured, the device plugin exposes MIG instances as distinct resources. For example, a 3g.40gb MIG instance on an A100-80GB appears as nvidia.com/mig-3g.40gb and can be requested in pod specs:

resources:
 limits:
   nvidia.com/mig-3g.40gb: 1


A single A100-80GB can be partitioned into seven 1g.10gb instances, three 2g.20gb instances, one 3g.40gb and one 4g.40gb, or other combinations — depending on your workload requirements.

Pros

• Hardware-enforced isolation. The "noisy neighbor" problem is eliminated. A runaway workload on one MIG instance does not affect adjacent instances. This is the critical difference from time-slicing.
• Predictable, consistent performance. Each instance has dedicated resources, so performance characteristics are stable and reproducible — essential for production inference SLAs.
• First-class observability. Each MIG instance is monitored as a separate device via DCGM-Exporter, giving you per-tenant GPU metrics without instrumentation overhead.

Cons

• Hardware dependency. MIG requires newer, higher-cost NVIDIA data center GPUs. It's not available on consumer-grade or older enterprise hardware.
• Static partition profiles. MIG configurations are not hot-swappable. Changing profile assignments requires draining workloads from the affected node — a disruptive operation. The static nature of MIG configurations can be limiting for platforms that need to rapidly adjust capacity allocation.
• Limited instance count. A maximum of seven instances per physical GPU caps your density ceiling.

Approach 3: Combining MIG and Time-Slicing

MIG and time-slicing aren't mutually exclusive. For GPU cloud operators building tiered service offerings, combining both enables a more flexible infrastructure model.

The pattern: use MIG to carve hardware-isolated partitions for different service tiers, then apply time-slicing within specific MIG instances for maximum density on lower-priority workloads.

Example tiered architecture on a single H100:

• 2x 3g.40gb MIG instances — assigned to premium customers requiring consistent, isolated GPU access for production inference
• 1x 1g.10gb MIG instance — time-sliced into 4 replicas, assigned to development workloads with loose performance requirements

This gives your infrastructure team both the strong isolation guarantees of MIG for high-value tenants and the density benefits of time-slicing for exploratory workloads — from a single physical GPU.

NVIDIA provides detailed documentation comparing time-slicing and MIG for those who need to break down when each mode applies.

How vCluster Completes the GPU Isolation Stack

GPU sharing at the hardware level — whether through MIG partitions or time-slicing — solves resource allocation. But it doesn't solve the harder problem: giving each tenant, team, or customer a completely isolated Kubernetes environment to schedule, manage, and observe their GPU workloads without interference from others.

That's the gap vCluster fills.

vCluster creates isolated tenant clusters — each with its own API server, etcd, RBAC, and CRDs — running inside your shared GPU host cluster. Every tenant gets the Kubernetes experience of a dedicated cluster: cluster-admin access, their own namespace hierarchy, their own custom resource definitions. The host cluster remains invisible to them.

This matters for GPU infrastructure operators specifically because:

• GPU operators and custom CRDs don't conflict across tenants. The NVIDIA GPU Operator and other GPU-adjacent tools deploy CRDs that would clash in a shared cluster. In isolated tenant clusters, each tenant's operator runs independently.
• Resource quotas and GPU allocation policies are enforced per tenant cluster, not just per namespace — eliminating the namespace-escape risk that resource quotas alone can't prevent.
• Flexible isolation tiers match your hardware investment. Shared nodes for lighter workloads, private nodes for high-value tenants, dedicated VMs with vNode kernel-native isolation for the strictest requirements — all managed from a single control plane.

vCluster works alongside MIG and time-slicing, not instead of them. MIG gives you hardware-level GPU partitioning. Time-slicing extends reach to more workloads. vCluster isolates the Kubernetes control plane so each tenant operates as if they own the cluster — without the cost of provisioning one physical cluster per customer.

GPU cloud providers like CoreWeave and Nscale run this stack in production at 100K+ GPU nodes. Explore the architecture for AI clouds or request a demo.

Alternative Work-Arounds and Custom Solutions

Before reaching for GPU Operator configurations, some teams attempt cruder workarounds. It's worth understanding their limitations.

Node Labeling and Taints

You can dedicate specific nodes to specific teams using Kubernetes taints and node selectors:

# Taint a node for exclusive use
kubectl taint nodes gpu-node-01 team=ml-team:NoSchedule

# Pod tolerates the taint
spec:
 tolerations:
 - key: "team"
   value: "ml-team"
   effect: "NoSchedule"
 nodeSelector:
   team: ml-team


This provides node-level isolation — one team's workloads won't land on another team's nodes. But it does nothing to enable GPU sharing. Each team still gets one pod per GPU, and it doesn't address the control plane isolation problem at all.

Namespace-Level Resource Quotas

Kubernetes ResourceQuota objects can cap GPU consumption per namespace:

apiVersion: v1
kind: ResourceQuota
metadata:
 name: gpu-quota
 namespace: team-a
spec:
 hard:
   requests.nvidia.com/gpu: "4"
   limits.nvidia.com/gpu: "4"


This limits how many GPUs Team A can claim. But it doesn't prevent a well-permissioned user from escaping namespace boundaries, and it provides zero control plane isolation. Teams still share the same API server, the same etcd, and the same CRD namespace — meaning conflicting operator deployments remain a real problem.

CUDA Multi-Process Service (MPS)

NVIDIA's CUDA MPS enables multiple CUDA processes to share a single GPU context, reducing context-switch overhead compared to time-slicing. It's useful for tightly-coupled workloads from trusted users but is explicitly not designed for tenant isolation — memory is still shared between processes, and a crash in one process can affect others.

All three of these approaches are partial solutions. They address specific symptoms of the GPU sharing problem without providing the complete isolation stack that GPU cloud operators actually need.

Conclusion

GPU sharing doesn't end at the hardware layer. MIG and time-slicing handle the physical GPU — but you still need to isolate the Kubernetes layer above it. Without isolated tenant clusters, different teams' GPU operators, quotas, and workloads collide in ways that MIG alone can't prevent.

By combining MIG, time-slicing, and vCluster's tenant cluster isolation, GPU cloud operators can run hundreds of fully isolated customer environments on shared GPU infrastructure — each with dedicated GPU access, independent control planes, and no blast-radius between tenants.

If you're building GPU infrastructure for multiple tenants or customers, our ebook "GPU-Enabled Platforms on Kubernetes" covers the full architecture — from how GPUs meet Kubernetes, to orchestrating GPU sharing, hardware isolation, and platform design for tenant isolation. Download the free ebook.

Frequently Asked Questions

What is GPU sharing in Kubernetes?

GPU sharing in Kubernetes refers to techniques that allow multiple containers or pods to run concurrently on a single physical GPU. This overcomes Kubernetes' default behavior of assigning an entire GPU to a single pod, regardless of its actual usage. By partitioning a GPU's resources, either through software-level context switching (time-slicing) or hardware-level partitioning (NVIDIA MIG), operators can significantly increase hardware utilization and run workloads more cost-effectively.

What is the difference between NVIDIA time-slicing and MIG?

The primary difference is the level of isolation: time-slicing is a software-based scheduling feature with no memory or fault isolation, while MIG (Multi-Instance GPU) is a hardware-based partitioning feature that provides fully isolated GPU instances. Time-slicing divides a GPU's execution time among multiple workloads, but they all share the same memory and fault domain. MIG physically partitions a GPU's streaming multiprocessors, memory, and cache, creating smaller, independent GPUs that are fully firewalled from each other.

When should I use time-slicing over MIG?

Use time-slicing for development, batch processing, or low-priority workloads where performance predictability and strict isolation are not critical. It is ideal for maximizing density when running many small, non-production tasks. Because time-slicing lacks memory and fault isolation, it is generally not recommended for production inference or isolated tenant environments with strict SLAs.

Is time-slicing secure for isolated tenant environments?

No, time-slicing is not considered secure for isolated tenant environments where tenants are untrusted. The lack of memory isolation creates a potential security risk for data exposure between different tenants' workloads sharing the same GPU. All pods sharing a time-sliced GPU operate in the same memory space, which could allow a malicious actor in one pod to potentially access or interfere with the data of another pod.

How does MIG provide better isolation than time-slicing?

MIG provides superior isolation by partitioning the GPU at the hardware level, giving each MIG instance its own dedicated memory, cache, and compute resources. This creates a strong, hardware-enforced boundary that time-slicing's software-level approach lacks. Unlike time-slicing where a "noisy neighbor" can cause an OOM error that crashes all other pods, a fault within one MIG instance is completely contained and cannot affect other instances on the same physical GPU.

Why is hardware-level GPU sharing not enough for full tenant isolation?

Hardware-level sharing isolates the GPU itself, but it does not isolate the Kubernetes control plane where tenants manage their workloads. Without control plane isolation, tenants share the same API server and can run into conflicts with CRDs, RBAC policies, and resource quotas. True tenant isolation requires isolating both the hardware and the management layer, which can be achieved with solutions like vCluster that create tenant clusters for each tenant.

Can I use time-slicing on top of a MIG partition?

Yes, you can combine MIG and time-slicing to create a tiered service offering. This allows you to leverage MIG for hardware-isolated partitions and then apply time-slicing within one of those partitions to achieve maximum density for lower-priority workloads. For example, you could partition a GPU into several MIG instances for premium customers while time-slicing a smaller MIG instance to serve a pool of development workloads.

‍