MANAGED INFERENCE
Managed inference, built your way.
Give every customer and every internal team a private inference endpoint in minutes. Compose the service from the engines and tools you choose, and run it on your own infrastructure.
Trusted by AI clouds and AI factories across 100K+ GPUs
The problem
Everyone wants inference. Few teams can serve it.
OPTION A
Build it yourself
Months of work and a dedicated platform team.
OPTION B
Adopt a rigid, one-size service
One opinionated path that can’t bend to your hardware, models or users.
EITHER WAY
Your GPUs sit behind a queue
Teams and customers wait while GPUs sit underused.
A rigid service vs. vCluster
Rigid, one-size service
Inference engine
One engine, chosen for you
Any engine, swap anytime
Your hardware
Built for their reference setup
Runs on your GPUs and network
When needs change
Wait for their roadmap
Change the Stack and redeploy
WHO IT’S FOR
One platform, two ways to put it to work.
Serve your own teams, or build an offering for your customers.
Enterprise AI factories
Private inference for every internal team: Every team gets its own endpoint. No ticket queue, no sharing.
- An endpoint per team or app
- Showback per team
- Models and data stay in-house
- Any engine, no lock-in
AI cloud providers
The foundation for your own inference offering: Offer managed inference under your brand, not just GPU-hours.
- An isolated endpoint per customer
- Token usage tracked per customer
- Your brand and pricing
- Sovereign by design: data stays in your data center
HOW IT WORKS
From bare metal to a private endpoint in minutes.
One definition creates the cluster, installs the inference service in the right order, and hands over an endpoint.
1
Machines
vMetal provisions GPU nodes as bare metal or VMs through one stable API.
2
Cluster
Each customer or team gets an isolated cluster on Private Nodes: dedicated GPUs, no sharing.
3
Inference stack
The Stack installs the engine, gateway, autoscaling and observability, health-gated step by step.
4
Endpoint
Your user gets an endpoint, API keys and a usage view.
INSIDE AN INFERENCE STACK
Opinionated defaults. Swap anything.
A single, rigid offering doesn’t fit the reality on the ground. Here’s an example inference Stack, step by step. Start with the defaults, or swap any layer for the tool you already trust.
Secrets
Database password and gateway master key, generated per deployment.
Kubernetes Secrets
Model servers
The inference engine that loads and serves the model.
Autoscaling
Adds model server replicas as requests queue up, removes them as traffic drops.
KEDA
Metrics
Time to first token, output tokens per second, throughput.
Prometheus
Database
API keys, usage records and service state.
Postgres
Rate limiting
Per-key and per-customer limits.
Valkey
Gateway
One OpenAI-compatible endpoint, so existing apps and SDKs work unchanged.
LLM gateway
Verify
Health-checks the service and reports the endpoint, latency and throughput.
Built-in
Every step is open and forkable in the vClusterStacks repository, alongside partner Stacks like Saturn Cloud’s AI token factory.
YOUR USERS’ EXPERIENCE
Your users see an endpoint. Everything else is yours to run.
Customers and teams see the service, nothing beneath it.
A private endpoint
Its own isolated cluster. Nothing else visible.
Works with existing SDKs
Point existing code at it. No new libraries.
Self-service API keys
Create, revoke and block their own keys.
Usage and a playground
See tokens and latency. Test models first.
OPERATOR EXPERIENCE
Your hundredth endpoint costs what your first did.
Each new customer or team is a deployment, not a project.
One console for every endpoint
Every customer and team in one view.
Isolation by default
Its own cluster and nodes for every owner.
Showback built in
Tokens and cost per owner, from the gateway.
API first, your portal
Use our console or put your own in front.
SEE IT WORK
Watch an inference service go live.
“Adding a service to an AI cloud shouldn’t mean starting a new integration project each time. vCluster built the Saturn Cloud Stack, so an operator can deploy it like any other service. Each customer gets private inference in an environment of their own, ready to use right away, and a small team can keep adding services without adding headcount.”
Hugo Shi
Co-founder and CTO, Saturn Cloud
Co-founder and CTO, Saturn Cloud
No items found.
THE SAME MACHINERY
Inference is one more cluster type, not another platform.
Same control plane virtualization, same isolation model, same Platform console you use for every other cluster you run.
Kubernetes Clusters
Conformant clusters with their own control plane per tenant.
Slurm Clusters
Tenants see only their own partition and jobs.
NVIDIA Run:ai Clusters
Tenants see only their own projects and jobs.
Ray Clusters
Tenants see only their own head node and workers.
Launch managed inference on your infrastructure.
Get a live walkthrough built around your GPUs, your engines and your users.
