How Ada Empowers Developers With Easier Kubernetes Testing Using vCluster

Getting In Each Others’ Way

Initially, we tried to have everyone share 2 development clusters, but found that the developers were overwriting each others’ work. We actually had a shared calendar and required developers to reserve time on the cluster to debug and test.

We thought about giving each developer their own cluster and a copy of our entire stack, but we determined it would be way too costly and also difficult for each developer to keep their cluster and stack up-to-date.

In another attempt, we also tried running the tests using our CI pipelines inside separate Kubernetes namespaces, but some parts of our application had the namespace hardcoded which made sharing a cluster for CI impossible.

Drawbacks

• Needed to go through a PR and approval just to get an environment for testing or demos

• Have to run the entire heavyweight stack locally during development, including large datastores
• Developers had to wait for an open slot on the calendar to test their code against the rest of the stack

Fast Ephemeral Test Environments

We looked at Okteto and Roost before settling on vCluster, and tried developing our own solution as well. Other solutions give you a unique environment, but vCluster seemed to be the most enterprise-grade solution with more customization options and advanced features.

Especially the virtual clusters in vCluster seem like a game-changer because they allow us to provision entire clusters as needed without the huge cost implications and with far less management overhead.

With vCluster, developers at Ada can now run integration tests themselves and provision ephemeral Kubernetes dev and test environments whenever they need them.

Advantages

• vCluster integrates with our SSO provider and our CI tooling which made adopting it easier.
• Automatic RBAC handling and group mapping right from our SSO provider.
• Engineers use Loft and DevSpace to work directly inside Kubernetes rather than having to spin up the entire stack locally.
• Namespaces can be deleted or put to sleep when not needed to save a large chunk of cost.

Easy Integrations With Our Existing Tools

We like vCluster and DevSpace because they can integrate with our existing tech stack, CI tooling and even our SSO provider. Even our CI pipeline now uses devspace deploy to create a namespace or a virtual cluster, run our integration tests, and then tear everything down again afterward.

The teams developing our most critical services now use devspace dev for their day-to-day development. We’re looking forward to working with Loft to teach the rest of our teams how to use devspace dev too.