Enterprise
Available in these plans	Free	Dev	Prod	Scale
Auto Sleep for Virtual Clusters

info

Platform auto sleep has been integrated into the standard sleep configuration and is no longer a separate feature.

Auto sleep and auto delete for tenant clusters

The platform provides two features to reduce Kubernetes costs:

• Auto Sleep puts tenant clusters to sleep when nobody is using them. This removes all pods while keeping all resources inside the tenant clusters during periods of inactivity.
• Auto-delete deletes tenant clusters that have been idle for a while.

Both features rely on the platform's inactivity detection.

Work with auto sleep

With auto sleep, you can put tenant clusters to sleep which means that vCluster Platform sets replicas: 0 for all replica-controlled resources such as Deployments and StatefulSets inside the tenant cluster. This means that Kubernetes deletes all pods but the entire configuration of resources within the tenant cluster is still there.

Auto sleep can be:

• Invoked manually

• Triggered by an inactivity timeout (no one has run a kubectl command or accessed an ingress in this tenant cluster for X minutes)

• Scheduled using a CRON syntax

Manual and automatic auto sleep

Start sleep manually

1. Click on Spaces to find the tenant cluster that you want to put to sleep.

2. Hover over the row of the tenant cluster that you want to put to sleep.

3. Click on the vertical ellipsis button and select Sleep to put the tenant cluster to sleep.

4. Notice how the Status column shows that the tenant cluster is now sleeping.

Automatic Wakeup

Note that the tenant cluster will automatically wake up again, once you run a kubectl command or send a Kubernetes request via another tool to the tenant cluster.

Configure auto sleep

Manual Support only

This only works for vCluster instances without a template, if you want to change the auto sleep configuration for a vCluster instance created by a template, it must be configured in the template.

1. Click on Spaces to find the vCluster instance that you want to configure auto sleep for.

2. Hover over the row of the vCluster instance that you want to configure

3. Click on Edit on the vCluster instance that you want to edit.

4. Use the Sleep After Inactivity (minutes) field to specify the time to wait before putting the vCluster instance to sleep if there is no more user activity in this vCluster instance.

5. Click on the button to save the changes.

Wake up tenant cluster

1. In the Spaces view, hover over the tenant cluster that you want to wake up.

2. Click on the vertical ellipsis button and select Wake Up to wake up the tenant cluster.

3. Notice how the Status column shows that the tenant cluster is now running again after a Pending state.

Scheduled auto sleep

Configure scheduled sleep and wake-up

Manual Support only

This only works for tenant clusters without a template, if you want to change the auto sleep configuration for a tenant cluster created by a template, it must be configured in the template.

1. In the Spaces view, hover over the tenant cluster that you want to configure auto sleep for.

2. Click on the button.

3. In the Definition section, under the Sleep Mode section. Use the Sleep Schedule field and/or the Wake-Up Schedule field to specify the Cronjob Times when the respective tenant cluster should be put to sleep or woken up.

4. [Optional] Input a Schedule Timezone for the timezone to apply the schedule.

5. Click on the button to save the changes.

Work with auto-delete

The platform lets you configure auto-delete for tenant clusters that have not been used for a certain period of time.

Configure auto-delete timeout

Manual Support only

This only works for tenant clusters without a template, if you want to change the auto sleep configuration for a tenant cluster created by a template, it must be configured in the template.

1. In the Spaces view, hover over the tenant cluster that you want to configure auto-delete for.

2. Click on the button.

3. Under the Sleep Mode section. Use the Delete After Inactivity (minutes) field to specify the time to wait before deleting the tenant cluster if there was no more user activity within this tenant cluster.

4. Click on the button to save the changes.

Inactivity detection

Last activity update interval

The platform periodically checks for activity to determine when an environment should enter or remain in auto sleep. This check interval—called the Auto Sleep Last Activity Update Interval—defines how often the platform updates the "last active" timestamp for each environment.

By default, the interval is 5 minutes, meaning that the inactivity timeout will only be reset if it occurs more than 5 minutes after the previous update.

You can customize this interval by setting the platform environment variable SLEEP_MODE_LAST_ACTIVITY_UPDATE_INTERVAL via the vCluster-platform chart Helm values:

env:
  SLEEP_MODE_LAST_ACTIVITY_UPDATE_INTERVAL: "1m"

Virtual cluster Kubernetes API requests

All virtual cluster Kubernetes API requests that are made through the platform count as activity in the tenant cluster.

If your kube-context points to the platform's API server as a proxy before the actual connected cluster's API server, every kubectl request will be an activity and reset the inactivity timeout.

Ingress Requests

For ingress-nginx (deprecated) based Ingresses and Istio Gateways, activity detection works automatically. For nginx based ingresses, the platform adds the mirror-target annotation to each ingress that will track activity and reset the timer when a request is made to that ingress. For Istio, the metrics from the istio-proxy container are used to determine activity.

Ingress Wakeup

vCluster Platform supports waking up a tenant cluster through an ingress independent of the underlying ingress controller. Just navigate to the ingress host and you should see a wakeup page for the tenant cluster.

Advanced configuration

Exclude resources from auto sleep

The platform allows you to specify resources that should not sleep within a tenant cluster by providing the annotation sleepmode.loft.sh/exclude: 'true' on either a Deployment, StatefulSet, ReplicaSet or Pod. For example the following Deployment would not sleep, if the tenant cluster is sleeping:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
  annotations:
    sleepmode.loft.sh/exclude: "true"
spec:
  replicas: 2
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
      containers:
        - name: busybox
          image: busybox
          command:
            - sleep
            - "3600"

Long-living connections

It's possible that certain requests such as kubectl exec or kubectl port-forward keep an active connection open to the tenant cluster that prevents it from sleeping. This is in most cases wanted since the tenant cluster shouldn't start sleeping when someone is still clearly using it, however there are cases where the connection might be idle (someone left the laptop open, but is not using it anymore) in which you want to terminate such connections and put the tenant cluster to sleep.

There are multiple ways to approach this problem of active connections preventing a tenant cluster from sleeping:

1. You can tell the platform to timeout idle streaming connections (such as kubectl exec, kubectl port-forward etc.) after a certain time period with the annotation loft.sh/streaming-connection-idle-timeout: '3600' on a cluster. With this annotation set, the platform will close connections automatically that are idle after the given seconds. By default, the platform will not timeout any connections. This will only apply to new opened connections to that cluster and not affect already running connections.
2. Tell the platform to ignore all active connections for determining tenant cluster activity via the annotation sleepmode.loft.sh/ignore-active-conntections: 'true'. This will put a tenant cluster to sleep even though there still might be open connections such as kubectl exec or kubectl port-forward.
3. Configure your kubelets with the flag --streaming-connection-idle-timeout duration(see Kubernetes docs). This behaves essentially as option 1, however this configuration is independent of the platform.

For detailed vCluster-specific auto sleep configuration and examples, see the vCluster auto sleep documentation.

Auto sleep operations

Auto sleep involves two main actions: sleeping and waking. These actions help save resources when the cluster is not in use.

Sleeping reduces resource consumption and costs during inactive periods.

• Delete bare pods: Pods not managed by controllers are removed permanently.
• Scale down resources: The following resources are scaled down to reduce resource usage:
  • Deployments
  • ReplicaSets
  • ReplicationControllers
  • DaemonSets

Waking restore scaled-down resources.

• Restore scaled-down resources: Deployments, ReplicaSets, ReplicationControllers, and DaemonSets are returned to their original state.
• Bare pods are not restored: Bare pods deleted during the Sleeping step cannot be recovered.

Without an agent

How it works: The tenant cluster instance manages auto sleep itself by scaling down workloads (pods, Deployments, StatefulSets, etc.) while keeping the control plane active.

What happens:

• Workloads are scaled down to zero replicas
• Control plane remains running (API server, etcd, syncer)
• Cluster can detect activity and wake up automatically
• Resource savings are moderate (workloads only)

Use when: You want automatic sleep/wake feature but don't have the platform or prefer instance-managed auto sleep.

With a platform agent

How it works: When connected to the platform using an agent, the platform agent takes over auto sleep management and can shut down the entire control plane.

What happens:

• Workloads are scaled down to zero replicas
• Control plane can be completely shut down (maximum resource savings)
• Platform manages sleep/wake operations
• Enables additional features like auto-deletion

Use when: You want maximum resource savings and advanced features like auto-deletion, or you're already using the platform for tenant cluster instance management.

Key differences

Feature	Without agent	With agent
Control plane	Remains active	Can be shut down completely
Resource savings	Moderate (workloads only)	Maximum (workloads + control plane)
Auto-deletion	Not available	Available
Management	tenant cluster instance manages itself	Platform agent manages
Activity detection	Supported	Supported
Scheduling	Supported	Supported

Resource exemption in auto sleep

With auto sleep, you can prevent specific resources from entering sleep by configuring them with the following options:

• Add the annotation sleepmode.loft.sh/exclude: true to the resource.
• Configure sleep with specific labels to define which resources should remain active. This allows targeting specific resources based on their labels.
• Add labels to workloads that you want to remain active. By using labels, it is possible to control which resources are exempt from sleep at a more granular level.

For example, the following Deployment would not sleep when the tenant cluster is sleeping:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
  annotations:
    sleepmode.loft.sh/exclude: "true"
spec:
  replicas: 2
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
      containers:
        - name: busybox
          image: busybox
          command:
            - sleep
            - "3600"

Detect activity in auto sleep

Auto sleep tracks certain actions to detect activity and wake the cluster when necessary. To wake a sleeping cluster or update the last active time, auto sleep captures the following actions:

• Accessing cluster resources through API calls (for example kubectl get <resource>).
• Attempting to contact ingress endpoints (NGINX and Istio only).

note

Ingress activity detection with the ingress-nginx (deprecated) ingress controller relies on the mirror-target annotation, which will overwrite any previously set mirror-target annotation.

Last activity update interval

The platform periodically checks for activity to determine when an environment should enter or remain in auto sleep. This check interval—called the Auto Sleep Last Activity Update Interval—defines how often the platform updates the "last active" timestamp for each environment.

By default, the interval is 5 minutes, meaning that the inactivity timeout will only be reset if it occurs more than 5 minutes after the previous update.

You can customize this interval by setting the platform environment variable SLEEP_MODE_LAST_ACTIVITY_UPDATE_INTERVAL via the vCluster-platform chart Helm values:

env:
  SLEEP_MODE_LAST_ACTIVITY_UPDATE_INTERVAL: "1m"

Ingress Wakeup

vCluster Platform supports waking up a tenant cluster through an ingress independent of the underlying ingress controller. Just navigate to the ingress host and you should see a wakeup page for the tenant cluster.

Ignore other types of activity in auto sleep

The resource exemption feature keeps specifically configured resources active when the tenant cluster instance enters auto sleep. The Configure request handling for auto sleep table describes how to ignore requests that keep the cluster awake or wake it up if it is asleep.

Configure request handling for auto sleep table

Annotation	Behavior	Format
sleepmode.loft.sh/ignore-all	Ignore all activity.	true or false
sleepmode.loft.sh/ignore-ingresses	Ignore requests to associated ingresses.	true or false
sleepmode.loft.sh/ignore-groups	Ignore specified groups the requesting user belongs to.	group2,group3
sleepmode.loft.sh/ignore-resources	Ignore requests to specific resource types.	pods,resource2,resource3
sleepmode.loft.sh/ignore-verbs	Ignore requests with specific verbs.	create,verb1,verb2
sleepmode.loft.sh/ignore-resource-verbs	Ignore requests with specific verbs to specific resources.	myresource.mygroup=create update delete, myresource2.mygroup=create update
sleepmode.loft.sh/ignore-resource-names	Ignore requests to specific resources with specific names.	myresource.mygroup=name1 name2,myotherresource.mygroup=name1
sleepmode.loft.sh/ignore-user-agents	Ignore specific user agents with trailing wildcard support.	kubectl*,argo,useragent3
sleepmode.loft.sh/disable-ingress-wakeup	Ignore ingress access.	true

note

Add the annotations to the tenant cluster instance workload, such as the StatefulSet or Deployment running the tenant cluster. You can dynamically ignore any request by adding the X-Sleep-Mode-Ignore header.

Long-living connections

It's possible that certain requests such as kubectl exec or kubectl port-forward keep an active connection open to the tenant cluster that prevents it from sleeping. This is in most cases wanted since the tenant cluster shouldn't start sleeping when someone is still clearly using it, however there are cases where the connection might be idle (someone left the laptop open, but is not using it anymore) in which you want to end such connections and put the tenant cluster to sleep.

There are multiple ways to approach this problem of active connections preventing a tenant cluster from sleeping:

1. You can tell the platform to timeout idle streaming connections (such as kubectl exec, kubectl port-forward etc.) after a certain time period with the annotation loft.sh/streaming-connection-idle-timeout: '3600' on a cluster. With this annotation set, the platform will close connections automatically that are idle after the given seconds. By default, the platform will not timeout any connections. This will only apply to new opened connections to that cluster and not affect already running connections.
2. Tell the platform to ignore all active connections for determining tenant cluster activity via the annotation sleepmode.loft.sh/ignore-active-conntections: 'true'. This will put a tenant cluster to sleep even though there still might be open connections such as kubectl exec or kubectl port-forward.
3. Configure your kubelets with the flag --streaming-connection-idle-timeout duration (see Kubernetes docs). This behaves essentially as option 1, however this configuration is independent of the platform.

Connect to the platform

When you connect a tenant cluster instance that has sleep configured to the platform (via an agent), the platform agent automatically takes over auto sleep management. The tenant cluster instance stops managing its own auto sleep and the agent handles all sleep/wake operations.

Transition from instance-managed to agent-managed auto sleep

When a tenant cluster instance with sleep configured connects to the platform:

• The platform reads the existing sleep configuration
• The agent takes over auto sleep management automatically
• No configuration changes are needed (for vCluster 0.24.0+ and platform 4.3.0+)
• The tenant cluster instance's sleep configuration continues to work, but is now managed by the platform agent

This requires compatible versions of tenant cluster instance and the platform (vCluster 0.24.0+ and platform 4.3.0+).

Ingress configuration

Sync to host

If you install ingress controllers in the tenant cluster instance, prevent them from sleeping. This keeps the controllers active and allows them to handle requests that would wake the tenant cluster instance. If you install the ingress controller in the control plane cluster, enable ingress syncing to the host.

Enable ingress syncing

sync:
  toHost:
    ingresses:
      enabled: true

Reachability for ingress activity detection

To enable ingress activity detection in your tenant cluster instance, ensure the following conditions are met:

• The ingress controller needs to be able to discover the tenant cluster instance pod to route traffic correctly.
• Ensure that the DNS lookup for <vcluster-namespace>.<vcluster-svc-name>.svc.cluster.local resolves properly. This allows the ingress controller to reach the tenant cluster instance's service address.
• Ensure that the correct dnsPolicy is set in the ingress controller's Helm chart. This ensures DNS resolution works as expected and allows communication between the ingress controller and the tenant cluster instance.