Skip to main content
Version: v0.32 Stable

Auto Sleep

Enterprise
Available in these plansFreeDevProdScale
Auto Sleep
Supported Configurations
Running the control plane as a container with:
Enterprise-Only Feature

This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.

Not all workloads need to run continuously. Scaling them down reduces costs. With auto sleep, you can scale workloads based on a set schedule or user activity and ingress.

warning

Auto sleep is intended for pre-production use cases only, and has limitations when used on a virtual cluster instance not connected to the platform.

Configure​

Enable auto sleep​

To enable auto sleep, add the following configuration to your vcluster.yaml:

Sleep configuration
sleep:
  auto:
    afterInactivity: 1h

Deployment example​

A deployment resource in Kubernetes manages a set of identical pods. Configuring auto sleep for a deployment scales down the pods while keeping the control plane active. This setup allows the virtual cluster to reduce resource usage while still being able to monitor activity and trigger wake-up actions when needed.

Control plane behavior
  • Without agent: The control plane remains active, allowing the virtual cluster instance to monitor activity and wake up automatically.
  • With agent: The platform agent can shut down the control plane completely for maximum resource savings.
Configure auto sleep with a deployment resource

Configure auto sleep for a deployment resource​

  1. Create the kind cluster.

    create kind cluster
    kind create cluster --name sleep-mode-demo

  2. Deploy a virtual cluster.

    Use the following vcluster.yaml to create a virtual cluster on your host. Save this file as vcluster.yaml

    vCluster config for auto sleep
    pro: true
    sleep:
      auto:
        afterInactivity: 30s
        exclude:
          selector:
            labels:
              sleep: no-thanks

    And run the following command:

    Create vCluster with autoSleep config
    vcluster create my-vcluster -f vcluster.yaml

    Workloads with the label sleep: no-thanks don't enter auto sleep after 30 seconds.

  3. Create demo deployments in your virtual cluster.

    Use the following deployment YAML to create two deployments.

    Example deployments
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: sleepy-deployment
      labels:
        app: sleepy-dep
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: demo-dep-1
      template:
        metadata:
          labels:
            app: demo-dep-1
        spec:
          containers:
            - command:
                - /agnhost
                - serve-hostname
                - --http=true
                - --port=8080
              image: registry.k8s.io/e2e-test-images/agnhost:2.39
              name: sleepy-demo

    ---

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: no-sleep-deployment
      labels:
        sleep: no-thanks
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: demo-dep-2
      template:
        metadata:
          labels:
            app: demo-dep-2
        spec:
          containers:
            - command:
                - /agnhost
                - serve-hostname
                - --http=true
                - --port=8080
              image: registry.k8s.io/e2e-test-images/agnhost:2.39
              name: not-sleepy-demo

    The first deployment does not have any special configurations for auto sleep. You can replace it with another deployment if needed. The second deployment includes a special label on the Deployment, preventing it from scaling down after 30 seconds.

    You can verify this by waiting 30 seconds and then getting information about the Deployments. For example

  4. Verify Deployments sleep status.

    deployment sleep check
    > sleep 30; kubectl get deployments
    NAMESPACE     NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
    default       no-sleep-deployment   2/2     2            2           1m
    default       sleepy-deployment     0/2     0            0           1m

    The sleepy-deployment reports 0/2 replicas after 30 seconds. Running kubectl counts as cluster activity, which is why it reports 0/2 instead of 0/0. The kubectl command triggers vCluster to update the replica count back to the original value of 2, but the replicas haven't become ready by the time kubectl get ... returns.

Next steps​

Experiment with the auto sleep feature by trying the following:

  • Add the sleep: no-thanks label to the first deployment and verify neither sleeps.
  • Remove the sleep: no-thanks label from both the deployments and verify that both go to sleep.

Ingress controller example​

An ingress controller, such as NGINX, manages external HTTP/S traffic to services in a Kubernetes cluster. To configure auto sleep for the ingress controller, make sure it stays responsive to incoming traffic and can wake up the cluster if needed. This setup keeps the ingress controller active, even when the virtual cluster instance is in auto sleep and allows the controller to handle requests that trigger the virtual cluster to wake up.

Configure auto sleep with an ingress resource

Configure auto sleep for an ingress controller​

  1. Create the kind cluster.

    Create a kind cluster
    kind create cluster --name ingress-demo --config - <<EOF
    kind: Cluster
    apiVersion: kind.x-k8s.io/v1alpha4
    networking:
      apiServerAddress: "0.0.0.0"
    nodes:
      - role: control-plane
        extraPortMappings:
          - containerPort: 80
            hostPort: 80
            protocol: TCP
    EOF

  2. [Deprecated]: Install the NGINX IngressController.

    install ingress controller
    helm install ingress-nginx ingress-nginx/ingress-nginx \
      --namespace ingress-nginx \
      --create-namespace \
      --set controller.dnsPolicy=ClusterFirstWithHostNet \
      --set controller.hostNetwork=true \
      --set controller.service.type=ClusterIP

  3. Create the vCluster.

    Use the following vcluster.yaml to create a virtual cluster on your host. Save this file as vcluster.yaml

    vCluster config for auto sleep
    pro: true
    sync:
      toHost:
        ingresses:
          enabled: true
    sleep:
      auto:
        afterInactivity: 30s

    And run the following command:

    Create vCluster with autoSleep config
    vcluster create my-vcluster -f vcluster.yaml

  4. Enable local DNS resolution for the virtual cluster.

    Add 127.0.0.1 backend.local to your /etc/hosts file to match the host configured in the Ingress rules of the next step.

  5. Create resources for the Ingress such as a Deployment and Service.

    Use the following manifest to create:

    • Namespace named bar
    • Deployment for the pods backing the Service
    • Service for the Ingress
    • Ingress resource
    Example deployments
    apiVersion: v1
    kind: Namespace
    metadata:
      name: bar

    ---

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: bar-deployment
      namespace: bar
      labels:
        app: bar-dep
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: bar
      template:
        metadata:
          labels:
            app: bar
        spec:
          containers:
            - command:
                - /agnhost
                - serve-hostname
                - --http=true
                - --port=8080
              image: registry.k8s.io/e2e-test-images/agnhost:2.39
              name: bar-app

    ---

    kind: Service
    apiVersion: v1
    metadata:
      name: bar-service
      namespace: bar
    spec:
      selector:
        app: bar
      ports:
        # Default port used by the image
        - port: 8080

    ---

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: example-ingress
      namespace: bar
    spec:
      ingressClassName: nginx # the ingress-nginx project has been deprecated, we recommend using a different ingress class
      rules:
        - http:
            paths:
              - pathType: Prefix
                path: /bar
                backend:
                  service:
                    name: bar-service
                    port:
                      number: 8080
          host: backend.local

  6. Verify the ingress is working properly with curl.

    Test the Ingress endpoint within the 30-second activity window by running curl --silent backend.local/bar. The pod name from the Deployment that responds is displayed.

  7. Allow the virtual cluster to go to sleep.

    Wait 30 seconds for the cluster to sleep, then run the curl command again. For convenience, run watch -d curl --silent backend.local/bar to repeatedly test the endpoint. On the first attempt, you’ll see the message Client sent an HTTP request to an HTTPS server. because an HTTP request was sent to the HTTPS wake endpoint. Subsequent requests display new pod names.

Istio gateway example​

note

Enable Istio integration for auto sleep to work with Istio resources.

When Istio is installed on the host cluster and the Istio integration is enabled, the virtual cluster instance syncs Gateway and VirtualService resources to the host cluster.

With auto sleep enabled, any traffic routed to a Service in the virtual cluster instance is tracked as activity and keeps the virtual cluster instance awake. If the vCluster is asleep, incoming traffic wakes it up.

Configure an ingress gateway for cluster external access
info

This example continues from the Istio integration example.

  1. Complete the Istio integration example.

  2. Install the Istio ingress gateway controller to allow routing from outside the cluster.

    Save the following manifest as ingress.yaml:

    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    metadata:
      name: ingress
    spec:
      profile: empty # Do not install CRDs or the control plane
      components:
        ingressGateways:
        - name: istio-ingressgateway
          namespace: istio-ingress
          enabled: true
          label:
            istio: ingressgateway
      values:
        gateways:
          istio-ingressgateway:
            injectionTemplate: gateway

  3. Create the istio-ingress namespace and install the ingress gateway:

    kubectl create namespace istio-ingress                                                                                                      
    istioctl install -f ingress.yaml

  4. Create a Gateway resource that uses a selector matching the newly installed ingress gateway in your vCluster.

    apiVersion: networking.istio.io/v1
    kind: Gateway
    metadata:
      name: istio-sm-gateway
      namespace: test
    spec:
      # The selector matches the ingress gateway pod labels.
      # If you installed Istio using Helm following the documentation, this is "istio=ingress"
      selector:
        istio: ingressgateway
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "smdemo.local"

  5. Update the vCluster with auto sleep enabled.

    Use the following vcluster.yaml to update your virtual cluster on your host cluster with --upgrade. Save this file as vcluster.yaml

    vCluster config for auto sleep
    pro: true
    sleep:
      auto:
        afterInactivity: 30s
    integrations:
      istio:
        enabled: true

    Run the following command:

    Create vCluster with autoSleep config
    vcluster create <your-vcluster-name> -f vcluster.yaml --upgrade

  6. Enable local DNS resolution for the virtual cluster.

    Add 127.0.0.1 smdemo.local to your /etc/hosts file to match the host configured in the Gateway configuration.

  7. Add the Gateway you created and the smdemo.local host to the VirtualService from the [Istio integration example](../vcluster- yaml/integrations/istio). The updated VirtualService should look like the following:

    Example resources
    apiVersion: networking.istio.io/v1
    kind: VirtualService
    metadata:
      name: nginx-service
      namespace: test
    spec:
      gateways:
        - istio-sm-gateway
      hosts:
        - smdemo.local
      http:
        - match:
            - uri:
                prefix: /v2
          name: nginx-v2
          rewrite:
            uri: /
          route:
            - destination:
                host: nginx-service.test.svc.cluster.local
                subset: v2
        - name: nginx-v1
          route:
            - destination:
                host: nginx-service.test.svc.cluster.local
                subset: v1

  8. Use curl to verify that the Gateway is working correctly.

    Test the Gateway endpoint within the 30-second activity window by running either curl --silent smdemo.local/v1 or curl --silent smdemo.local/v2.

  9. Allow the virtual cluster to go to sleep.

    Wait 30 seconds for the cluster to sleep. Then run the curl command again. To repeatedly test the endpoint, run watch -d curl --silent smdemo.local/v2. While the cluster is asleep or waking up, a 503 error is displayed.

Enable auto sleep with label selectors and schedules​

Use label selectors and schedules to configure auto sleep based on inactivity or specific timing:

Auto sleep label selectors and schedule
Sleep after 3 hours of inactivity, anything that does not have the label dont=sleep
sleep:
  auto:
    afterInactivity: 3h  # Uses Go duration with a max unit of hour
    exclude:
      selector:
        labels:
          dont: sleep
Sleep every Friday at 17:30 and wake every Monday at 7:00 in Mountain timezone
sleep:
  auto:
    timezone: America/Denver
    schedule: 30 17 * * 5
    wakeup:
      schedule: 0 7 * * 1

Config reference​

sleep required object ​

Sleep holds configuration for automatically putting the virtual cluster to sleep.

auto required object ​

Auto holds automatic sleep configuration

afterInactivity required string ​

AfterInactivity represents how long a vCluster can be idle before workloads are automatically put to sleep

schedule required string ​

Schedule represents a cron schedule for when to sleep workloads

exclude required object ​

Exclude holds configuration for labels that, if present, will prevent a workload from going to sleep

selector required object ​
labels required object ​

Labels defines what labels should be looked for

wakeup required object ​

Wakeup holds configuration for waking the vCluster on a schedule

schedule required string ​

timezone required string ​

Timezone specifies time zone used for scheduled sleep operations. Defaults to UTC. Accepts the same format as time.LoadLocation() in Go (https://pkg.go.dev/time#LoadLocation). The value should be a location name corresponding to a file in the IANA Time Zone database, such as "America/New_York".