Skip to main content
Version: main 🚧

Configure External Access & TLS

After deploying the platform, you may want to access the deployment via a routable IP address or a resolvable domain name. You may also wish to secure the deployment with a TLS certificate. This is particularly useful for sharing platform access with other team members who may or may not have direct access to Kubernetes.

Requirements

Setting up the platform for access via routable IP or domain name, and securing it via TLS requires the following:

  • The cluster where the platform is deployed must have an available load balancer or ingress controller
  • Appropriate DNS settings must be in place
  • A mechanism for assigning TLS certificates (such as cert-manager) must be available

External access​

The platform, like any other service in Kubernetes, can be exposed in multiple ways. The LoadBalancer method is recommended. The NGINX ingress controller examples below are preserved for reference but the ingress-nginx project is deprecated upstream. Use a LoadBalancer or a Gateway API-compatible controller for new deployments.

[Deprecated]: External access via NGINX ingress controller​

Deprecated

The ingress-nginx project has been deprecated. We recommend using a different ingress controller.

Heads up.

This section assumes you already have the nginx ingress controller installed, if you don't, check out the 'Manual Ingress Controller Installation' tab.

  1. Run the command:

    Start the platform
    vcluster platform start --host=vcluster-platform.mydomain.tld
    Update the Domain.

    Make sure you update the hostname in the command.

  2. Set the VERSION variable to the platform version you want to upgrade to or set it to the current version using:

    Set VERSION variable
    CHART=$(kubectl get service loft -n vcluster-platform -o jsonpath={.metadata.labels.chart})
    VERSION=${CHART:5}

  3. To upgrade the platform via vCluster CLI, update $PLATFORM_VERSION with a valid platform version and run:

    Upgrade the platform
    vcluster platform start --upgrade --version=$PLATFORM_VERSION --values=vcluster-platform.yaml
    Versions and Values

    The $PLATFORM_VERSION variable is an environment variable set to your desired platform version to deploy. The vcluster-platform.yaml file is an optional YAML file that contains Helm values you would like to use when upgrading your platform deployment.

  4. Determine the External-IP address:

    Get the loft-ingress external IP address
    kubectl get ingress -n vcluster-platform
    NAME           CLASS    HOSTS                ADDRESS          PORTS     AGE
    loft-ingress   <none>   vcluster-platform.mydomain.tld    x.x.x.x          80, 443   10m
    Address

    If you never get a valid IP Address in the ADDRESS column shown in the command, you might need to work with your Kubernetes administrator to ensure your ingress is properly configured.

  5. Set up a DNS A record to the ingress address. Make sure the platform is reachable at the address, you can check that with curl as shown below, or simply browse to the address in your browser of choice:

    Check the platform version
    curl https://vcluster-platform.mydomain.tld/version --insecure | jq
    {
      "kind": "Version",
      "apiVersion": "version.loft.sh",
      "metadata": {
        "creationTimestamp": null
      },
      "version": "v3.0.0",
      "major": "3",
      "minor": "0",
      "instance": "",
      "kubeVersion": "v1.24.2",
      "newerVersion": "",
      "shouldUpgrade": false
    }

External access via LoadBalancer​

    AWS Load Balancers

    If you are using AWS, make sure you are using a Network Load Balancer (NLB) to route traffic, since other load balancers do not support the SPDY protocol Kubernetes requires.

    Create a LoadBalancer Service
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        # Make sure to adjust the next line:
        service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:eu-west-2:xxx:certificate/xxx"
        service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
        service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
      name: vcluster-platform-loadbalancer
      namespace: vcluster-platform
    spec:
      type: LoadBalancer
      ports:
        - name: http
          port: 80
          protocol: TCP
          targetPort: 8080
        - name: https
          port: 443
          protocol: TCP
          targetPort: 10443
      selector:
        app: loft

  2. Create the load balancer with this command:

    Create the LoadBalancer Service
    kubectl apply -f vcluster-platform-loadbalancer.yaml

  3. Wait until the load balancer receives an External-IP address:

    Get the vcluster-platform-loadbalancer external IP address
    kubectl get svc vcluster-platform-loadbalancer -n vcluster-platform
    NAME                            TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)          AGE
    vcluster-platform-loadbalancer       LoadBalancer   10.112.2.142    x.x.x.x          443:30933/TCP    3m16s

  4. Make sure the platform is reachable at the load balancer's external address, you can check that with curl as shown below, or simply browse to the address in your browser of choice:

    Check the platform version
    curl https://x.x.x.x/version --insecure | jq
    {
      "kind": "Version",
      "apiVersion": "version.loft.sh",
      "metadata": {
        "creationTimestamp": null
      },
      "version": "v3.0.0",
      "major": "3",
      "minor": "0",
      "instance": "",
      "kubeVersion": "v1.24.2",
      "newerVersion": "",
      "shouldUpgrade": false
    }

Configure TLS​

This section provides various options for configuring TLS for your platform deployment. Choose the method that best fits your infrastructure and requirements.

Cert-Manager​

  1. Install cert-manager to your cluster:

    Install cert-manager
    helm upgrade --install  cert-manager cert-manager  --repository-config=''\
      --namespace cert-manager  --create-namespace \
      --repo https://charts.jetstack.io \
      --set installCRDs=true \
      --wait

  2. Edit your existing vcluster-platform.yaml file, or create a new file named vcluster-platform.yaml with the following content:

    vcluster-platform.yaml
    ingress:
      enabled: true
      annotations:
        # Make sure the following line matches the name of your issuer (or use the section below to create one)
        cert-manager.io/cluster-issuer: lets-encrypt-http-issuer
      tls:
        enabled: true
        secret: tls-vcluster-platform

    certIssuer:
      create: true # Change this if you already have your own cert-issuer
      name: lets-encrypt-http-issuer
      email: "YOUR_EMAIL" # REQUIRED
      secretName: vcluster-platform-letsencrypt-credentials
      httpResolver:
        enabled: true
        ingressClass: nginx # the ingress-nginx project has been deprecated
      resolvers: []
      server: https://acme-v02.api.letsencrypt.org/directory

  3. Set the VERSION variable to the platform version you want to upgrade to or set it to the current version using:

    Set VERSION variable
    CHART=$(kubectl get service loft -n vcluster-platform -o jsonpath={.metadata.labels.chart})
    VERSION=${CHART:5}

  4. Upgrade the platform

    Upgrade considerations
    • Review the release notes for the target version to understand any breaking changes or new features.
    • Test the upgrade in a non-production environment before applying it to your production setup.

    Upgrade the platform via:

    To upgrade the platform using the vcluster CLI, update $PLATFORM_VERSION with a valid platform version and run:

    Upgrade the platform using vCluster CLI
    RELEASE_NAMESPACE=vcluster-platform # Replace with the namespace of the platform deployment if different
    PLATFORM_VERSION='' # Set this to a specific version or leave empty for latest

    vcluster platform start --upgrade --version=$PLATFORM_VERSION --namespace=$RELEASE_NAMESPACE --values=vcluster-platform.yaml

AWS Certificate Manager (ACM)​

  1. Determine the External-IP address of your ingress:

    Get ingress external IP
    kubectl get ingress -n vcluster-platform
    NAME           CLASS    HOSTS                ADDRESS          PORTS     AGE
    loft-ingress   <none>   vcluster-platform.mydomain.tld    x.x.x.x          80, 443   10m
                                                 ^^^^^^^

  2. Find the AWS Elastic Load Balancer (ELB) for this IP address in the AWS console.

  3. Switch to the "Listeners" tab.

  4. In the "SSL Certificates" column, click on the "View/edit certificates" link.

  5. Click on the "+" symbol next to the "Certificates" tab and add your Access Control Manager (ACM) managed certificate to the ingress controller's Load Balancer.

Manually provisioned certificate​

  1. Create a Kubernetes secret from your certificate:

    Create TLS secret
    kubectl create secret generic tls-vcluster-platform -n vcluster-platform --type=kubernetes.io/tls \
      --from-file=tls.crt=tls.crt \
      --from-file=tls.key=tls.key

  2. Edit your existing vcluster-platform.yaml file, or create a new file named vcluster-platform.yaml with the following content:

    vcluster-platform.yaml for ingress TLS
    ingress:
      tls:
        enabled: true
        secret: tls-vcluster-platform # Make sure this matches the name of your cert from the previous step

    If you want to configure the platform with additionalCA, the Certificate Authority used for signing this certificate has to be added as a base64 encoded value:

    Encode CA certificate
    cat ca.crt | base64

    Use this output in the .additionalCA helm value.

  3. Set the VERSION variable to the platform version you want to upgrade to or set it to the current version using:

    Set VERSION variable
    CHART=$(kubectl get service loft -n vcluster-platform -o jsonpath={.metadata.labels.chart})
    VERSION=${CHART:5}

  4. Upgrade the platform

    Upgrade considerations
    • Review the release notes for the target version to understand any breaking changes or new features.
    • Test the upgrade in a non-production environment before applying it to your production setup.

    Upgrade the platform via:

    To upgrade the platform using the vcluster CLI, update $PLATFORM_VERSION with a valid platform version and run:

    Upgrade the platform using vCluster CLI
    RELEASE_NAMESPACE=vcluster-platform # Replace with the namespace of the platform deployment if different
    PLATFORM_VERSION='' # Set this to a specific version or leave empty for latest

    vcluster platform start --upgrade --version=$PLATFORM_VERSION --namespace=$RELEASE_NAMESPACE --values=vcluster-platform.yaml

Self-signed certificate​

For testing or internal use, you can create and use a self-signed certificate.

  1. Create a new private key for Certificate Authority:

    Generate CA private key
    openssl genrsa 2048 > ca-key.pem

  2. Create a new certificate authority:

    Generate CA certificate
    openssl req -new -x509 -nodes -days 365000 -key ca-key.pem -out ca.crt

  3. Create a new private key:

    Generate private key
    openssl genrsa -out tls.key 4096

  4. Create a file named ssl.conf with the following content:

    ssl.conf
    [ req ]
    default_bits       = 4096
    distinguished_name = req_distinguished_name
    x509_extensions    = v3_ca
    req_extensions     = v3_req
    x509_extensions    = usr_cert

    [ req_distinguished_name ]
    organizationName          = Organization Name (for example company)
    organizationName_default  = vcluster-platform
    commonName                = Common Name (for example server FQDN or YOUR name)
    commonName_default        = vcluster-platform.mydomain.tld

    [ usr_cert ]
    basicConstraints  = CA:FALSE
    nsCertType        = client, server
    keyUsage          = digitalSignature
    extendedKeyUsage  = serverAuth, clientAuth

    [ v3_req ]
    subjectAltName   = @alt_names
    extendedKeyUsage = serverAuth, clientAuth
    basicConstraints = CA:FALSE
    keyUsage         = digitalSignature

    [ alt_names ]
    DNS.1 = localhost

    Set commonName_default and DNS.1 to the domain you want to use.

  5. Create a certificate signing request:

    Generate certificate signing request
    openssl req -new -sha256 \
        -out tls.csr \
        -key tls.key \
        -config ssl.conf

  6. Generate the certificate:

    Generate certificate
    openssl x509 -req \
        -sha256 \
        -days 3650 \
        -in tls.csr \
        -out tls.crt \
        -extensions v3_req \
        -extfile ssl.conf \
        -CA ca.crt \
        -CAkey ca-key.pem

  7. Create a Kubernetes secret from your certificate:

    Create TLS secret
    kubectl create secret generic tls-vcluster-platform -n vcluster-platform --type=kubernetes.io/tls \
      --from-file=tls.crt=tls.crt \
      --from-file=tls.key=tls.key

    If you want to configure the platform with additionalCA, use:

    Encode CA certificate
    cat ca.crt | base64

    as an input for the .additionalCA helm value.

  8. Edit your existing vcluster-platform.yaml file, or create a new file named vcluster-platform.yaml with the following content:

    vcluster-platform.yaml for ingress TLS
    ingress:
      tls:
        enabled: true
        secret: tls-vcluster-platform # Make sure this matches the name of your cert from the previous step

    tls:
      enabled: true # Virtual clusters running on the same host as the platform directly use the platform service.
      secret: tls-vcluster-platform # Ensure this matches the name of your cert from the previous step

  9. Set the VERSION variable to the platform version you want to upgrade to or set it to the current version using:

    Set VERSION variable
    CHART=$(kubectl get service loft -n vcluster-platform -o jsonpath={.metadata.labels.chart})
    VERSION=${CHART:5}

  10. Upgrade the platform

    Upgrade considerations
    • Review the release notes for the target version to understand any breaking changes or new features.
    • Test the upgrade in a non-production environment before applying it to your production setup.

    Upgrade the platform via:

    To upgrade the platform using the vcluster CLI, update $PLATFORM_VERSION with a valid platform version and run:

    Upgrade the platform using vCluster CLI
    RELEASE_NAMESPACE=vcluster-platform # Replace with the namespace of the platform deployment if different
    PLATFORM_VERSION='' # Set this to a specific version or leave empty for latest

    vcluster platform start --upgrade --version=$PLATFORM_VERSION --namespace=$RELEASE_NAMESPACE --values=vcluster-platform.yaml

Import tenant clusters with certificate validation​

When importing externally managed tenant clusters, you can enable secure communication by using the same certificate authority configured for the platform.

  1. Get the current certificate authority value from your platform:
Get the certificate authority value
helm get values --namespace $RELEASE_NAMESPACE $RELEASE_NAME --all | grep additionalCA
  1. Import the tenant cluster with certificate validation:
Import a tenant cluster with certificate validation
vcluster platform add vcluster <VCLUSTER_NAME> \
  --namespace=<VCLUSTER_NAMESPACE> \
  --project=<PROJECT_NAME> \
  --ca-data <BASE64_CA_VALUE>

The imported tenant cluster restarts and reports Ready once it establishes a secure connection to the platform.