Skip to content

Install on Kubernetes Enterprise

The DataKitchen Helm chart installs DataOps Data Quality TestGen Enterprise into an existing Kubernetes cluster, with the application database on a PostgreSQL server outside the cluster.

This page covers Azure Kubernetes Service (AKS) and Amazon Elastic Kubernetes Service (EKS). The chart is platform-neutral, so the same steps apply to any conformant cluster — only the cluster prerequisites, ingress configuration, and DNS records differ.

Prerequisites

  • Request access to the private enterprise Docker image from your DataKitchen representative.
  • Install kubectl and Helm.
  • Choose the domain names for the TestGen UI and for the API and MCP server. Both are set during the install.

Kubernetes cluster

  • Kubernetes 1.34 or later.
  • The Amazon VPC CNI, kube-proxy, and CoreDNS add-ons.
  • A node group with at least one node of 8 vCPUs, 32 GB memory, and 150 GB disk, for example, t3.2xlarge.
  • kubectl access to the cluster.
  • Subnets tagged per the Application Load Balancer requirements, and the AWS Load Balancer Controller installed.
  • A certificate in AWS Certificate Manager covering both the UI and API domains. Note its ARN.

TestGen profiles and tests DuckDB databases and file data sources with the application pod's own memory and disk. If you plan to connect them, see Size TestGen for DuckDB and File Data Sources.

PostgreSQL server

  • PostgreSQL 14 or later, reachable from the cluster on its listening port (5432 by default).
  • An administrative account that can create users.
  • Recommended compute: 4 vCPUs and 128 GB disk, for example, Standard_D4ds_v5 on Azure Database for PostgreSQL or db.m6g.xlarge on Amazon RDS.

Prepare the application database

Connect to the PostgreSQL server with an administrative account and create the user that TestGen will use.

CREATE USER testgen WITH PASSWORD '<postgres password>' CREATEDB;

The CREATEDB privilege is required: on first start, TestGen creates its datakitchen database and the testgen schema.

Create the namespace and secrets

  1. Create the datakitchen namespace and make it the default for subsequent commands.

    kubectl create namespace datakitchen
    kubectl config set-context --current --namespace=datakitchen
    
  2. Create the testgen-secret secret, replacing the placeholders. Use the password of the testgen user you created for postgres-password, and generate a random value for each of the others.

    kubectl create secret generic testgen-secret \
        --from-literal=postgres-password='<postgres password>' \
        --from-literal=decrypt-salt='<random string>' \
        --from-literal=decrypt-password='<random string>' \
        --from-literal=jwt-hashing-key='<base64-encoded random string>'
    

    Warning

    Create this secret yourself and keep a copy of the values. TestGen encrypts the credentials of every database connection you configure using decrypt-salt and decrypt-password. If these values are lost or regenerated, the stored connection credentials can no longer be decrypted and every connection must be re-entered.

  3. Create the datakitchen-dockerhub registry secret, using the credentials provided by your DataKitchen representative.

    kubectl create secret docker-registry datakitchen-dockerhub \
        --docker-username='<username>' \
        --docker-password='<password>'
    
  4. Confirm that the cluster can reach the PostgreSQL server, and that the testgen user has the privilege the install needs. The value must come back as t.

    kubectl run pg-check -n datakitchen --rm -i --restart=Never \
        --image=postgres:16-alpine \
        --env="PGCONNECT_TIMEOUT=10" \
        --env="PGPASSWORD=$(kubectl get secret testgen-secret -n datakitchen -o jsonpath='{.data.postgres-password}' | base64 -d)" \
        --command -- psql -h <postgres host> -U testgen -d postgres \
        -c "SELECT rolcreatedb FROM pg_roles WHERE rolname = current_user;"
    

    Tip

    If the command hangs rather than returning an error, the server's network rules are dropping the connection. Check that the security group, firewall, or virtual network rules allow inbound connections from the cluster's node subnets.

  5. Confirm that the cluster can pull the enterprise image with the registry secret. This also warms the image on the node, so the install itself starts faster.

    kubectl run image-check -n datakitchen --rm -i --restart=Never \
        --image=datakitchen/dataops-testgen-enterprise:v<version> \
        --overrides='{"apiVersion":"v1","spec":{"imagePullSecrets":[{"name":"datakitchen-dockerhub"}]}}' \
        --command -- echo "Image pull succeeded"
    

    Tip

    An ImagePullBackOff means either the cluster has no egress to Docker Hub or the registry secret is wrong. Run kubectl describe pod image-check -n datakitchen to tell the two apart.

Install TestGen

  1. Add the DataKitchen Helm repository.

    helm repo add datakitchen-testgen https://datakitchen.github.io/dataops-testgen/
    helm repo update datakitchen-testgen
    
  2. Create a values-tg-app.yaml file and replace the placeholders. Keep this file — the same values are used for future upgrades.

    image:
      repository: datakitchen/dataops-testgen-enterprise
      tag: v<version>
    
    imagePullSecrets:
      - name: datakitchen-dockerhub
    
    testgen:
      databaseHost: <postgres host>
      databasePort: 5432
      databaseName: datakitchen
      databaseSchema: testgen
      databaseUser: testgen
      databasePasswordSecret:
        name: testgen-secret
        key: postgres-password
      authSecrets:
        create: false
        name: testgen-secret
      uiUser: admin
      uiPassword: <initial admin password>
      instanceId: <instance name>
      uiBaseUrl: https://<UI domain>
      baseUrl: https://<API domain>
    

    For what each value does, see the Values reference.

  3. Add the ingress configuration for your platform to values-tg-app.yaml. This publishes the UI and the REST API and MCP server, which the chart serves on separate ports of the same service.

    ingress:
      enabled: true
      className: webapprouting.kubernetes.azure.com
      hosts:
        - host: <UI domain>
          paths:
            - path: /
              pathType: Prefix
              servicePort: http
        - host: <API domain>
          paths:
            - path: /
              pathType: Prefix
              servicePort: api
    
    ingress:
      enabled: true
      className: alb
      annotations:
        alb.ingress.kubernetes.io/scheme: internet-facing
        alb.ingress.kubernetes.io/target-type: instance
        alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
        alb.ingress.kubernetes.io/ssl-redirect: "443"
        alb.ingress.kubernetes.io/certificate-arn: <certificate ARN>
        alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10
        alb.ingress.kubernetes.io/healthcheck-path: /_stcore/health
        alb.ingress.kubernetes.io/success-codes: 200-404
        alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=90
      hosts:
        - host: <UI domain>
          paths:
            - path: /
              pathType: Prefix
              servicePort: http
        - host: <API domain>
          paths:
            - path: /
              pathType: Prefix
              servicePort: api
    

    The health check path applies to every target group on the ingress, and the API does not serve the UI health path. success-codes: 200-404 accepts both responses so that each target group reports healthy.

  4. Install the chart. The --wait flag holds the command open until the database setup job has finished and the pod is ready.

    helm install -n datakitchen dk-tg-app datakitchen-testgen/testgen \
        --values values-tg-app.yaml --wait --timeout 15m
    

    Warning

    If the install fails after the setup job has started, do not re-run helm install against the same database. The setup job does not run twice against a populated schema and will stop partway through. Run helm uninstall dk-tg-app, then drop the database before starting again.

    DROP DATABASE IF EXISTS datakitchen;
    
  5. Confirm that the pod is running.

    kubectl get pods -n datakitchen
    
  6. Find the address of the load balancer, and create DNS records pointing the UI and API domains at it. The address can take a few minutes to appear.

    kubectl get service -n app-routing-system nginx \
        -o jsonpath="{.status.loadBalancer.ingress[0].ip}"
    
    kubectl get ingress dk-tg-app-testgen -n datakitchen \
        -o jsonpath="{.status.loadBalancer.ingress[0].hostname}"
    
  7. Open https://<UI domain> in a browser and sign in with the credentials from testgen.uiUser and testgen.uiPassword.

    Until the DNS records resolve, you can reach the UI at http://localhost:8501 by forwarding the port.

    kubectl port-forward svc/dk-tg-app-testgen 8501:8501 -n datakitchen
    

Tip

To configure email notifications or set up single sign-on, see Administer TestGen.

To upgrade an instance of TestGen Enterprise to the latest released version, see Upgrade TestGen.

Values reference

Value Description
image.repository, image.tag The enterprise image and version. Set tag to an exact version rather than a major version such as v5, so that an upgrade pulls the intended image. For a list of recent releases, see TestGen Release Notes.
imagePullSecrets The registry secret granting access to the private enterprise image.
testgen.databaseHost, testgen.databasePort Endpoint of the PostgreSQL server.
testgen.databaseName Database that TestGen creates and uses. Defaults to datakitchen.
testgen.databaseSchema Schema that TestGen creates and uses. Defaults to tgapp.
testgen.databaseUser The user created in Prepare the application database.
testgen.databasePasswordSecret Secret and key holding that user's password.
testgen.authSecrets Secret holding decrypt-salt, decrypt-password, and jwt-hashing-key. Set create: false so the chart uses the secret you created instead of generating new values.
testgen.uiUser, testgen.uiPassword Username and password for the initial system administrator account, read only when the database is first prepared. After installing, sign in with these credentials to create accounts for your team. This account has no special status — once other system administrators exist, any of them can remove it if it is no longer needed.
testgen.instanceId A name that identifies this TestGen instance.
testgen.uiBaseUrl The externally reachable URL of the UI.
testgen.baseUrl The externally reachable URL of the API and MCP server.
testgen.emailNotifications Sender address and SMTP server for email notifications. See Configure Email Server.
ingress Ingress class, annotations, hosts, and TLS. Disabled by default.
extraEnv Any additional environment variable, in Kubernetes environment format. See Set Environment Variables.
extraVolumes, extraVolumeMounts Additional volumes, for mounting configuration files such as sso_auth.toml.
resources CPU and memory requests and limits for the application pod.
replicaCount Number of application pods. Leave at 1; browser sessions are held in the pod serving them.

For the complete set, see the values.yaml file in the chart.

Encrypt the connection to the application database

To require TLS on the connection to the application database, set the PostgreSQL client variables PGSSLMODE and PGSSLROOTCERT through extraEnv. verify-full is the strongest setting: it fails unless the connection is encrypted, the certificate is signed by a trusted authority, and the certificate's host name matches the server.

extraEnv:
  - name: PGSSLMODE
    value: verify-full
  - name: PGSSLROOTCERT
    value: system

system is the image's trust store, which covers servers presenting a certificate from a public authority, such as Azure Database for PostgreSQL. For a server with a private authority, such as Amazon RDS, store the authority's PEM bundle in a secret, mount it, and point PGSSLROOTCERT at the mounted path.

kubectl create secret generic testgen-db-ca --from-file=ca.pem=./rds-ca.pem
extraEnv:
  - name: PGSSLMODE
    value: verify-full
  - name: PGSSLROOTCERT
    value: /dk/certs/ca.pem

extraVolumes:
  - name: db-ca
    secret:
      secretName: testgen-db-ca

extraVolumeMounts:
  - name: db-ca
    mountPath: /dk/certs
    readOnly: true