Kubernetes on CloudFleet and Hetzner

This guide adapts the standard Kubernetes deployment to a CloudFleet Kubernetes cluster in Helsinki, with Hetzner ARM64 compute and Hetzner Object Storage. The storage bucket holds published data and completed recovery backups. Hydrated files on node-local NVMe are caches; unpublished hot rows have a separate durability boundary. See Reliability and completion.

1. Prepare compute and object storage

Create a Hetzner Object Storage bucket in hel1 and credentials scoped to that bucket. Use the endpoint shown by Hetzner, for example:

https://hel1.your-objectstorage.com

Create a CloudFleet node pool in the same region with at least three nodes. Choose instances matching the release architecture and size CPU, memory and local storage for your ingestion and query workload. Label the pool so the chart can select it, for example:

boilstream.com/nodepool=arm64-hel

Spread the three BoilStream replicas across nodes when capacity permits. The example uses preferred anti-affinity so temporary capacity pressure does not deadlock scheduling.

2. Install cluster prerequisites

Install cert-manager and Envoy Gateway before BoilStream:

helm install cert-manager jetstack/cert-manager \
  --version v1.16.2 \
  --namespace cert-manager --create-namespace \
  --set crds.enabled=true

helm install eg oci://docker.io/envoyproxy/gateway-helm \
  --version v1.2.1 \
  --namespace envoy-gateway-system --create-namespace

Create the ClusterIssuer named by your values. Because the public listeners use TLS passthrough, use a DNS-01 issuer for automated public certificates. The internal cluster certificate must use a private trust root and is never shared with external clients.

3. Create Secrets

Keep credentials and keys out of Helm values:

kubectl create namespace boilstream

kubectl -n boilstream create secret generic boilstream-s3-creds \
  --from-literal=access_key='<HETZNER_S3_ACCESS_KEY>' \
  --from-literal=secret_key='<HETZNER_S3_SECRET_KEY>'

kubectl -n boilstream create secret generic boilstream-superadmin \
  --from-literal=password='<STRONG_PASSWORD>'

kubectl -n boilstream create secret generic boilstream-encryption \
  --from-literal=encryption_key="$(openssl rand -hex 32)" \
  --from-literal=token_signing_key="$(openssl rand -base64 32)" \
  --from-literal=cluster_token="$(openssl rand -base64 32)"

Back up both generated keys in an external secret manager. Every replica and restore incarnation needs the same values. If you automate superadmin login, also pre-create the optional MFA Secret described in the chart values.

4. Create the values overlay

Copy charts/boilstream/values-hetzner-example.yaml and replace the example values:

image:
  repository: <release-registry>/boilstream
  tag: <release-version>
  # Replace this non-resolving example with the published ARM64 release digest.
  digest: sha256:0000000000000000000000000000000000000000000000000000000000000000
  pullPolicy: IfNotPresent

configRenderer:
  image:
    # Replace with the published Python multi-architecture manifest digest.
    digest: sha256:0000000000000000000000000000000000000000000000000000000000000000

domain: data.your-company.example

s3:
  endpoint: https://hel1.your-objectstorage.com
  bucket: boilstream-state
  region: hel1
  prefix: production/
  forcePathStyle: true
  existingCredsSecret: boilstream-s3-creds
  accessKey: ""
  secretKey: ""

encryptionKey:
  existingSecret: boilstream-encryption

superadmin:
  existingSecret: boilstream-superadmin

cpMetadata:
  enabled: true
  ducklakeMultiRaft: true
  usersMultiRaft: true
  tokenSecretKey: cluster_token
  s3Dr:
    enabled: true
    clusterId: company-prod-hel1
    backend: primary-s3
    prefix: quack-dr
    durabilityMode: strict
    bootstrapMode: new
    recoveryEpoch: 1
    recoveryId: company-prod-hel1-first-install

clusterTls:
  enabled: true
  issuer:
    name: boilstream-ca-issuer
    kind: ClusterIssuer

tls:
  issuer:
    create: false
    name: letsencrypt-prod
    kind: ClusterIssuer

httpIngestion:
  enabled: true
  port: 8446
  allowedOrigins:
    - https://data.your-company.example

nodeSelector:
  boilstream.com/nodepool: arm64-hel

persistence:
  enabled: false

Hetzner Object Storage uses path-style addressing in the tested chart profile. Keep a deployment-specific prefix and never share its cluster identity with another installation. After the first completed recovery backup, change bootstrapMode to restore. The release digest selects the exact ARM64 image bytes; the tag is not used for pulling while image.digest is set. Pin the config-renderer image by digest as well.

5. Install and configure DNS

helm lint ./charts/boilstream \
  --values ./charts/boilstream/values-hetzner-example.yaml \
  --values my-values.yaml

helm install boilstream ./charts/boilstream \
  --namespace boilstream \
  --values ./charts/boilstream/values-hetzner-example.yaml \
  --values my-values.yaml \
  --set-string image.digest='sha256:<release-image-digest>' \
  --set-string configRenderer.image.digest='sha256:<python-image-digest>'

kubectl -n boilstream rollout status statefulset/boilstream
kubectl -n boilstream get gateway boilstream \
  -o jsonpath='{.status.addresses[0].value}'

Point both the base domain and its wildcard at the Gateway address:

Record	Type	Target
data.your-company.example	A/AAAA	Gateway address
*.data.your-company.example	A/AAAA	Gateway address

Wait for the public certificate to become ready before testing clients.

6. Verify before use

kubectl -n boilstream get pods,gateway,certificate
boilstream-admin auth login \
  --server https://data.your-company.example \
  --email admin@example.com
boilstream-admin cluster status

Use the credentials and endpoints vended by BoilStream. Verify managed DuckDB, PGWire, Kafka, HTTP/2, FlightRPC, and FlightSQL independently. Then delete the current leader pod, confirm clients can reconnect and write, and complete a rolling restart.

Before production, run the all-node-local-state-loss drill in staging: fence the previous incarnation, increment the recovery epoch once, assign a new recovery ID, restore solely from the same S3 prefix and encryption key, and verify catalog access plus writes after a leader failover. A PVC must not be part of the recovery procedure.

Operations

• Scale infrastructure before increasing replicas; retain a quorum throughout changes.
• Upgrade one replica at a time and wait for readiness.
• Monitor object-storage recovery publication as well as pod health.
• Treat local NVMe and optional PVCs as rebuildable caches.
• Retain the previous release artifacts, values, keys, and recovery identity for rollback.