Skip to content

Deploy on Hetzner with CloudFleet

operations
TL;DR

Use the Kubernetes deployment with a provider overlay for CloudFleet compute and Hetzner Object Storage in Helsinki. Match compute architecture to the release, configure the storage endpoint and scope secrets to the deployment. Follow the same quorum, TLS, key-preservation and recovery rules as the base installation.

ArchitectureReplicas coordinate metadata and share published object storage
Replicas coordinate metadata and share published object storageClients connect to a server replica. The replicas participate in metadata consensus, while published files live in shared object storage. Recovery also depends on local state, completed backups and original encryption keys.Cluster: separate server processesSQL clientSupported entry pointReplica ALocal state and serverReplica BLocal state and serverReplica CLocal state and serverObject storagePublished files andbackupsConnectMetadata consensusConsensusConsensusPublish / readReplicas coordinate metadata and share published object storageClients connect to a server replica. The replicas participate in metadata consensus, while published files live in shared object storage. Recovery also depends on local state, completed backups and original encryption keys.Cluster: separate server processesSQL clientSupported entry pointReplica ALocal state andserverReplica BLocal state andserverReplica CLocal state andserverObject storagePublished files andbackupsConnectMetadata consensusConsensusConsensusPublish / read
Each replica uses shared storage; one storage path is shown. Metadata quorum alone does not prove that every acknowledged recent row is recoverable.

This guide adapts the standard Kubernetes deployment to a CloudFleet Kubernetes cluster in Helsinki, with Hetzner ARM64 compute and Hetzner Object Storage. The storage bucket holds published data and completed recovery backups. Hydrated files on node-local NVMe are caches; unpublished hot rows have a separate durability boundary. See Reliability and completion.

Create a Hetzner Object Storage bucket in hel1 and credentials scoped to that bucket. Use the endpoint shown by Hetzner, for example:

Output
https://hel1.your-objectstorage.com

Create a CloudFleet node pool in the same region with at least three nodes. Choose instances matching the release architecture and size CPU, memory and local storage for your ingestion and query workload. Label the pool so the chart can select it, for example:

Output
boilstream.com/nodepool=arm64-hel

Spread the three BoilStream replicas across nodes when capacity permits. The example uses preferred anti-affinity so temporary capacity pressure does not deadlock scheduling.

Install cert-manager and Envoy Gateway before BoilStream:

Shell
helm install cert-manager jetstack/cert-manager \
--version v1.16.2 \
--namespace cert-manager --create-namespace \
--set crds.enabled=true
helm install eg oci://docker.io/envoyproxy/gateway-helm \
--version v1.2.1 \
--namespace envoy-gateway-system --create-namespace

Create the ClusterIssuer named by your values. Because the public listeners use TLS passthrough, use a DNS-01 issuer for automated public certificates. The internal cluster certificate must use a private trust root and is never shared with external clients.

Keep credentials and keys out of Helm values:

Shell
kubectl create namespace boilstream
kubectl -n boilstream create secret generic boilstream-s3-creds \
--from-literal=access_key='<HETZNER_S3_ACCESS_KEY>' \
--from-literal=secret_key='<HETZNER_S3_SECRET_KEY>'
kubectl -n boilstream create secret generic boilstream-superadmin \
--from-literal=password='<STRONG_PASSWORD>'
kubectl -n boilstream create secret generic boilstream-encryption \
--from-literal=encryption_key="$(openssl rand -hex 32)" \
--from-literal=token_signing_key="$(openssl rand -base64 32)" \
--from-literal=cluster_token="$(openssl rand -base64 32)"

Back up both generated keys in an external secret manager. Every replica and restore incarnation needs the same values. If you automate superadmin login, also pre-create the optional MFA Secret described in the chart values.

Copy charts/boilstream/values-hetzner-example.yaml and replace the example values:

YAML
image:
repository: <release-registry>/boilstream
tag: <release-version>
# Replace this non-resolving example with the published ARM64 release digest.
digest: sha256:0000000000000000000000000000000000000000000000000000000000000000
pullPolicy: IfNotPresent
configRenderer:
image:
# Replace with the published Python multi-architecture manifest digest.
digest: sha256:0000000000000000000000000000000000000000000000000000000000000000
domain: data.your-company.example
s3:
endpoint: https://hel1.your-objectstorage.com
bucket: boilstream-state
region: hel1
prefix: production/
forcePathStyle: true
existingCredsSecret: boilstream-s3-creds
accessKey: ""
secretKey: ""
encryptionKey:
existingSecret: boilstream-encryption
superadmin:
existingSecret: boilstream-superadmin
# Explicit operator acceptance: use only with a continuous-clock rate of 0.9–1.1.
# The chart default stays untrusted and rejects clustered startup.
quack:
sharedSupport:
profile: clock-profile-1
cpMetadata:
enabled: true
ducklakeMultiRaft: true
usersMultiRaft: true
tokenSecretKey: cluster_token
s3Dr:
enabled: true
clusterId: company-prod-hel1
backend: primary-s3
prefix: quack-dr
durabilityMode: strict
bootstrapMode: new
recoveryEpoch: 1
recoveryId: company-prod-hel1-first-install
clusterTls:
enabled: true
issuer:
name: boilstream-ca-issuer
kind: ClusterIssuer
tls:
issuer:
create: false
name: letsencrypt-prod
kind: ClusterIssuer
httpIngestion:
enabled: true
port: 8446
allowedOrigins:
- https://data.your-company.example
nodeSelector:
boilstream.com/nodepool: arm64-hel
persistence:
enabled: false

Hetzner Object Storage uses path-style addressing in the tested chart profile. Keep a deployment-specific prefix and never share its cluster identity with another installation. After the first completed recovery backup, change bootstrapMode to restore. The release digest selects the exact ARM64 image bytes; the tag is not used for pulling while image.digest is set. Pin the config-renderer image by digest as well.

Shell
helm lint ./charts/boilstream \
--values ./charts/boilstream/values-hetzner-example.yaml \
--values my-values.yaml
helm install boilstream ./charts/boilstream \
--namespace boilstream \
--values ./charts/boilstream/values-hetzner-example.yaml \
--values my-values.yaml \
--set-string image.digest='sha256:<release-image-digest>' \
--set-string configRenderer.image.digest='sha256:<python-image-digest>'
kubectl -n boilstream rollout status statefulset/boilstream
kubectl -n boilstream get gateway boilstream \
-o jsonpath='{.status.addresses[0].value}'

Point both the base domain and its wildcard at the Gateway address:

Record Type Target
data.your-company.example A/AAAA Gateway address
*.data.your-company.example A/AAAA Gateway address

Wait for the public certificate to become ready before testing clients.

Shell
kubectl -n boilstream get pods,gateway,certificate
boilstream-admin auth login \
--server https://data.your-company.example \
--email admin@example.com
boilstream-admin cluster status

Use the credentials and endpoints supplied by BoilStream. Verify managed DuckDB, PGWire, Kafka, HTTP/2, FlightRPC, and FlightSQL independently. Then delete the current leader pod, confirm clients can reconnect and write, and complete a rolling restart.

Before production, run the all-node-local-state-loss drill in staging: fence the previous incarnation, increment the recovery epoch once, assign a new recovery ID, restore solely from the same S3 prefix and encryption key, and verify catalog access plus writes after a leader failover. A PVC must not be part of the recovery procedure.

  • Scale infrastructure before increasing replicas; retain a quorum throughout changes.
  • Upgrade one replica at a time and wait for readiness.
  • Monitor object-storage recovery publication as well as pod health.
  • Rebuild hydrated historical files from object storage; do not treat unpublished recent rows as disposable cache.
  • Retain the previous release artifacts, values, keys, and recovery identity for rollback.