Architecture
BoilStream brings streaming ingestion, DuckDB SQL, DuckLake storage and optional full-text search into one service. Use it to query recent events, analyze stored history and build continuous transformations.
Data flow
This diagram shows the available workflows. Their results become available independently: a producer response does not mean that a view, search index or subscriber has caught up. See Reliability and completion.
Recent data and stored history
Recent streaming data can be queried before its Parquet files are published to object storage. PGWire combines available recent data with published history for streaming tables. Do not assume the same recent-data visibility through every query interface.
Published history uses DuckLake tables and Parquet files. Keep both the data and its catalog available for queries. Hydration preloads existing files to improve repeated reads; it does not make newly accepted input durable.
Query paths
| Client | Use it for |
|---|---|
| PostgreSQL wire protocol (PGWire) | DuckDB SQL from SQL clients, including recent streaming data |
| Native DuckDB with the BoilStream extension | Authenticate, attach a catalog by UUID and run SQL remotely |
| FlightSQL | Preview Arrow query results and metadata discovery |
Use the interface matrix to choose a client. Managed DuckDB and FlightSQL do not currently promise the same recent-data visibility as PGWire. Direct managed attachment from DuckDB-WASM is unavailable.
Access control
Users access catalogs through tenant membership or explicit grants. A catalog UUID identifies the catalog; a display name or local SQL alias does not grant access. Sharing gives another user access to the existing catalog without copying its data.
Managed clients use short-lived credentials and TLS. They do not need your object-storage credentials. See Authentication and Catalog sharing.
Deployment
Run one server for development or use the cluster deployment for production planning. Availability depends on healthy replicas, object storage and the configured recovery settings. Recent data that has not reached durable storage can be lost on failure. See Reliability and completion before choosing retention and retry policies.