Ingestion and query interfaces
BoilStream exposes separate interfaces for producing rows, running SQL, and consuming stream output. Choose by operation and client ecosystem; do not assume every protocol has the same SQL or failover behavior.
The interface compatibility matrix is the authoritative summary of available operations, preview features and known limits.
Ingest rows
| Interface | Payload and clients | Typical use |
|---|---|---|
| FlightRPC | Arrow Flight record batches from DuckDB, PyArrow, or custom clients | Arrow-native service and bulk ingestion |
| HTTP/2 Arrow | Arrow IPC over TLS from browsers or HTTP clients | Web and service event ingestion |
| Kafka protocol | RecordBatch v2 with Confluent Avro payloads | Existing Kafka producer applications |
| PGWire / managed DuckDB | Streaming SQL INSERT | SQL applications; COPY input is unavailable |
These interfaces write to tables provisioned in a streaming DuckLake whose display name ends in __stream.
Run queries
| Interface | Clients | Primary purpose |
|---|---|---|
| PostgreSQL wire protocol | psql, DBeaver, and compatible BI tools | Server-side SQL, DDL, and recent streaming reads |
| FlightSQL | Arrow FlightSQL clients | Arrow-native SQL and metadata discovery |
| Managed DuckDB | Native DuckDB with the public BoilStream extension | Attach a catalog by UUID for authorized DuckLake access |
PGWire combines committed recent streaming rows with published history. Managed DuckDB runs SQL remotely; it and FlightSQL do not currently promise the same visibility of unpublished recent rows. Browser DuckDB-WASM cannot attach managed catalogs directly; browser applications use server APIs or an application backend.
Consume stream output
Server-Sent Events is a preview interface for Arrow IPC batches. Applications must handle interruptions, repeated batches and missing data. Kafka consumption and Flight live cursors are also preview interfaces. Kafka replay is limited to retained recent data; use SQL for published analytical history.
Authentication boundaries
Use the authentication method in each API guide and trusted TLS connections. Managed DuckDB clients authenticate through the extension and do not need catalog object-storage keys. Catalog access depends on the requesting user's permissions, not a chosen database name or SQL alias. See Multi-tenancy and catalog sharing.
Capacity and retry behavior
Capacity depends on payload shape, batch size, TLS and durability settings, hardware, object storage, and concurrent query load. Benchmark the exact interface and settings you plan to deploy.
An interrupted connection does not imply that every operation is safe to replay. Follow the acknowledgement, idempotency, and retry guidance for the selected API.
Next steps
- API reference — protocol-specific requests and client examples
- Getting started — first ingestion and query workflow
- DuckLake — streaming catalogs and managed DuckDB attachment
- Architecture — durability and cluster behavior