Skip to content
BoilStream

Choose an interface

directory
TL;DR

Choose a client by the operation you need: PGWire for SQL and recent reads, HTTP Arrow or Kafka for producers, native DuckDB for remote catalog queries, and SSE for preview live output. Payloads, credentials and retry rules differ. Check the compatibility matrix before substituting one interface for another.

ComparisonChoose the interface for the operation
Choose the interface for the operationIngestion, SQL queries and live consumption use different interfaces and credentials. This comparison groups them by the job the client needs to perform.Client jobInterfaceSend rowsContinuous inputHTTP Arrow or Kafka; Flighthas limitsRun SQLCreate, insert or queryPGWire; managed DuckDB;Flight SQL previewFollow eventsLive application updatesSSE with a stream-purposetokenChoose the interface for the operationIngestion, SQL queries and live consumption use different interfaces and credentials. This comparison groups them by the job the client needs to perform.Client jobInterfaceSend rowsContinuousinputHTTP Arrowor Kafka;Flight haslimitsRun SQLCreate,insert orqueryPGWire;managedDuckDB;Flight SQLpreviewFolloweventsLiveapplicationupdatesSSE with astream-purposetoken
Start with the operation, then check authentication, completion semantics and interface support.

BoilStream exposes separate interfaces for producing rows, running SQL, and consuming stream output. Choose by operation and client ecosystem; do not assume every protocol has the same SQL or failover behavior.

The interface compatibility matrix is the authoritative summary of available operations, preview features and known limits.

Ingest rows

Section titled “Ingest rows”
Interface Payload and clients Typical use
FlightRPC Arrow Flight record batches from DuckDB, PyArrow, or custom clients Arrow-native service and bulk ingestion
HTTP/2 Arrow Arrow IPC over TLS from browsers or HTTP clients Web and service event ingestion
Kafka protocol RecordBatch v2 with Confluent Avro payloads Existing Kafka producer applications
PGWire / managed DuckDB Streaming SQL INSERT SQL applications; COPY input is unavailable

These interfaces write to tables provisioned in a streaming DuckLake whose display name ends in __stream.

Run queries

Section titled “Run queries”
Interface Clients Primary purpose
PostgreSQL wire protocol psql, DBeaver, and compatible BI tools Server-side SQL, DDL, and recent streaming reads
FlightSQL Arrow FlightSQL clients Arrow-native SQL and metadata discovery
Managed DuckDB Native DuckDB with the public BoilStream extension Attach a catalog by UUID for authorized DuckLake access

PGWire combines committed recent streaming rows with published history. Managed DuckDB runs SQL remotely; it and FlightSQL do not currently promise the same visibility of unpublished recent rows. Browser DuckDB-WASM cannot attach managed catalogs directly; browser applications use server APIs or an application backend.

Consume stream output

Section titled “Consume stream output”

Server-Sent Events is a preview interface for Arrow IPC batches. Applications must handle interruptions, repeated batches and missing data. Kafka consumption and Flight live cursors are also preview interfaces. Kafka replay is limited to retained recent data; use SQL for published analytical history.

Authentication boundaries

Section titled “Authentication boundaries”

Use the authentication method in each API guide and trusted TLS connections. Managed DuckDB clients authenticate through the extension and do not need catalog object-storage keys. Catalog access depends on the requesting user’s permissions, not a chosen database name or SQL alias. See Multi-tenancy and catalog sharing.

Capacity and retry behavior

Section titled “Capacity and retry behavior”

Capacity depends on payload shape, batch size, TLS and durability settings, hardware, object storage, and concurrent query load. Benchmark the exact interface and settings you plan to deploy.

An interrupted connection does not imply that every operation is safe to replay. Follow the acknowledgement, idempotency, and retry guidance for the selected API.

Next steps

Section titled “Next steps”