DuckLake integration

BoilStream manages DuckLake catalogs, their access permissions and streaming-table provisioning. Native DuckDB clients connect through the BoilStream extension.

Catalog types

Display name	Purpose
Ends in __stream, such as events__stream	Streaming tables with ingestion topics and metadata columns
No streaming suffix, such as analytics	Warehouse tables with ordinary DuckLake SQL

The suffix needs a base name; __stream alone is reserved. When default provisioning is enabled, a new tenant receives duckie and boil__stream. Each tenant's catalogs have different UUIDs even when their display names match.

Native DuckDB clients

Use DuckDB 1.5 or newer with the BoilStream extension. Its build must match your DuckDB version, platform and server release. Login and catalog discovery do not attach a catalog automatically.

INSTALL boilstream FROM community;
LOAD boilstream;

PRAGMA boilstream_bootstrap_session('https://your-server.example/secrets:<token>');
FROM boilstream_ducklakes();

-- Replace <catalog-uuid> with the selected catalog_id from the result.
PRAGMA boilstream_attach_ducklake('<catalog-uuid>', 'my_catalog');
USE my_catalog;
SELECT * FROM main.events LIMIT 10;

The alias my_catalog belongs to the client session. Use the catalog UUID returned by BoilStream. Queries run remotely, and the extension handles short-lived authentication. Clients verify the server's TLS certificate and do not need catalog object-storage credentials. Direct managed attachment from DuckDB-WASM is unavailable.

For interactive login instead of a bootstrap token:

PRAGMA boilstream_login('https://your-server.example/user@example.com',
    'password', '123456');

Use the actual MFA code. Keep credentials out of saved scripts, logs and screenshots.

Create a streaming catalog

PRAGMA boilstream_create_ducklake('events__stream', 'Application events');
FROM boilstream_ducklakes();

-- Use the UUID of the catalog just created, not its display name.
PRAGMA boilstream_attach_ducklake('<catalog-uuid>', 'events');
USE events;

CREATE TABLE main.activity (
    event_id BIGINT,
    region VARCHAR,
    event_time TIMESTAMP
);

Creating the table provisions its ingestion topic. Send rows through streaming SQL INSERT, Kafka, HTTP/2 Arrow or FlightRPC. Ordinary warehouse tables use standard SQL operations rather than streaming ingestion. See Ingestion interfaces.

Partitioning

Streaming tables receive time partitions based on the ingestion metadata timestamp. Add custom partitions without repeating the automatic year/month/day/hour columns:

ALTER TABLE events.main.activity SET PARTITIONED BY (region);

Use SQL to query and manage partitions. Object names and directory layouts are managed by BoilStream; do not use them as application identifiers or edit files directly.

Hot/cold tiered storage

Recent streaming rows may be queryable before their Parquet files are published to object storage. These have different durability and visibility boundaries.

Visibility paths

Client	Visibility
PGWire	Combines committed recent streaming rows with published history
Managed DuckDB	Remote SQL; does not promise the same unpublished recent-data visibility as PGWire
FlightSQL	Preview Arrow queries; does not promise the same unpublished recent-data visibility as PGWire

An admission response or completed upload does not mean that all rows are available in table queries. See Reliability and completion before choosing when to delete source data or retry a request.

Configuration

Local commit and cold finalization intervals are separate settings. This is an excerpt; retain the other required fields in your generated configuration:

duckdb_persistence:
  commit_interval_ms: 1000

processing:
  snapshot_interval_minutes: 10

Intervals schedule work; they are not end-to-end latency guarantees. See the generated configuration reference.

Cold-tier hydration preloads existing object files for repeated reads. It does not turn newly accepted input into durable data.

Catalog functions

Form	Purpose
PRAGMA boilstream_bootstrap_session(url)	Authenticate with a bootstrap token
PRAGMA boilstream_register_user(url, password)	Begin account registration
PRAGMA boilstream_verify_mfa(totp_code)	Complete MFA registration
PRAGMA boilstream_login(url, password, mfa)	Interactive login
FROM boilstream_ducklakes()	List authorized catalogs and their UUIDs
FROM boilstream_secrets()	List cached non-catalog secrets
PRAGMA boilstream_create_ducklake(name, description)	Create a catalog; it still needs explicit attachment
PRAGMA boilstream_attach_ducklake(id, alias)	Attach by UUID under a local alias
PRAGMA boilstream_help	Show extension help

Sharing grants access to the owner's catalog without copying its data. See Multi-tenancy and catalog sharing for access modes and revocation.