Introducing BoilStream

Originally published in January 2025. Examples and interface descriptions updated in October 2026; use the current release documentation for deployment compatibility.

BoilStream brings ingestion, DuckDB SQL, DuckLake storage and optional Tantivy search into one service. It is useful when the same incoming events need recent-data queries and analytical history in object storage.

It is not a full replacement for every Kafka or stream-processing workflow. Keep a broker when you require durable broker replay, transactions or its connector ecosystem.

Start with a streaming catalog

Follow the Quick Start to authenticate, create a catalog ending in __stream, list its UUID and attach it under a local alias. Creating or listing a catalog does not attach it automatically.

In that attached catalog:

SQL

CREATE TABLE main.user_events (
    event_id BIGINT,
    user_id VARCHAR,
    event_type VARCHAR,
    event_time TIMESTAMP
);

The table becomes an ingestion topic. You can send rows through streaming SQL INSERT, Kafka with Confluent Avro, HTTP/2 Arrow or FlightRPC. PGWire COPY FROM STDIN is unavailable.

Transform incoming rows

A streaming view filters or transforms future input into a derived stream:

SQL

CREATE STREAMING VIEW login_events AS
  SELECT event_id, user_id, event_time
  FROM user_events
  WHERE event_type = 'login';

A normal CREATE TABLE AS SELECT is not the streaming-view API. For reusable logic that runs only when queried, use ordinary CREATE VIEW.

Aggregate windows

A materialized view runs a query when its window closes:

SQL

CREATE MATERIALIZED VIEW events_per_minute AS
  SELECT COUNT(*) AS events
  FROM user_events
  WITH (
    window_type = 'tumbling',
    window_size = '1 minute',
    timestamp_column = 'event_time'
  );

Materialized results become available after their windows are processed. A successful source write does not establish exactly-once derived output. See Views and materialized views.

Query the right data boundary

PGWire combines committed recent rows with published history. Managed DuckDB runs SQL remotely; it and FlightSQL do not currently promise the same visibility of unpublished recent rows. Choose the producer and query client using the interface matrix.

Add search where needed

Enable Tantivy for the tables and text fields that need indexing. Other tables are not indexed by default. Search results can become available at a different time from SQL results. See Full-text search for configuration and SQL.

Evaluate a real workload

Begin with a small dataset and check exact results. Then measure the schema, batch sizes, queries, storage backend and failure cases that matter to your application. The examples provide small datasets and expected query results; Reliability and completion explains write and recovery limits.