Quick Start

Get up and running with BoilStream in under 5 minutes.

Choose Your Version

Free Version: Basic streaming without authentication or TLS
Pro Version: Enterprise SSO, TLS encryption, and advanced features

Prerequisites

BoilStream

S3-compatible storage (AWS S3, MinIO, etc.)
Linux or macOS
arm64 or x86

DuckDB Client

Pro Version Additional Requirements

TLS certificates (self-signed or CA-issued)
JWT identity provider (AWS Cognito, Azure AD, Google Cloud, Auth0, or Okta)

Installation

Download Binary

Download the latest release for your platform:

Linux x86_64Linux arm64macOS IntelmacOS Apple Silicon

bash

curl -L -o boilstream https://www.boilstream.com/binaries/linux-x86/boilstream-0.2.2
chmod +x boilstream

bash

curl -L -o boilstream https://www.boilstream.com/binaries/linux-aarch64/boilstream-0.2.2
chmod +x boilstream

bash

curl -L -o boilstream https://www.boilstream.com/binaries/darwin-x86/boilstream-0.2.2
chmod +x boilstream

bash

curl -L -o boilstream https://www.boilstream.com/binaries/linux-aarch64/boilstream-0.2.2
chmod +x boilstream

Free Version Setup

Start BoilStream

bash

boilstream --config local-dev.yaml

Connect from DuckDB (Free Version)

Open DuckDB and connect to BoilStream:

NOTE: You can build the Airport extension your self, if it is not yet available from the DuckDB community extensions repository. This forked repository added the CREATE VIEW support for creating BoilStream materialised views for derived topics (forked streams onto S3).

sql

-- Install and load the airport extension (if not already done)
INSTALL airport FROM community;
LOAD airport;

-- Connect to BoilStream (no authentication)
ATTACH 'boilstream' (TYPE AIRPORT, location 'grpc://localhost:50051/');

-- List topics
SHOW ALL TABLES;

Pro Version Setup

Start BoilStream Pro

bash

# Set up authentication (example with AWS Cognito)
export AUTH_PROVIDERS="cognito"
export COGNITO_USER_POOL_ID="eu-west-1_gti5vAfvC"
export COGNITO_REGION="eu-west-1"
export COGNITO_AUDIENCE="7ml47mngu8lrcb198epbkkfi5s"

# Optional: Set up authorization rules
export ADMIN_GROUPS="admin"
export READ_ONLY_GROUPS="readonly"
export WRITE_GROUPS="writer"

# Start with pro features enabled
./boilstream

Connect from DuckDB (Pro Version)

NOTE: You can build the Airport extension your self, if it is not yet available from the DuckDB community extensions repository. This forked repository added the CREATE VIEW support for creating BoilStream materialised views for derived topics (forked streams onto S3).

The pro version requires TLS and authentication:

sql

-- Install and load the airport extension
INSTALL airport FROM community;
LOAD airport;

-- Create authentication secret with your JWT token
CREATE SECRET boilstream_auth (
    type airport,
    auth_token 'eyJraWQiOiJ6YWZsU0RYQnorRWFyQUgyc1Nwa2pBZE5ja0JoZjVwQUtPTnNjZzlpSW04PSIsImFsZyI6IlJTMjU2In0.eyJzdWIiOiIyMmU1MTQ5NC05MGYxLTcwNGUtYTRkNS0yNTQwNWM5Njk3ODQiLCJjb2duaXRvOmdyb3VwcyI6WyJhZG1pbiJdLCJpc3MiOiJodHRwczpcL1wvY29nbml0by1pZHAuZXUtd2VzdC0xLmFtYXpvbmF3cy5jb21cL2V1LXdlc3QtMV9ndGk1dkFmdkMiLCJjbGllbnRfaWQiOiI3bWw0N21uZ3U4bHJjYjE5OGVwYmtrZmk1cyIsIm9yaWdpbl9qdGkiOiIzMGM2MWZiYy05ZWNlLTQ5YzEtYjdhYy0wOGM3ZTM2YmNiYjQiLCJldmVudF9pZCI6IjZlMDc5NmQ4LWU1ZDktNGNkNS04MDQ3LTYyMTdmNzczMWViZiIsInRva2VuX3VzZSI6ImFjY2VzcyIsInNjb3BlIjoiYXdzLmNvZ25pdG8uc2lnbmluLnVzZXIuYWRtaW4iLCJhdXRoX3RpbWUiOjE3NDk4MTcwMTIsImV4cCI6MTc0OTgyMDYxMiwiaWF0IjoxNzQ5ODE3MDEyLCJqdGkiOiJlMzMwYzQyZC0xZmQ3LTQ2ZWMtYTMyNy0yNGNmMWE3MThlZGQiLCJ1c2VybmFtZSI6InRlc3R1c2VyIn0.QUyp--JBCcmqk787oRoeJYP9b35kmInEdfqOpj_lCh7-oqr7lzMrt_xCxhicxGwElwkoUxEzvlRVHNegwwFIwJXepM8TuMNMbQV0NPZxUnM5r8pGeDWjgqHQKrJMnTPUXJZOoIUtJuQUDqlZHRoCzZNaPgj54qKSAQpHl8XXsghGPtzfxMpIvSfe19ojRunI77O0CYm_MD9snu3bU1FyoteRMkpDReL4ZC7b_mSPM6Bw3Pa0QdUnL1lyEIWUCjm2cS13ToMR3A86qo-lf8IazG5FqnYqvg2CzSJBe9fJEGRl7g2bDzsAqH67ImIS9of1vnYHDYWFAZhp7wPPUuk1fQ',
    scope 'grpc+tls://localhost:50051/'
);

-- Connect to BoilStream with TLS and authentication
ATTACH 'boilstream' (TYPE AIRPORT, location 'grpc+tls://localhost:50051/');

-- List topics (requires read permissions)
SHOW ALL TABLES;

TLS Certificate Setup

For the pro version with TLS, you need to set up certificates. For development, you can generate self-signed certificates:

bash

# Generate self-signed certificate (development only)
mkdir -p certs
openssl req -x509 -newkey rsa:4096 -keyout certs/server.key -out certs/server.crt -days 365 -nodes -subj "/CN=localhost"

# Set environment variable for DuckDB client
export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH=certs/server.crt

Getting JWT Tokens

To get JWT tokens for authentication, see our provider-specific guides:

Connect via PostgreSQL Protocol

BoilStream includes a built-in PostgreSQL wire protocol server that allows you to connect with any PostgreSQL-compatible client, including BI tools like DBeaver, Tableau, and command-line tools like psql.

Quick PostgreSQL Connection

bash

# Connect with psql
psql -h localhost -p 5432 -U boilstream -d boilstream

# Connection string format
psql "postgresql://boilstream:boilstream@localhost:5432/boilstream"

DBeaver Connection

Create a new PostgreSQL connection
Host: localhost, Port: 5432
Database: boilstream
Username: boilstream, Password: boilstream

Query Your Streaming Data

sql

-- List all available topics
SELECT table_name FROM information_schema.tables 
WHERE table_schema = 'public';

-- Query recent data
SELECT * FROM your_topic_name 
ORDER BY event_time DESC 
LIMIT 100;

-- Real-time aggregation
SELECT 
    date_trunc('hour', event_time) as hour,
    count(*) as events,
    avg(value) as avg_value
FROM sensor_data 
WHERE event_time >= now() - interval '24 hours'
GROUP BY hour
ORDER BY hour;

PostgreSQL Protocol Benefits

Universal Compatibility: Works with any PostgreSQL client
BI Tool Integration: Connect Tableau, Power BI, Grafana directly
Cursor Support: Efficient handling of large result sets
Prepared Statements: Full parameter binding support
Real-time Analytics: Query streaming data as it arrives
TLS Encryption: Available in Pro tier for secure connections

See the PGWire Server Guide for comprehensive setup instructions and advanced features.

Your First Stream

Create Topic and Stream Data

The SQL commands are identical for both free and pro versions - only the connection setup differs:

Free VersionPro Version

sql

-- Already connected with: ATTACH 'boilstream' (TYPE AIRPORT, location 'grpc://localhost:50051/');

-- Create sample topic (if not already exists)
CREATE TABLE boilstream.s3.people (
    name VARCHAR,
    age INT,
    tags VARCHAR[]
);

-- Stream to BoilStream
INSERT INTO boilstream.s3.people
SELECT
    'boilstream_' || i::VARCHAR as name,
    (i % 100) + 1 as age,
    ['airport', 'ducklake'] as tags
FROM generate_series(1, 1000) as t(i);

sql

-- Already connected with: ATTACH 'boilstream' (TYPE AIRPORT, location 'grpc+tls://localhost:50051/');
-- Using authentication secret for secure access

-- Create sample topic (requires write permissions)
CREATE TABLE boilstream.s3.people (
    name VARCHAR,
    age INT,
    tags VARCHAR[]
);

-- Stream to BoilStream (requires write permissions)
INSERT INTO boilstream.s3.people
SELECT
    'secure_' || i::VARCHAR as name,
    (i % 100) + 1 as age,
    ['airport', 'enterprise'] as tags
FROM generate_series(1, 1000) as t(i);

Verify the Data

Check that your data landed in S3:

bash

aws s3 ls s3://my-data-lake/events/

Query the data from your data lake:

sql

-- Query directly from S3
SELECT
    event_type,
    count(*) as count,
    avg(value) as avg_value
FROM 's3://my-data-lake/events/*.parquet'
GROUP BY event_type;

What Just Happened?

Free VersionPro Version

text

1. BoilStream received your data via FlightRPC (unencrypted)
2. Validated the schema and data (no authentication required)
3. Optimized the data into Parquet format
4. Uploaded directly to S3 with multipart uploads
5. Acknowledged completion back to DuckDB

Your data is now immediately available for analytics!

text

1. BoilStream received your data via FlightRPC (TLS encrypted)
2. Authenticated your JWT token and verified permissions
3. Validated the schema and data with authorized access
4. Optimized the data into Parquet format
5. Uploaded directly to S3 with multipart uploads
6. Acknowledged completion back to DuckDB

Your data is now securely stored and immediately available for analytics!

Next Steps

For Free Version Users

Explore more examples with different data types
Set up monitoring with Prometheus and Grafana

For Pro Version Users

Configure multiple identity providers
Set up fine-grained authorization
Deploy with production security best practices
Consider upgrading to enterprise support for SLA guarantees

Upgrade to Pro

Ready to secure your production deployment? Contact us for BoilStream Pro licensing and enterprise support.

Quick Start ​

Prerequisites ​

BoilStream ​

DuckDB Client ​

Pro Version Additional Requirements ​

Installation ​

Download Binary ​

Free Version Setup ​

Start BoilStream ​

Connect from DuckDB (Free Version) ​

Pro Version Setup ​

Start BoilStream Pro ​

Connect from DuckDB (Pro Version) ​

Connect via PostgreSQL Protocol ​

Quick PostgreSQL Connection ​

DBeaver Connection ​

Query Your Streaming Data ​

Your First Stream ​

Create Topic and Stream Data ​

Verify the Data ​

What Just Happened? ​

Next Steps ​

For Free Version Users ​

For Pro Version Users ​

Quick Start

Prerequisites

BoilStream

DuckDB Client

Pro Version Additional Requirements

Installation

Download Binary

Free Version Setup

Start BoilStream

Connect from DuckDB (Free Version)

Pro Version Setup

Start BoilStream Pro

Connect from DuckDB (Pro Version)

Connect via PostgreSQL Protocol

Quick PostgreSQL Connection

DBeaver Connection

Query Your Streaming Data

Your First Stream

Create Topic and Stream Data

Verify the Data

What Just Happened?

Next Steps

For Free Version Users

For Pro Version Users