Skip to content
BoilStream

Discover Avro schemas

reference
TL;DR

Discover schemas for tables already provisioned through SQL. Authenticate with a Schema Registry token, list accessible subjects and retrieve the schema and ID returned by the service. Configure serializers to use existing schemas. Registry requests do not create arbitrary table schemas, and Kafka ingest tokens are not registry credentials.

Message sequenceSQL owns the table; schema discovery helps encode producers
SQL owns the table; schema discovery helps encode producersAn operator creates the target table through SQL. A producer reads a compatible schema from the registry and uses it to serialize records. Registry discovery does not create or alter the SQL table.SQL clientBoilStreamProducerCreate target tableRead registry subject / schemaCompatible Avro schemaSerialize matching recordsSend records to ingestionTime moves downward · dashed arrows are responsesSQL owns the table; schema discovery helps encode producersAn operator creates the target table through SQL. A producer reads a compatible schema from the registry and uses it to serialize records. Registry discovery does not create or alter the SQL table.SQL clientBoilStreamProducerCreate target tableRead registry subject /schemaCompatible Avro schemaSerialize matchingrecordsSend records to ingestionTime moves downward · dashed arrows are responses
The registry is read-only. Manage table shape through the supported SQL path.
Jump to reference ↓

BoilStream provides Confluent-compatible Avro schema discovery at https://HOST/schema-registry. Create the streaming table through SQL before producing records. Its schema is managed through the table definition.

Authentication

Section titled “Authentication”

Every request requires a Schema Registry token from the dashboard, sent as a bearer token. An ingest token has a different purpose and cannot be used here. Requests without a valid token return 401; a token with the wrong purpose returns 403.

Set REGISTRY_URL to your deployment’s registry base URL and SCHEMA_REGISTRY_TOKEN to the issued token. Use your deployment’s trusted CA when it has a private certificate.

Shell
curl --fail-with-body --silent --show-error \
  --header "Authorization: Bearer $SCHEMA_REGISTRY_TOKEN" \
  "$REGISTRY_URL/subjects"

Use the returned subject name unchanged. A subject uses the provisioned Kafka topic name with a -value suffix; it is not necessarily just the SQL table name. Results depend on the authenticated user’s catalog access.

Find a schema

Section titled “Find a schema”

Set SUBJECT to a URL-encoded subject returned by discovery:

Shell
curl --fail-with-body --silent --show-error \
  --header "Authorization: Bearer $SCHEMA_REGISTRY_TOKEN" \
  "$REGISTRY_URL/subjects/$SUBJECT/versions/latest"

The response contains subject, version, id and schema. The schema is a JSON string containing an Avro schema. Use the returned ID and the matching schema when serializing records; do not assume an ID or version from another deployment.

Read endpoints

Section titled “Read endpoints”

All paths below are relative to /schema-registry and require authentication.

Method Path Result
GET / Health response
GET /v1/metadata/id Registry identifier
GET /schemas/types Supported schema types: Avro
GET /subjects Accessible subjects
GET /subjects/{subject}/versions Versions for a subject
GET /subjects/{subject}/versions/{version} Schema and ID; latest is accepted
GET /subjects/{subject}/versions/{version}/schema Schema without the metadata wrapper
GET /schemas/ids/{id} Schema by ID
GET /schemas/ids/{id}/schema Schema without the metadata wrapper
GET /schemas/ids/{id}/versions Subject/version pairs for the ID
GET /schemas/ids/{id}/subjects Subjects for the ID
GET /config or /config/{subject} Compatibility configuration
GET /mode or /mode/{subject} Registry mode

Lookup and compatibility

Section titled “Lookup and compatibility”

These POST requests inspect schemas; they do not change table definitions:

Method Path Purpose
POST /subjects/{subject} Find a schema already registered for a subject
POST /compatibility/subjects/{subject}/versions/{version} Compare a schema with one version
POST /compatibility/subjects/{subject}/versions Compare a schema with existing versions

Send JSON with a schema field containing the Avro schema as a string. Check the response before changing a producer. A compatibility result is not a migration of existing data.

Read-only schema management

Section titled “Read-only schema management”

Use SQL table definitions to create or evolve schemas. The registry does not accept arbitrary new schemas, deletions or compatibility/mode changes. A registration request to POST /subjects/{subject}/versions can return the existing ID when the schema already matches; it does not create a new schema.

Compatibility reads use one registry-wide policy (initially BACKWARD). Subject configuration requires access to the current owner; an unclaimed historical subject remains readable with access to its retained schema. Subject-name overrides are not supported and do not transfer when a table is renamed or its old name is reused. Configuration PUT/DELETE requests return HTTP 422 with error code 42205.

Configure serializers with auto.register.schemas=false and authenticated registry access. Kafka SASL credentials and registry bearer authentication are separate settings. Test the serializer version against the table’s returned Avro schema.

Kafka payload

Section titled “Kafka payload”

The Confluent Avro payload contains a zero magic byte, a four-byte big-endian schema ID and the Avro data. Kafka RecordBatch v2 wraps these payloads. See Kafka for producer settings and acknowledgement limits.

Errors

Section titled “Errors”

Check both the HTTP status and response body. Schema lookup errors can contain error_code and message; authentication failures may return plain text. A missing subject or schema may mean that it is absent or inaccessible to the requesting user. Correct token, subject and schema errors before retrying.