FlightRPC ingestion
The producer FlightRPC service accepts Arrow batches through DoExchange. It is separate from the FlightSQL query service. Both use gRPC over HTTP/2, but their operations and completion rules differ.
Before sending data
Create the destination table in a streaming catalog. Obtain the producer endpoint, TLS trust and scoped credentials from your deployment. The typical producer port is 50051, but it is configurable. Use the exact client and extension builds validated with the server release.
For a complete SQL-based setup, use the Quick Start.
Direct Arrow Flight clients must use the producer's DoExchange contract, including its destination descriptor and response handling. A generic Flight connection or a successful write call alone does not establish ingestion completion. Keep input until the exchange has returned its terminal result and your required completion boundary is established.
Methods
| Operation | Purpose or limit |
|---|---|
DoExchange | Bidirectional streaming ingestion of Arrow batches |
Producer-service DoPut | Unimplemented; do not use for bulk uploads |
FlightSQL DoPut | Separate query service; do not use it as a supported bulk-ingestion endpoint |
Completion and visibility
Recent queries, stored history and live output can become available at different times.
With hot-only storage, exchange completion confirms admission rather than local commit. Do not use its reported count alone to prove that every input row is stored. With cold storage enabled, upload progress is still distinct from catalog registration and complete cold publication. A successful exchange does not certify completion of every derived view, search index or consumer.
Use Reliability and completion to choose the required boundary. An interrupted exchange may have accepted part of its input. Preserve the input and reconcile before replaying; automatic retries can duplicate rows.
Client errors
Authentication failures require valid credentials; schema failures require corrected input. Capacity and transport errors may be temporary, but retry safety depends on whether any data was accepted. Record both the transport result and the application-level response.
See HTTP/2 Arrow for a simpler request/row-count API and Interface compatibility for the current interface limits.