Stream into Iceberg with Fluss

Tier streaming data from Apache Fluss into Iceberg tables governed by Lakekeeper, then query the result from any Iceberg engine.

Runnable source: https://github.com/lakekeeper/lakekeeper/tree/main/examples/fluss.

Streaming and the lakehouse usually live in separate worlds: a stream store for fresh data, a catalog for historical tables, and a brittle pipeline gluing them together. Apache Fluss collapses that gap — it’s a streaming storage layer that can tier its data straight into Iceberg tables. Point that tiering at Lakekeeper and your streams land as governed Iceberg tables, queryable by any engine.

See it running: the fluss example runs Fluss, Flink, storage, and Lakekeeper together and drives the full flow end to end.

What Fluss adds

Fluss is a streaming data platform. With datalake tiering enabled, data written to a Fluss table is automatically pushed into Iceberg on a schedule — hot data stays in Fluss for low-latency reads, and the same records archive into Iceberg for cheap, long-term, engine-agnostic analytics. Lakekeeper is the Iceberg catalog the tiering service writes through, so tiered data is immediately governed and discoverable like any other table.

1. Start the stack

cd lakekeeper/examples/fluss
docker compose up -d
./run-demo.sh

docker compose up -d starts the services; run-demo.sh creates a datalake-enabled Fluss table, inserts data, and walks it through tiering into Iceberg.

Endpoints:

2. How the tiering connects to Lakekeeper

The Fluss tiering service is configured to use Lakekeeper’s REST catalog endpoint:

http://lakekeeper:8181/catalog

That’s the only integration point. Because Fluss writes through the catalog rather than straight to storage, the tiered tables inherit everything Lakekeeper provides — a single source of truth, credential vending, and access control — with no special-casing for the streaming path.

3. Query the tiered data

Once run-demo.sh has tiered a batch, the data is a normal Iceberg table. The demo queries it with DuckDB, but the point is that any Iceberg-compatible engine — Spark, Trino, StarRocks, Flink — reads the same table through the same Lakekeeper catalog. Your streaming output is now a first-class lakehouse table, not a bespoke sink.

Why this matters

Routing streaming ingestion through the catalog means the boundary between “streaming” and “the lakehouse” disappears from a governance point of view. The same permissions, the same short-lived credentials, and the same audit trail apply whether a row arrived via a batch job or a stream. There’s no second access model to secure.

Where to go next