Enhance your career, get your certificate as a Data Streaming Engineer | Get your Certificate

Confluent Developer Blog

Apache Kafka

Catching Schema Drift at Service Startup

How to catch breaking schema changes early, then verify the same contract again when a Spring Boot service starts against the registry it will actually use.

How We Build

Optimizing the Confluent Platform release pipeline

How Confluent Platform, an on-premises product, went from a 7.5-hour build and months-long releases to shipping in 3–4 days: parallelized builds, pilot builds gating every PR, pipeline failures treated as SEV-2 incidents, a single shared test quality score, and components released independently of the core.

Apache Kafka

Streaming IoT sensor data from MQTT to Kafka

Stream MQTT sensor data into Kafka: a walkthrough of Mosquitto, the Confluent MQTT Connector, HiveMQ, Zilla, AWS IoT Core & Kepware.

Apache Flink

Deduplicating Streams with Flink SQL

A deep dive into deduplicating streams with Flink SQL, weighing state growth, changelog modes, late events, and latency, and why a custom PTF can beat clever SQL.

Apache Flink

How I Review Flink SQL Solutions

A framework for reviewing Flink SQL solutions: state growth, append-only vs. updating output, late events, latency, and determinism before going to production.

Apache Flink

Flink SQL Evolution: Handling Custom CDC with FROM_CHANGELOG and TO_CHANGELOG

How the new FROM_CHANGELOG and TO_CHANGELOG built-in functions let Flink SQL read custom CDC formats and turn updating streams back into append-only ones.

Apache Flink

Absolutely Everything You Always Wanted to Know About Watermarks in Apache Flink - Part 2: Confluent Cloud for Apache Flink

Confluent Cloud for Apache Flink watermarks deep-dive. Differences and extensions to Apache Flink: default watermarks, idleness, alignment, late events handling, watermark observability.

Apache Flink

Absolutely Everything You Always Wanted to Know About Watermarks in Apache Flink - Part 1: Apache Flink

Deep-dive reference on Apache Flink watermarks: generation, propagation, idle partition detection, alignment, and why lateness is non-deterministic.

Apache Parquet

A Fast Path for Fixed-Length Lists in Parquet

How Hardwood detects fixed-length lists in Parquet's encoded definition and repetition levels, bypassing Dremel reconstruction for a 1.1×–3.9× parse speed-up.

Apache Kafka

Kafka Queues for Quarkus

Learn how to use Queues for Apache Kafka (KIP-932) with the Quarkus framework, including share group configuration, explicit message acknowledgment, and cooperative consumption from a single partition.