April 22, 2019 | Episode 29

Diving into Exactly Once Semantics with Guozhang Wang

  • Notes

It has been said that in distributed messaging, there are two hard problems: 2) exactly once delivery, 1) guaranteed order of messages and 2) exactly once delivery. Apache Kafka® has offered exactly once processing since version 0.11, which allows properly configured producers and consumers to make the guarantee that each message will be processed exactly one time. 

In this episode, Kafka Streams engineer Guozhang Wang walks through the implementation of transactional messaging in Kafka in some detail, including the idempotent producer API, the transaction coordinator responsible for managing the transaction log and consumer configurations. It’s a complex topic, but he takes us through it carefully and completely.

EPISODE LINKS




Continue Listening

Episode 30May 2, 2019 | 38 min

Load-Balanced Apache Kafka: Derivco’s Globally Distributed Gaming Business

Derivco must maintain central data infrastructure and application clusters in geographically diverse locations due to the heavily regulated nature of their business. In this episode, we talk about the challenges associated with that unusual but ingenious network architecture, along with other lessons they learned as they built their Kafka-centric gaming platform.

Episode 31May 8, 2019 | 33 min

Ask Confluent #13: Machine Learning with Kai Waehner

Gwen and Kai chat about machine learning architectures, and whether software engineers and data science can learn to get along.

Episode 32May 16, 2019 | 50 min

Chris Riccomini on the History of Apache Kafka and Stream Processing

Chris Riccomini tells us how Apache Kafka® and the stream processing framework Samza came about, and also what he’s doing these days at WePay—building systems that use Kafka as a primary datastore.

Got questions?

If there's something you want to know about Apache Kafka, Confluent or event streaming, please send us an email with your question and we'll hope to answer it on the next episode of Ask Confluent.

Email Us

Never miss an episode!

Be the first to get updates and new content

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.