Streaming Audio: A Confluent Podcast about Apache Kafka

Stream processing has become an important part of the big data landscape as a new programming paradigm to implement real-time data-driven applications. One of the biggest challenges for streaming systems is to provide correctness guarantees for data processing in a distributed environment. Guozhang Wang (Distributed Systems Engineer, Confluent) contributed to a leadership paper, along with other leaders in the Apache Kafka community, on consistency and completeness in streaming processing in Apache Kafka in order to shed light on what a reimagined, modern infrastructure looks like.

Using large amounts of streaming data increasingly requires interactive, real-time analytics and dashboards—and this applies to any industry, including tech. CTO and Co-Founder of Rockset Dhruba Borthakur shares how his company uses Apache Kafka to perform complex joins, search, and aggregations on streaming data with low latencies. The Kafka database integrations allow his team to make a cloud-native analytics database that is a fundamental piece of enterprise infrastructure.

TensorFlow is Google's version of wrangling various technologies to help group them together and work smoothly as large amounts of data flow through. Chris Mattmann, author of Machine Learning with TensorFlow, explains neural networks, how technology simulates cerebral processes that take place when our visual cortex receives a new image, and a use case that involves Apache Kafka® and event streaming to achieve TensorFlow's goals.

Kai Waehner defines machine learning in depth, describes the architecture of his dream machine learning pipeline, shares its relevance to Apache Kafka and the related ecosystem, and discusses the importance of security and fraud detection.

Tim Berglund and Michael Drogalis talk about all things stream processing: why it’s complex, how it has evolved, and what’s on the horizon to make it simpler.