January 6, 2021 | Episode 137

Event Streaming Trends and Predictions for 2021 ft. Gwen Shapira, Ben Stopford, and Michael Noll

  • Transcript
  • Notes

Coming out of a whirlwind year for the event streaming world, Tim Berglund sits down with Gwen Shapira (Engineering Leader, Confluent), Ben Stopford (Senior Director, Office of the CTO, Confluent), and Michael Noll (Principal Technologist, Office of the CTO, Confluent) to take a guess at what 2021 will bring. 

The experts share what they believe will happen for analytics, frameworks, multi-cloud services, stream processing, and other topics important to the event streaming space. These Apache Kafka® related predictions include the future of the Kafka cluster partitions and removing restrictions that users have found in the past, such as too many variations and excessive concurrency as it relates to your number of partitions.

Ben also thinks that ZooKeeper will continue to maintain open source servers for highly reliable application distribution. Kafka clusters will still be able to keep the most important data while growing in size at record speed with ZooKeeper, although it will no longer be required with KIP-500 removing ZooKeeper dependency. This upgrade allows Kafka and ZooKeeper to run independently in deployment while Kafka’s cluster capability will increase.

Michael expects a continued need for COVID-19 tracking as well as enhanced event streaming capabilities. Ben believes that scalable Tiered Storage for Kafka will increase productivity and benefit workloads. Gwen predicts that databases will become more conventional by the end of next year, leading to new data architectural design with the support of Kafka.

Continue Listening

Episode 138January 11, 2021 | 43 min

Change Data Capture and Kafka Connect on Microsoft Azure ft. Abhishek Gupta

What’s it like being a Microsoft Azure Cloud advocate working with Apache Kafka® and change data capture (CDC) solutions? Abhishek Gupta would know!

Episode 139January 20, 2021 | 34 min

Scaling Developer Productivity with Apache Kafka ft. Mohinish Shaikh

Confluent Platform and Confluent Cloud run efficiently largely because of the dedication of the Developer Productivity team. Mohinish Shaikh (Developer, Confluent) talks about how his team builds the product pipelines for the entire event streaming platform and ensures seamless delivery of several engineering processes across engineering and the rest of the org.

Episode 140January 25, 2021 | 44 min

Distributed Systems Engineering with Apache Kafka ft. Guozhang Wang

Tim Berglund picks the brain of a distributed systems engineer, Guozhang Wang, tech lead in the Streaming department of Confluent. Guozhang explains what compelled him to join the Stream Processing team at Confluent coming from the Apache Kafka® core infrastructure.

Got questions?

If there's something you want to know about Apache Kafka, Confluent or event streaming, please send us an email with your question and we'll hope to answer it on the next episode of Ask Confluent.

Email Us

Never miss an episode!

Confluent Cloud is a fully managed Apache Kafka service available on all three major clouds. Try it for free today.

Try it for free