Hello and happy New Year to all our readers! A LOT happened in the world of data streaming in 2023, and before we say hello to everything coming this year, we thought it’s only right to reflect on some highlights from last year. Read on for a few of our favorite moments and readings, and what we are looking forward to in the new year!

Apache Kafka® and Apache Flink® Highlights From 2023:

• The Kafka community brought us three releases in 2023: 3.4, 3.5, and 3.6! These were packed with tons of new features and updates, including Tiered Storage early access, Versioned State Stores for Kafka Streams, and ZooKeeper being marked deprecated in 3.5.
• The Flink community was hard at work in 2023 as well, bringing us Flink 1.17 and 1.18! This meant upgrades like SQL Gateway, streaming SQL semantic enhancements, time travel SQL syntax, improvements to the Adaptive Scheduler, and much more!
• In May, we gathered for Kafka Summit London 2023. We heard incredible talks and stories from the community like A Kafka Client’s Request: There and Back Again, Playing with Xbox Data, and Versioned State Stores in Kafka Streams. You can relive all the talks with videos and slides from the event.
• In September, we had the opportunity to meet and hear from amazing members from across the data streaming world at Current 2023! You can now watch all the session recordings from the event on YouTube. Some crowd favorites from the event included Unlocking the Power of Apache Flink: An Introduction in 4 Acts, The Nuts and Bolts of Kafka Streams---An Architectural Deep Dive, and Evolution of Real-time User Engagement Event Consumption at Pinterest.
• Flink Forward 2023 convened in Seattle in November.! If you missed it, the recordings from Flink Forward sessions are now available on Ververica’s website.
• 2023 learning resources we loved: Some of our team’s favorites from the year for Kafka and Flink learners include…
  • Courses on Apache Flink 101, Mastering Production Data Streaming Systems with Apache Kafka, and Building Apache Flink Apps in Java
  • A set of 7 handy Flink SQL tutorials
  • The newly updated guide to running Kafka on Windows from Jim Galasyn
  • Stanislav Kozlovski’s 2 Minute Streaming
  • The award-winning paper Kora: A Cloud-Native Streaming Platform for Kafka
  • The Building an Event-Driven Data Mesh book by Adam Bellemare
  • Lucia Cerchie’s What Is an Apache Kafka Cluster? (And Why You Should Care) article and David Anderson’s A Developer’s Guide to Getting Started with Apache Flink for beginners to each technology
  • Bite-sized YouTube videos from Wade Waldron on Consuming and Producing Kafka Messages using Flink and Java

Now that we’ve reflected on 2023, it’s time to get excited about 2024! And there is plenty to look forward to already…

Looking at 2024 and Beyond…

• The release of Apache Kafka 3.7 is coming up quickly, and we’re getting closer and closer to Kafka 4.0! The 4.0 major release will fully remove ZooKeeper from Apache Kafka.
• On the Flink side, Apache Flink 1.19 is on the way in the near future, and the second major release of Flink is likely coming in 2024 with Flink 2.0! Hear from Martijn Visser what we have to look forward to with 2.0.
• 2024 is set to be full of exciting events! Kafka Summit is coming back even bigger this year, with Kafka Summit London on March 19-20 and the FIRST EVER Kafka Summit Bangalore on May 2nd. You can register for KSL now, and registration for KSB is coming soon!
• Current 2024 is coming to Austin, Texas on September 17-18 (don’t forget to submit your abstract to the call for speakers before February 26).
• Veverica announced that Flink Forward will also return in 2024, in Berlin on October 10th! More information on the event will arrive soon—we hope to see you there.

On to our usual programming…

Terminal Tip of the Week:

You can use kcat to consume from a specific partition, rather than the default, which is all the partitions in the topic.

❯ kcat -t topicname -C -p 0

% Reached end of topic topicname [0] at offset 902

From the example above, you can see that the way to consume from a specific partition is to use the -p flag followed by the number of the partition you want to consume from.

Links From Around the Web:

• Don’t forget to take a stretch break
• Real-time babies: See every time a baby is born in a specific country with the Baby Map
• Trolling chatbots with made-up memes

Upcoming Events:

• In-person meetup in Kraków (Jan 16): Building an Event Analytics Pipeline with Kafka, ksqlDB, and Druid
• Hybrid event in Mountainview, CA (Jan 18): Stream Processing with Apache Kafka, Samza, and Flink
• In-person meetup in Singapore (Jan 23): Using Kafka for Building Data Processing Pipeline
• In-person meetup in Sunnyvale, CA (Jan 31): Uber X Apache Flink® Meetup

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see this year, or you want to submit your own resource for consideration, send us an email at devx_newsletter@confluent.io!

If you’d like to view previous editions of the newsletter, visit our archive.

If you’re viewing this newsletter online, know that we appreciate your readership and that you can get this newsletter delivered directly to your inbox by filling out the signup form on the left-hand side.

P.S. If you want to learn more about Kafka, Flink, or Confluent Cloud, visit our developer site at Confluent Developer.