Get Started Free

Forking Apache Flink® and Surviving an Apache Kafka® Outage

February 22, 2024

This week, we’ve got a new take on the terminal tip: it’s a challenge! Can you see what the error in the SQL is? Check it out below, and then email us the answer at What happens when things go wrong in data streaming setups? If you shiver in your boots at the thought of an Apache Kafka® outage, read Jakub Korab’s post on battening down the hatches. Or, if you have a microservice setup, then you might lose transactions in the dual write problem, as explained in Wade Waldron’s new video. On the Apache Flink® side of things, Seth Saperstein outlines the issue when the team at Lyft was seeing no upstream data.

We hope you enjoy this edition of the newsletter! We are always looking for submissions and suggestions at the email address above.

Apache Kafka® Resources:

Apache Flink® Resources:

  • You might want to customize Flink on the fly if you’re under a time crunch, and you need to adjust a feature or fix a bug yourself. In that case, branching, versioning, building and maintenance are all very important. Yaroslav Tkachenko has some practical tips in his blog post.
  • Seth Saperstein outlines a unique problem he encountered using Flink as a processor and Kinesis as a source, and the steps the team at Lyft took to solve it
  • Ever run into an “unknown magic number” error when using Flink? Check your serialization setup! See the paragraph in this blog post on why you should.
  • A Guide to Windowing in Kafka Streams and Flink SQL: read this post by Bill Bejeck to understand the syntax and philosophy behind windowing in these two streaming technologies.
  • Have you been trying to install PyFlink… with no luck? Read Robin Moffatt’s guide to get up and running.

A Droplet From Stack Overflow:

Need process data in Flink, but you’re consuming and producing back to Kafka? And this is all happening on Docker? Martijn Visser has a sample docker file for you!

Got your own favorite Stack Overflow answer related to Flink or Kafka? Send it in to!

Terminal Tip of the Week:

Can you spot the error in this Flink SQL query in Confluent Cloud? We won’t give it away until the next newsletter! Email us the answer and we’ll give you a shoutout:

  `rice_krispy` STRING,
  `canoli` STRING,
  `gulab_jamun` STRING
) WITH (
  'connector' = 'kafka'

Links From Around the Web:

In Other News:


__Hybrid: __

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see, or you want to submit your own resource for consideration, email us at!

If you’d like to view previous editions of the newsletter, visit our archive.

If you’re viewing this newsletter online, know that we appreciate your readership and that you can get this newsletter delivered directly to your inbox by filling out the signup form on the left-hand side.

P.S. If you want to learn more about Kafka, Flink, or Confluent Cloud, visit our developer site at Confluent Developer.

Subscribe Now

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.

Recent Newsletters