This week, we’ve got a new take on the terminal tip: it’s a challenge! Can you see what the error in the SQL is? Check it out below, and then email us the answer at devx_newsletter@confluent.io. What happens when things go wrong in data streaming setups? If you shiver in your boots at the thought of an Apache Kafka® outage, read Jakub Korab’s post on battening down the hatches. Or, if you have a microservice setup, then you might lose transactions in the dual write problem, as explained in Wade Waldron’s new video. On the Apache Flink® side of things, Seth Saperstein outlines the issue when the team at Lyft was seeing no upstream data.

We hope you enjoy this edition of the newsletter! We are always looking for submissions and suggestions at the email address above.

Apache Kafka® Resources:

• A guide from community member Dave Klein: Streaming Event Data to Iceberg with Kafka Connect.
• “Availability needs to be treated like security…” wise words from Jakub Korab in his blog post How to Survive an Apache Kafka Outage.
• Wade Waldron de-mystifies the dual write problem in this video about microservices.
• Understand the architecture behind building a real-time field ops alert system with Confluent Cloud and Slack by reading this blog post from Matt Mangia.

Apache Flink® Resources:

• You might want to customize Flink on the fly if you’re under a time crunch, and you need to adjust a feature or fix a bug yourself. In that case, branching, versioning, building and maintenance are all very important. Yaroslav Tkachenko has some practical tips in his blog post.
• Seth Saperstein outlines a unique problem he encountered using Flink as a processor and Kinesis as a source, and the steps the team at Lyft took to solve it
• Ever run into an “unknown magic number” error when using Flink? Check your serialization setup! See the paragraph in this blog post on why you should.
• A Guide to Windowing in Kafka Streams and Flink SQL: read this post by Bill Bejeck to understand the syntax and philosophy behind windowing in these two streaming technologies.
• Have you been trying to install PyFlink… with no luck? Read Robin Moffatt’s guide to get up and running.

A Droplet From Stack Overflow:

Need process data in Flink, but you’re consuming and producing back to Kafka? And this is all happening on Docker? Martijn Visser has a sample docker file for you!

Got your own favorite Stack Overflow answer related to Flink or Kafka? Send it in to devx_newsletter@confluent.io!

Terminal Tip of the Week:

Can you spot the error in this Flink SQL query in Confluent Cloud? We won’t give it away until the next newsletter! Email us the answer and we’ll give you a shoutout: devx_newsletter@confluent.io

CREATE TABLE Treat (
  `rice_krispy` STRING,
  `canoli` STRING,
  `gulab_jamun` STRING
) WITH (
  'connector' = 'kafka'
  ...
);

Links From Around the Web:

• Google’s AI-powered TextFX project
• False proofs that look correct from the CS Stack Exchange
• What did YouTube look like in 2005? Jog your memory with the web design museum

In Other News:

In-person:

• In-person meetup in the Bay Area (Feb 28): Apache Kafka® Meetup in the Bay Area - Feb 2024

__Hybrid: __

• Current 2024 (Sep 17-18): The call for papers for Current 2024 is now open! Submit your abstracts no later than Feb 26.
• Kafka Summit London (March 19-20): Registration for KSL is open! In addition, you can now see the agenda and speakers for the event.
• Kafka Summit Bangalore (May 2): Registration for KSB is expected to open soon, stay tuned for Kafka Summit, Bangalore edition!

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see, or you want to submit your own resource for consideration, email us at devx_newsletter@confluent.io!

If you’d like to view previous editions of the newsletter, visit our archive.

If you’re viewing this newsletter online, know that we appreciate your readership and that you can get this newsletter delivered directly to your inbox by filling out the signup form on the left-hand side.

P.S. If you want to learn more about Kafka, Flink, or Confluent Cloud, visit our developer site at Confluent Developer.