Get Started Free

Subscribe Now

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.

The Confluent DevX Newsletter

Tuning Apache Flink® Clusters & a Kafka Diagram

January 25, 2024

Hello everyone, hope your year is off to a good start! In our resource list today we’ve got some very cool code repository diagrams, sage advice from Pinterest engineers on tuning Flink clusters, and a post on why ZenDesk uses mTLS with Kafka. Our "terminal tip" is Flink-themed this week, with a snippet demonstrating how to fix watermarking issues. You will also find several in-person meetups coming up soon, so if you’re near London, Bangalore, Berlin, Kansas City, or Sunnyvale in the next month or so, stop on by!

Apache Kafka® Resources:

Apache Flink® Resources:

  • This map by Amelia Wattenberger outlines the Apache Flink repository.
  • Read Soumil Shah’s post about how to read data from a Kinesis stream locally with the Flink Python client.
  • Learn how to tune Flink clusters for availability and scalability from this Pinterest engineering blog post.

Terminal Tip of the Week:

This week’s terminal tip focuses on Flink SQL: if your windows or MATCH_RECOGNIZE queries fail to produce any results, often the root cause is a problem with watermarking. You can use the built-in CURRENT_WATERMARK function to check if watermarks are being generated correctly. E.g., if your time attribute column is named $rowtime:

SELECT ..., $rowtime, CURRENT_WATERMARK($rowtime) FROM table;

Links From Around the Web:

Upcoming Events:

In-person:

In-person and online:

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see, or you want to submit your own resource for consideration, send us an email at devx_newsletter@confluent.io!

If you’d like to view previous editions of the newsletter, visit our archive.

If you’re viewing this newsletter online, know that we appreciate your readership and that you can get this newsletter delivered directly to your inbox by filling out the signup form on the left-hand side.

P.S. If you want to learn more about Kafka, Flink, or Confluent Cloud, visit our developer site at Confluent Developer.