Introducing flink-watermarks.wtf

November 6, 2025

Introducing flink-watermarks.wtf — a visual explainer for Apache Flink® watermarks

If you are a data streaming engineer, chances are you have definitely spent time diving deep into watermarks. Here's a whole website dedicated to learning the JEDI art of mastering Apache Flink® watermarks! An essential visual learning tool created by Elijah Meeks, Robin Moffatt, David Anderson, and Gunnar Morling from Confluent, dedicated to the data streaming community!

This new interactive site, flink-watermarks.wtf demystifies how watermarks drive time-based operations in Apache Flink, especially in Flink SQL. It walks through event-time vs. processing-time, out-of-order and delayed events, and how watermark delay affects the accuracy–latency balance, using animated, hands-on simulations.

What you will find inside:

  • Concept-first animations that show how windows close only when the watermark passes the window end, and why late events are dropped
  • Practical guidance on choosing a watermark delay (like, avoid true "zero" by using a tiny positive delay)
  • Gotchas around idle sources and end-of-day gaps, plus patterns to "flush" pipelines so results emit as expected
  • Curated links to official docs and talks

flink watermarks gif

                                                                            flink-watermarks.wtf

Be the young Padawan and learn Apache Flink watermarks from this site! May the “watermark” be with you!

New Confluent Cloud Security Whitepapers

Confluent announces the release of four new in-depth white papers on the Confluent Trust Center.

This new library of white papers offers comprehensive insights into the security, compliance, and operational practices that underpin Confluent Cloud, reinforcing our commitment to transparency and trust. They are designed to help Confluent Cloud customers better understand and leverage Confluent's security capabilities and manage your own security and compliance needs when using Confluent.

Here's a look at the new papers and what data streaming engineers can expect to learn:

These new resources will empower data streaming engineers o build and maintain more secure and compliant data streaming applications.

Where to Access:

All four white papers are available for immediate download in the White Papers section of the Confluent Trust Center.

Data Streaming Resources

  • Confluent Cloud users have been enjoying Tableflow to materialize Kafka topics, but there is a need for a BYOC version of Tableflow so that all of Confluent's WarpStream users can get the same benefits of Tableflow in their own cloud account with a BYOC deployment model. That's precisely what Richard Artoul, Co-Founder of WarpStream, talks about when he introduces WarpStream Tableflow. Learn more from his fascinating blog, diving deep into Iceberg metadata purge strategies, small file compaction, and a lot more!
  • The data streaming team at SoftwareMill has published an open-source, AI-powered Kafka Diagnostics Tool written in Rust that helps identify Kafka cluster issues. KCPilot is a CLI tool that helps data streaming engineers gather all the data and analyze it to identify common problems that can occur in a Kafka cluster. It utilizes LLMs to execute each analysis task and is easily extensible, allowing you to add new tasks and solve more problems in the future. Read more from the SoftwareMill blog on KCpilot. Here’s the github repo with installation instructions.

Links From Around the Web:

  • Learn from Yingjun Wu, the founder of RisingWave, how RisingWave built and utilizes Foyer, a hybrid caching library in Rust that unifies memory and disk caching into one coherent layer for low-latency real-time applications. A cache keeps hot data in memory and warm data on local disk and reduces the number of requests sent to cloud object stores like S3. This lowers latency, stabilizes performance, and reduces recurring storage costs. Applications no longer need to fetch the same data repeatedly from remote storage but can instead serve most queries locally while still relying on S3 for durability and scale. Read the thought-provoking blog from Yingjun Wu, published on Medium. Here’s the GitHub repo for Foyer.
  • On October 19th and 20th, 2025, the world witnessed the dreaded AWS "region failure," which lasted for over 15 hours due to a DNS issue! While it is easy to complain and point fingers, it is wiser to learn and understand the different ways in which planet-scale distributed systems like AWS fail. Here’s the official summary from AWS on the DynamoDB service disruption, which is a gold mine for any architect planning to design distributed systems. It is a fascinating piece of post-mortem analysis done by some of the smartest IT people on our planet!

In-Person Meetups:

Stay up to date with all Confluent-run meetup events - by copying the following link into your personal calendar platform: https://airtable.com/app8KVpxxlmhTbfcL/shrNiipDJkCa2GBW7/iCal?timeZone=America%2FChicago&userLocale=en (Instructions for GCal, iCal, Outlook, etc.) 

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see, or you want to submit your own resource for consideration, email us at devx_newsletter@confluent.io.

If you’d like to view previous editions of the newsletter, visit our archive.

If you’re viewing this newsletter online, know that we appreciate your readership and that you can get this newsletter delivered directly to your inbox by filling out the sign-up form below.

P.S. If you want to learn more about Kafka, Flink, or Confluent Cloud, visit our developer site at Confluent Developer.

Subscribe Now

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.

Recent Newsletters

November 6, 2025

Introducing flink-watermarks.wtf

September 25, 2025

Introducing Apache Kafka® 4.1.0

August 28, 2025

Event-driven AI agents