Get Started Free

Apache Flink® Stateful Functions, Pub/Sub vs Point-to-Point, & CDC

February 8, 2024

This week’s resources are rich in code samples and demos. We’ve got an end-to-end CDC demo, an Apache Flink® SQL learning lab, and a Flink cookbook. There’s also an opportunity to contribute to Confluent CLI plugins, with a blog post outlining the reasons why you might. Then, zoom out to a big picture view with our point-to-point vs pub/sub comparison video. If you’ve only got a few minutes to learn something new, then read from our new feature: a highlighted StackOverflow post. This time, it’s about why Flink needs watermarking.

Apache Kafka® Resources:

  • Give your tab-complete key a vacation with a custom CLI plugin! Bill Bejeck explains the why and how of creating plugins for Confluent’s CLI tool.
  • Point-to-Point vs publish/subscribe… what are the key differences and why would you pick one over the other? Wade Waldron explains in this new video.
  • Change Data Capture (CDC) is capturing database changes in realtime and sending them to a downstream application. Walk through this end-to-end demo to learn how to use CDC with Apache Kafka®.

Apache Flink® Resources:

  • Did you know Reddit uses Flink stateful functions? Find out why in this Reddit post.
  • Follow along in Jan Svoboda’s Flink SQL learning lab to get comfortable with Flink Tables, select statements, aggregations, time windows, and more!
  • Double, double, toil and trouble… learn how to create your own Flink magic with David Anderson’s Flink cookbook of recipes.
  • Flink 1.18.1 has been released! Developers now have access to log checkpoint statistics and an article on how to migrate from dataset to datastream.

A Droplet From Stack Overflow:

We loved the depth of this answer from David Anderson on why Flink needs watermarking.

Got your own favorite Stack Overflow answer related to Flink or Kafka? Send it in to devx_newsletter@confluent.io!

Terminal Tip of the Week:

Since Flink 1.18, you can specify watermarking options such as the idle-timeout on a per-query basis by using SQL hints, for example:

select ... from source_table /*+ 
OPTIONS('scan.watermark.idle-timeout'='1min') */

Links From Around the Web:

In Other News:

In-person:

Hybrid:

- Hybrid meetup in Tokyo, JP (Feb 16): Apache Kafka® Meetup Japan #14

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see, or you want to submit your own resource for consideration, email us at devx_newsletter@confluent.io !

If you’d like to view previous editions of the newsletter, visit our archive.

If you’re viewing this newsletter online, know that we appreciate your readership and that you can get this newsletter delivered directly to your inbox by filling out the signup form on the left-hand side.

Subscribe Now

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.

Recent Newsletters