Get Started Free

Kafka Cluster Strategy, Flink Apps in Java, and Temporal Joins

August 10, 2023

Greetings, developers, data engineers, architects, and data streaming professionals of all types! We’ve got some good resources for you this week. See why you should spin up more than one Kafka cluster in a new blog post, learn practical event modeling in a new video course, and dive deep into Flink’s Temporal Join. Without further ado…

Apache Kafka® Resources:

  • When Should You Spin Up Your Next Apache Kafka Cluster? Kafka clusters can handle, well, a lot of partitions. 200,000 in fact! But there are valid reasons for spinning up a new cluster before you hit that limit. Find out what they are in this technical article by Sanjay Garde, who has worked with Confluent customers for over three years and has seen many real-life situations that call for spinning up another cluster.
  • Introducing Software Bill of Materials for Confluent Platform: A Software Bill of Materials (SBOM) is a standardized way to track software components. Here, you can find a practical guide to using Confluent Platform’s SBOM in three different scenarios, including the use of Dependency Track.
  • Practical Event Modeling: Community member Bobby Calderwood has partnered with Confluent to create a course on modeling events for event-based information systems! Get started here.
  • Change Data Capture Demo: one way to keep track of all the metadata your relational databases generate is by using Kafka. You can get started with capturing this data by cloning this repo and digging around to see for yourself!

Apache Flink® Resources:

  • Building Apache Flink® Applications in Java: Learn how to build Flink apps in Java with our course, which we’ve updated recently with modules on keyed states, windowing and watermarks, and how to create branching data streams.
  • “Enter the Temporal Join, the DeLorean of data operations” Want to know what a Temporal Join is and how it can affect “time travel” for Flink joins? Read this article by DataSQRL to find out!

Terminal Tip of the Week:

Let's talk about the -Z flag in kcat for a moment.

This is a very special flag. As you can see from its definition in the code, it will interpret empty message values as null. What's important about that? Well, it enables us to set up tombstones via kcat:

Here, you pipe the name of the key you want to be removed directly into kcat, the -Z flag ensures that the empty message is interpreted as null and therefore becomes a tombstone. FYI, the -K flag is the key delimiter.

Links From Around the Web:

In Other News:

  • Upcoming, in-person meetup in Vienna: Thinking In Events/Kafka and System of Records on Sep 26. RSVP here, there will be snacks and drinks!
  • Registration is now open for Current 2023, in San Jose, Sep 26-27! Network with data streaming experts from across the globe and learn new approaches to using Kafka, Flink, and other streaming technologies in the breakout sessions.

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see, or you want to submit your own resource for consideration, send us an email at devx_newsletter@confluent.io!

If you’re viewing this newsletter online, know that we appreciate your readership and that you can get this newsletter delivered directly to your inbox by scrolling down to the bottom of the page and filling out the form.

P.S. If you want to learn more about Kafka or Confluent Cloud, visit our developer site at Confluent Developer.

Subscribe Now

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.

Recent Newsletters