Get Started Free

Flink community, filtering PII with Apache Kafka®, and the Laws of Simplicity

June 15, 2023

It’s June 15th, A.K.A Global Wind Day. In the spirit of renewable energy, we invite you to read Kai Waehner’s (of Confluent) blog post on using Kafka architecture to implement a smart grid. What is a smart grid? Read on! Also featured in today’s newsletter: tapping into the Flink community, detecting PII automatically in real time, and going back in time to your first GitHub commit.

Apache Kafka Resources:

  • Robbie Palmer describes how to detect PII in real time using Kafka. This solution is powered by machine learning and furthermore, it’s automatic!
  • What does teleportation have to do with Kafka? Confluent’s Wade Waldron is glad you asked. Learn the ways to deduplicate messages through a philosophical analogy.

Apache Flink® Resources:

  • If you’d like to get help with any Flink questions or just want to tap into the community, there’s the Flink community slack.
  • If you think you may have found a bug but you’re not positive, using the Flink mailing list is a great way to pre-determine whether you’re encountering a user error before filing a Jira ticket.

Terminal Tip of the Week:

Let’s go back to basics with kcat, a command line utility that helps test and debug Kafka projects. There are two modes in which kcat runs: consumer mode and producer mode. Let’s take a look at these two examples from Confluent’s docs.

 kcat -b localhost:9092 -t mysql_users -C -c1
{"uid":1,"name":"Cliff","locale":"en_US","address_city":"St Louis","elite":"P"}

Above, we see kcat running in consumer mode, determined by the -C flag. (The -b and -t flags set the broker and topic, and the -c1 flag only outputs the first line.) It works by reading the messages from the topic and printing them to stdout.

 kcat -b localhost:9092 -t new_topic -P

Here, kcat is running in producer mode since we’ve added the -P flag. It reads from stdin to send the message to the new_topic topic.

Links From Around the Web:

  • LessWrong is an interesting online community devoted to fine-tuning rationality
  • Enter your GitHub username and view your first-ever PR!
  • John Maeda’s Laws of Simplicity help smooth out business-oriented problem-solving

In Other News:

  • Join us on June 22 for a Confluent Live Lab on Kafka in Bangkok!
  • Registration for Current 2023 is now open! There are more than 100 sessions at Current, all focused on real-time data streaming. Join us, we’d love to have you!
  • Sessions and talks from Kafka Summit London are now available!
  • Make new friends and learn new data streaming skills at Camp Confluent! Registration is open now and camp starts July 10th.

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see, or you want to submit your own resource for consideration, send us an email at!

If you’re viewing this newsletter online, know that we appreciate your readership and that you can get this newsletter delivered directly to your inbox by scrolling down to the bottom of the page and filling out the form.

P.S. If you want to learn more about Kafka or Confluent Cloud, visit our developer site at Confluent Developer.

Subscribe Now

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.

Recent Newsletters