Get Started Free

Subscribe Now

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.

The Confluent DevX Newsletter

Sending With Librdkafka, Primer on PyFlink, & Apache Flink® 1.17.2!

December 14, 2023

‘Tis the season for lots of new resources! This week, we have an advent calendar from Confluent’s data streaming elves, a primer on PyFlink, the only guide you’ll need for producing messages with librdkafka, a roundup of the best free learning resources for Apache Kafka® beginners, and some quick and easy Apache Flink® learning. As always, we’re keeping you up to date with news on Kafka Summit London, Kafka Summit Bangalore, and Current.

With this edition, we’re signing off for 2023, and we’ll be back with lots more to share in 2024! Thank you for reading our newsletters this year, we hope they were helpful and entertaining. If you have any suggestions for what you’d like to see from us in 2024, please send your ideas to devx_newsletter@confluent.io. We’d love to hear from you.

Enjoy, and happy holidays to those celebrating!

Apache Kafka® Resources:

  • librdkafka is the main client library used in non-JVM environments, with wrapper libraries for Python, .Net, Go, and a growing list of clients (take a look!). If you want to get hands-on with librdkafka, check out Jakub Korab's new extensive guide on producing messages with the library.
  • Writing to and reading from Kafka looks simple on the surface, but under the covers, it is a complex process with a fascinating life cycle! Danica Fine’s talk at Berlin Buzzwords 2023 celebrates the journey.
  • Kafka beginners, rejoice! There is a lot of beginner learning material to sift through, so Peter Moskovits shared his top 7 free learning resources for complete beginners to Kafka. This curated list of courses and tutorials should get you started on your way to building your first Kafka app.
  • If you've found that your data streaming architecture has become complex with a number of siloed clusters, when does it make sense to consider centralizing to a single, or at least fewer, clusters? A new blog post from Lyndon Hedderly, Andrew Sellers, and Jon Vines helps you answer that and other questions about dealing with enterprise clusters.
  • Ali Moradi introduced Mockafka-py, an open source Python library designed specifically for simulating Kafka in a testing environment.

Apache Flink® Resources:

Terminal Tip of the Week:

kcat starts with a couple of defaults that are helpful to know about.

First of all, it automatically uses consumer mode:

❯ kcat -t topic_name
% Auto-selecting Consumer mode (use -P or -C to override)

If you want to run it in Producer mode, use -P, as indicated in the output.

Second, the default delimiter is newline:

kcat -t topic_name
% Auto-selecting Consumer mode (use -P or -C to override)
{"key":"value": ["string"]}
{"key":"value": ["string"]}

If you'd like to use a delimiter, add -K with your desired delimiter.

Links From Around the Web:

In Other News:

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see, or you want to submit your own resource for consideration, send us an email at devx_newsletter@confluent.io!

If you’d like to view previous editions of the newsletter, visit our archive.

If you’re viewing this newsletter online, know that we appreciate your readership and that you can get this newsletter delivered directly to your inbox by filling out the signup form on the left-hand side.

P.S. If you want to learn more about Kafka, Flink, or Confluent Cloud, visit our developer site at Confluent Developer.