Apache Kafka® and Apache Flink® resources abound this week…

On the architectural level, we’ve got a brand-new Confluent Developer course on event-driven microservices and a demo showing how to use Kafka as a data fabric. Then, zoom in and get technical with a Flink cookbook, a blog post on tumbling window aggregation in PyFlink, and a way to define alerts in Splunk based on Confluent Cloud audit logs. Plus, an exciting announcement: the agenda for Current 2024 has been revealed! Check it out! We hope you’ll be joining us for the keynote, talks from the community, and the Current 2024 party.

Apache Kafka Resources:

• Define alerts in Splunk based on Confluent Cloud audit logs by setting up a fully managed pipeline with this repository. This tool comes from Patrick Neff, a customer success technical architect from Confluent, bringing his professional experience to help us out.
• If you run into logging issues during configuration of Confluent Platform, consult the new documentation on configuring audit logs to get untangled.
• From monoliths to microservices… learn how to design an event-driven microservice with the new Confluent Developer course from Wade Waldron
• See a demo of Kafka as a way to implement a data fabric. What is a ‘central nervous system’, exactly, and how does Kafka decouple the cloud and edge layers? To answer this question, this demo emulates hundreds of edge devices communicating via MQTT, HTTP, RabbitMQ, Syslog, Kafka and CoAP transmitting temperature and GPS coordinates across the UK. All of this data is normalized using ksqlDB in Confluent Cloud.

Apache Flink Resources:

• Get a hands-on explainer of tumbling window aggregation in PyFlink with Diptiman Raichaudhuri’s blog post
• The Apache Flink cookbook is now available with all sorts of Flink “recipes” to help you concoct your Flink project
• Flink is now generally available on Confluent Cloud—you can get the details and how-to from the new docs section or see a summary of features in the video from Martijn Visser

A Droplet from Stack Overflow:

Learn how UNNEST works with Flink SQL queries where nested struct with multiple fields are present, here.

Got your own favorite Stack Overflow answer related to Flink or Kafka? Send it in to devx_newsletter@confluent.io!

Terminal Tip of the Week:

Find metadata and data dictionary information for Confluent Flink tables, using the INFORMATION_SCHEMA. Log in to the Confluent CLI shell:

confluent login --save --organization-id ${ORG_ID}

Run the following command to enter the Confluent Flink SQL shell:

confluent flink shell --compute-pool ${COMPUTE_POOL_ID} --environment ${ENV_ID}

Then run a SQL query to get details about catalogs using the following query:

SELECT
  `CATALOG_ID`,
  `CATALOG_NAME`
FROM `INFORMATION_SCHEMA`.`INFORMATION_SCHEMA_CATALOG_NAME`;

The output will be similar to the following:

║CATALOG_ID CATALOG_NAME                                             
║env-083b5 env_name 

Links From Around the Web:

• A guide to floating point numbers
• Someone got GPT2 running in a… spreadsheet? Read the blog
• Human vs. Machine, have a look at the text-2-SQL leaderboard for LLMs (humans still reign supreme with 92.96% accuracy, GPT-4 comes second with 65.45%!) – BIRD-Bench

Upcoming Events:

Hybrid

• Kafka Summit Bangalore (May 2): Last call for KSB registration! While we wait for the first-ever Kafka Summit in Bangalore, you can check out the agenda and speakers for the event.
• Current 2024 (Sep 17-18): On your marks… ready, set, go! The agenda for Current 2024 has been released. Register here to attend.

In-person

• Real-time database discussion (Apr 25): Hosted at the Confluent offices in London, UK.
• Eventing platform development, and using Kafka for opentelemetry (Apr 25): Two talks at the Kafka meetup in Krakow, PL.
• Kafka 3.7 arrive, Kafka 4.0 en vue (May 14): Talk by Frédérik Rouleau of Confluent in Lyon, FR.
• Apache Kafka and Apache Pinot (May 15): Danica Fine and Victor Gamov join forces to teach you about data streaming in Munich, DE.
• Navigating consumer lag in Kafka (May 23): Alberto Cruz shows us how to reduce lag from the administrator and consumer viewpoints in Barcelona, ES.
• Data enrichment patterns with Apache Flink (June 5): Subham Rakshit from AWS will speak on enriching streaming data on June 5 at Community Over Code in Bratislava, Slovakia.

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see, or you want to submit your own resource for consideration, email us at devx_newsletter@confluent.io! To view the content without subscribing, visit the Confluent Developer website.