"Summarizing data for effective decision-making requires that the data be shown in a way that draws the reader’s attention to important trends, patterns and outliers with the least amount of effort."

Elijah Meeks discusses the principles behind the development of some new table features in the Apache Flink® workspaces in Confluent Cloud, including sparklines and a selector to filter by columns.

Also featured in today’s newsletter: a fresh new module from our event-driven microservices course by Wade Waldron, a graph featuring five Kafka use cases from Stanislav Kozlovski, and a Debezium explainer from Gunnar Morling.

Data Streaming Resources:

• Should My Data Be a Table or a Chart? Yes. Elijah Meeks breaks down some elements of data visualization design following his experience developing a new UI feature at Confluent.

• Learn why asynchronous events are key to designing an event-driven microservice in this new course module by Wade Waldron.

• Gunnar Morling explains Debezium in this new YouTube Short.

• Stanislav Kozlovski covers five Kafka uses cases in one graphic, check it out on LinkedIn.

• Smart Retry Mechanisms with Kafka: Daniel Sazdov relates redefining reliability in distributed systems.

• Give REST your best and bridge the gap between data streaming and data at rest… a new blog post explains how.

A Droplet From Stack Overflow:

Want to know more about how env.setParallelism() works for Flink task slot allocation within a job? Read here.

Got your own favorite Stack Overflow answer related to Flink or Kafka? Send it in to devx_newsletter@confluent.io !

Terminal Tip of the Week:

For Confluent Cloud teams, checking Flink statement status (Completed, Shipped, Failed) comes in very handy with confluent flink CLI commands.

This CLI command displays completed, failed, and stopped Flink SQL statuses. In this way, teams can report statements that have failed or stopped directly from the CLI, and notify development teams.

To test the Flink CLI, login to the Confluent CLI:

confluent login --save

Then, execute the confluent flink statement command with the desired cloud provider tags, the Flink SQL status, and the desired output, for example:

confluent flink statement list --cloud aws --region us-east-2 --compute-pool <COMPUTE_POOL_ID> --environment <ENV_ID> --output json --status completed

Users can also change the verbosity of the output by adding the -v tag (-v for warn, -vv for info, -vvv for debug, -vvvv for trace). Notice the important output fields such as status_detail, as well as the exact Flink SQL statement that was executed.

{
    "creation_date": "2024-05-28T11:47:45.376922Z",
    "name": "workspace-2024-02-27-<WORKSPACE_ID>",
    "statement": "create table `aws-conf-drc`.`flinksql-test`.`device_charge`( device_id STRING, charge_consumed DOUBLE ) WITH ( 'changelog.mode' = 'retract' );",
    "compute_pool": "<COMPUTE_POOL_ID>",
    "status": "COMPLETED",
    "status_detail": "Table 'device_charge' created"
  },
{
    "creation_date": "2024-05-22T11:44:15.603632Z",
    "name": "workspace-2024-02-27-<WORKSPACE_ID>",
    "statement": "CREATE TABLE shoe_order_customer_product( order_id INT, first_name STRING, last_name STRING, email STRING, brand STRING, `model` STRING, sale_price INT, rating DOUBLE )WITH ( 'changelog.mode' = 'retract' );",
    "compute_pool": "<COMPUTE_POOL_ID>",
    "status": "COMPLETED",
    "status_detail": "Table 'shoe_order_customer_product' created"
  }

Links From Around the Web:

• How To Learn Stuff Quickly - Josh W. Comeau’s take

• Why You Own an iPad and Still Can’t Draw

• A clock where the time is in the song title

Upcoming Events:

Hybrid

• Current 2024 (Sep 17-18): The agenda for Current 2024 has been released. Check it out! Register here to attend.

In-person

Singapore, Singapore (Jun 25): Learn how a robust data pipeline using Kafka and Flink was built at Rakuten

San Francisco Bay Area, USA (Jun 27): Learn about Kafka Connect, Kafka Streams, and get Otter pops!

Melbourne, Australia (Jun 27): The theme is seamless integration with Kafka and Kafka Connect

Sydney, Australia (Jun 27): An introduction to Large Language Models

Chennai, India (Jun 29): See how Fidelity Investments has used a governance framework to be resilient during unprecedented failure scenarios Gurgaon, Haryana, India (Jun 29): A comprehensive look at Kafka

Pune, India (Jun 29): Understand how Kafka powers high-throughput stock trading

By the way…

We hope you enjoyed our curated assortment of resources! If you’d like to provide feedback, suggest ideas for content you’d like to see, or you want to submit your own resource for consideration, email us at devx_newsletter@confluent.io!

If you’re viewing this newsletter online, know that we appreciate your readership and that you can get this newsletter delivered directly to your inbox by filling out the sign-up form on the left-hand side.