From Crypto Streams to AI-Powered Predictions

What if you could see the market move before it happens?

In this hands-on workshop, you’ll harness the power of Confluent Cloud - the fully managed data streaming platform built on Apache Kafka®, Apache Flink®, and Apache Iceberg® - to build a live crypto-streaming pipeline that ingests, processes, stores, and predicts real-time data.

What You’ll Learn

• How to set up and manage a Kafka cluster in Confluent Cloud
• Build and deploy Flink SQL jobs for real-time analytics
• Convert streams to query-ready Iceberg tables with Tableflow
• Run analytics in DuckDB
• Apply AI/ML forecasting directly inside your Flink pipeline

What You’ll Build

• Stream live crypto data into Kafka using Kafka Connect
• Transform and enrich data in motion using Flink’s streaming queries
• “Freeze” those flowing insights into Iceberg tables via Tableflow
• Query and analyze it all in DuckDB
• Use Flink AI to forecast price trends

Workshop Overview

In this 2-hour hands-on workshop, you'll build an end-to-end streaming analytics pipeline that captures live cryptocurrency prices, processes them in real-time, and uses AI to forecast the future.

You will start by ingesting a live data feed of crypto data (courtesy of Coingecko Rest API) into Apache Kafka using Kafka Connect. Then tame that chaos with Apache Flink's stream processing superpowers. Next, we'll "freeze" those streams into queryable Apache Iceberg tables using Tableflow. Finally, we'll try to predict the future by using Flink's built-in AI capabilities to analyze historical patterns and forecast where prices might head next. No prior experience with Kafka, Flink, or Iceberg required! Just bring your curiosity and a laptop!

Prerequisites: Before you arrive

To make sure you can get hands on during this workshop, please make sure the following are installed on your system!

1. GitHub Account

• Zero Install (Recommended): Use GitHub Codespaces or open in Dev Container - everything pre-installed!

2. Local Setup: Install the tools below on your machine (takes ~10 minutes)

• VSCode with Confluent Extension: For accessing Confluent Cloud resources.
• Confluent CLI: To interact with Kafka clusters and topics.
• Install DuckDB: For querying Tableflow Iceberg tables.

3. Correctly setting up your Confluent Cloud account

This step is optional, as we will go through how to set up Confluent Cloud during the event. However, if you want to get ahead, make sure you sign up, as so, to make sure you don't have to input your Credit Card::

Use the code 'CONFLUENTDEV1' when you reach the payment methods window after signing up for Confluent Cloud via this link.

Workshop Segments and Features Covered

Segment	Duration	Features Covered	Objective
Introduction	15 min	Kafka, Flink, Tableflow Overview	Understand event-driven architecture
Setting Up Confluent Cloud	15 min	Kafka Cluster Creation	Set up a managed Kafka cluster
Kafka Hands-On	30 min	Kafka Topics, Producers, Consumers	Stream data with Kafka
Flink Hands-On	45 min	Flink Stream Processing	Process data in real-time
Tableflow Hands-On	30 min	Tableflow, Iceberg, DuckDB	Materialize and query analytics-ready data
Wrap-Up and Q&A	15 min	All features	Summarize and address questions