How can I enrich an Event Stream or Table with additional context?
We can combine Events in a stream with a Table or another Event Stream by performing a join between the two. The join is based on a key shared by the original Event Stream and the other Event Stream or Table. We can also provide a window buffering mechanism based on timestamps, so that we can produce join results when Events from both Event Streams aren't immediately available. Another approach is to join an Event Stream and a Table that contains more static data, resulting in an enriched Event Stream.
CREATE STREAM ratings (MOVIE_ID INT KEY, rating DOUBLE) WITH (KAFKA_TOPIC='ratings');
We can then create a Table from another existing Kafka topic that changes less frequently. This Table serves as our reference data (similar to dimension tables in data warehouses).
CREATE TABLE movies (ID INT PRIMARY KEY, title VARCHAR, release_year INT) WITH (KAFKA_TOPIC='movies');
To create a stream of enriched Events, we perform a join between the Event Stream and the Table.
SELECT ratings.movie_id AS ID, title, release_year, rating FROM ratings LEFT JOIN movies ON ratings.movie_id = movies.id EMIT CHANGES;
We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.