Get Started Free
Tutorial

How to split a stream of events into substreams with Kafka Streams

How to split a stream of events into substreams with Kafka Streams

If you have a stream of events in a Kafka topic and wish to route those events to different topics based on data in the events, KStream.split and BranchedKStream.branch can be used to route the source topic's events.

For example, suppose that you have a Kafka topic representing appearances of an actor or actress in a film, with each event also containing the movie genre. The following topology definition will split the stream into three substreams: one containing drama events, one containing fantasy, and one containing events for all other genres:

    builder.stream(INPUT_TOPIC, Consumed.with(Serdes.String(), actingEventSerde))
            .split()
            .branch((key, appearance) -> "drama".equals(appearance.genre()),
                    Branched.withConsumer(ks -> ks.to(OUTPUT_TOPIC_DRAMA)))
            .branch((key, appearance) -> "fantasy".equals(appearance.genre()),
                    Branched.withConsumer(ks -> ks.to(OUTPUT_TOPIC_FANTASY)))
            .branch((key, appearance) -> true,
                    Branched.withConsumer(ks -> ks.to(OUTPUT_TOPIC_OTHER)));
Do you have questions or comments? Join us in the #confluent-developer community Slack channel to engage in discussions with the creators of this content.