Get Started Free

Event Router

Event Streams may contain a subset of Events which need to be processed in isolation. For example, an inventory check system may be distributed across multiple physical systems, and the target system may depend on the category of the item being checked.

Problem

How can we isolate Events into a dedicated Event Stream based on an attribute of the Events?

Solution

event-router

Implementation

With the streaming database ksqlDB, we can continuously route Events to a different Event Stream. We use the CREATE STREAM syntax with an appropriate WHERE filter:

CREATE STREAM payments ...;

CREATE STREAM payments_france AS
    SELECT * FROM payments WHERE country = 'france';

CREATE STREAM payments_spain AS
    SELECT * FROM payments WHERE country = 'spain';

With the Apache Kafka® client library Kafka Streams, use a TopicNameExtractor to route Events to different Event Streams (called "topics" in Kafka). The TopicNameExtractor has one method to implement, extract(), which accepts three parameters:

  • The event key
  • The event value
  • The RecordContext, which provides access to headers, partitions, and other contextual information about the event

We can use any of the given parameters to generate and return the desired destination topic name for the given Event. Kafka Streams will complete the routing.

CountryTopicExtractor implements TopicNameExtractor<String, String> {
   String extract(String key, String value, RecordContext recordContext) {
      switch (value.country) {
        case "france":
          return "france-topic";
        case "spain":
          return "spain-topic";
      }
   }
}

KStream<String, String> myStream = builder.stream(...);
myStream.mapValues(..).to(new CountryTopicExtractor());

Considerations

  • Event Routers should not modify the Event itself, and instead should provide only proper routing to the desired destinations.
  • If an Event Router needs to attach additional information or context to an Event, consider using the Event Envelope pattern.

References

Confluent Cloud is a fully managed Apache Kafka service available on all three major clouds. Try it for free today.

Try it for free