You've learned that ksqlDB can apply filters to each event in a stream based on values in the events—even filtering events in nested fields. But what about events in an
ARRAY, or a
MAP field? You can use lambda functions for this. Lambda functions allow you to use filters, transformations, and reductions.
A transform will change the key or value of every element in a
CREATE STREAM transformed AS SELECT id, TRANSFORM(map_field,(k,v)=> UCASE(k), (k, v)=> (v * 2)) FROM stream1 EMIT CHANGES;
Note that for a
MAP field, two lambda expressions are required: one for the key and one for the value. If no change is required for the key or the value, the lambda can just return the existing data.
Reduce will take all of the elements from an
MAP and produce a single result, based on the lambda expression:
CREATE STREAM reduced AS SELECT name, REDUCE(scores,0,(s,x)=> (s+x)) AS total FROM stream2 EMIT CHANGES;
ARRAY, the lambda takes two arguments: the
ARRAY element and an accumulator. For a
MAP, the lambda takes three arguments: the key, the value, and an accumulator.
The filter function takes an
MAP and returns the same type, but filled with only the elements that meet the filter's criteria, which are given in the lambda expression.
CREATE STREAM filtered AS SELECT id, FILTER(numbers,x => (x%2 = 0)) AS even_numbers FROM stream3 EMIT CHANGES;
We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.