Apply Lambda Functions to Arrays and Maps

Apply Lambda Functions to Arrays and Maps

You've learned that ksqlDB can apply filters to each event in a stream based on values in the events—even filtering events in nested fields. But what about events in an ARRAY, or a MAP field? You can use lambda functions for this. Lambda functions allow you to use filters, transformations, and reductions.

Transform Function

A transform will change the key or value of every element in a MAP or ARRAY:

CREATE STREAM transformed 
  AS SELECT id,
  TRANSFORM(map_field,(k,v)=> UCASE(k), (k, v)=> (v * 2))
FROM stream1 EMIT CHANGES;

Note that for a MAP field, two lambda expressions are required: one for the key and one for the value. If no change is required for the key or the value, the lambda can just return the existing data.

Reduce Function

Reduce will take all of the elements from an ARRAY or MAP and produce a single result, based on the lambda expression:

CREATE STREAM reduced 
  AS SELECT name,
  REDUCE(scores,0,(s,x)=> (s+x)) AS total
FROM stream2 EMIT CHANGES;

For an ARRAY, the lambda takes two arguments: the ARRAY element and an accumulator. For a MAP, the lambda takes three arguments: the key, the value, and an accumulator.

Filter Function

The filter function takes an ARRAY or MAP and returns the same type, but filled with only the elements that meet the filter's criteria, which are given in the lambda expression.

CREATE STREAM filtered 
  AS SELECT id,
  FILTER(numbers,x => (x%2 = 0)) AS even_numbers
FROM stream3 EMIT CHANGES;