Transforming Data with ksqlDB

Transforming Data with ksqlDB

ksqlDB allows you to transform events in one stream and then send them to a new stream. For example, you may want to change a Unix timestamp to a human-readable format, or you may want to remove sensitive fields from a stream for broader use within a company.

ksqlDB can be used to perform many common transformations required when handling data, including:

• Changing data types (CAST)
• Reformatting date/time fields (TIMESTAMPTOSTRING)
• Changing field names (AS)
• Dropping fields
• Concatenating fields (CONCAT)

When creating a new stream from an existing one, you can pick and choose the fields you’d like to preserve in the new stream. You can also use functions to modify data or to create derived fields.

For example, here is how to convert the timestamps on your orders from the Unix epoch value to a more friendly data format:

There is really no limit to the transformations you can perform with ksqlDB.