Get Started Free
‹ Back to courses
course: ksqlDB 101

Push Queries and Pull Queries

2 min
Allison

Allison Walther

Integration Architect (Presenter)

Push Queries and Pull Queries

There are two ways of querying data in ksqlDB: push queries and pull queries.

  • Push queries are identified by the EMIT CHANGES clause. By running a push query, the client will receive a message for every change that occurs on the stream (that is, every new message).

  • Pull queries return the current state to the client, and then terminate. In that sense, they are much more akin to a SELECT statement executed on a regular RDBMS. They can only be used against ksqlDB tables with materialized state, that is, a table in which there is an aggregate function. Currently, tables declared against an existing Apache Kafka topic cannot be queried with a pull query (as of ksqlDB v0.15 / February 2021).

Applying this to the aggregate ORDERS_PER_HOUR_BY_MAKE table from Stateful Aggregations (Materialized Views), we can query it both ways. A pull query returns the current state of the aggregate:

pull-query-returns

New events will update the aggregate, but we’d need to execute the pull query again in order to see the new value:

aggregate-two

Compare this to a push query against the same table. Here, we receive a continuous stream of all changes to the aggregate:

KSQLDB PUSH QUERY

Use the promo code KSQLDB101 & CONFLUENTDEV1 to get $25 of free Confluent Cloud usage and skip credit card entry.

Get Started

Get started with Confluent Cloud

Try free

Total 28

  1. Introduction to ksqlDB
    2 min
  2. Interacting with ksqlDB
    2 min
  3. Hands On: Interacting with ksqlDB
    5 min
  4. Creating, Exporting, and Importing Data
    2 min
  5. Filtering with ksqlDB
    1 min
  6. Hands On: Filtering with ksqlDB
    1 min
  7. Lookups and Joins with ksqlDB
    2 min
  8. Hands On: Lookups and Joins with ksqlDB
    2 min
  9. Transforming Data with ksqlDB
    2 min
  10. Hands On: Transforming Data with ksqlDB
    2 min
  11. Flatten Nested Records with ksqlDB
    1 min
  12. Hands On: Flatten Nested Records with ksqlDB
    2 min
  13. Converting Data Formats with ksqlDB
    2 min
  14. Hands On: Converting Data Formats with ksqlDB
    2 min
  15. Merging Two Streams with ksqlDB
    1 min
  16. Hands On: Merging Two Streams with ksqlDB
    2 min
  17. Splitting Two Streams with ksqlDB
    2 min
  18. Hands On: Splitting Two Streams with ksqlDB
    2 min
  19. Streams and Tables
    2 min
  20. Stateful Aggregations (Materialized Views)
    2 min
  21. Hands On: Stateful Aggregations (Materialized Views)
    2 min
  22. Push Queries and Pull Queries
    2 min
  23. Hands On: Push Queries and Pull Queries
    2 min
  24. Under the Covers of ksqlDB
    2 min
  25. ksqlDB Code Lifecycle
    2 min
  26. Apply Lambda Functions to Arrays and Maps
    2 min
  27. Hands On: Apply Lambda Functions to Arrays and Maps
    2 min
  28. Monitoring ksqlDB
    1 min

Related Resources

Presentation

Inside ksqlDB

Docs

Queries

Docs

SELECT (Push Query)

Docs

SELECT (Pull Query)

Be the first to get updates and new content

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.

Push Queries and Pull Queries

Hi, I'm Allison Walther with Confluent. There are two types of queries in ksqlDB: push queries and pull queries. Let's take a look at them both now. You've already seen how we can create a table from a stream of events. For example, using the aggregations we talked about in the previous lesson. We can query that table to get the current state of the data for a given key. The result is returned and the query terminates, much like we'd expect from a relational database. in ksqlDB, this is referred to as a pull query. We can take that same query and add the keywords, "EMIT CHANGES," and it becomes a push query. This query will continue running and emit the results of the query for any changes that occur in the underlying stream. A push query will run until it is deliberately shut down. A table can be created from a stream of events. Think of it as a materialized view. So to recap, push queries are applicable to all stream and table objects and indicated with an EMIT CHANGES clause. Pull queries have some limitations currently. Really, they're only available against tables. So, that's it for pull and push queries. We're gonna move on to an exercise.