Course: ksqlDB 101

Under the Covers of ksqlDB

2 min
Allison WaltherIntegration Architect (Course Presenter)
Robin MoffattStaff Developer Advocate (Course Author)

Under the Covers of ksqlDB

You have learned about ksqlDB's syntax and functionality, but it can be beneficial to get a sense of its internals.

When multiple ksqlDB servers are organized as a cluster, partitions are used as the basis for scaling the workload. Just like Apache Kafka consumers or Kafka Streams, ksqlDB can parallelize throughput by allocating work on partitions to different nodes. In Confluent Cloud, both your ksqlDB applications and Kafka clusters are fully managed. But you can also run ksqlDB in self-managed mode to run a custom user-defined function, or for other reasons. In this case, you can still take advantage of Confluent Cloud to manage the Kafka cluster used by your ksqlDB applications.

ksqldb-kafka-db

ksqlDB consumes and produces from Kafka topics, using SQL declarations to build and execute Kafka Streams topologies.

RocksDB is used internally to store state for tables and aggregations, with Kafka topics providing a changelog for RocksDB.

Multiple ksqlDB servers can be grouped together to form a cluster, for parallel processing and resilience.

kafka-cluster

Organizations will usually deploy multiple clusters of ksqlDB servers, forming a topology much more akin to application deployments than to a behemoth RDBMS.

Read more in the ksqlDB documentation and within the Confluent Developer course Inside ksqlDB. You can also find some deep-dives on ksqlDB functionality on the Confluent blog, including:

Use the promo code KSQLDB101 to get $101 of free Confluent Cloud usage

Be the first to get updates and new content

We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.