You have learned about ksqlDB's syntax and functionality, but it can be beneficial to get a sense of its internals.
When multiple ksqlDB servers are organized as a cluster, partitions are used as the basis for scaling the workload. Just like Apache Kafka consumers or Kafka Streams, ksqlDB can parallelize throughput by allocating work on partitions to different nodes. In Confluent Cloud, both your ksqlDB applications and Kafka clusters are fully managed. But you can also run ksqlDB in self-managed mode to run a custom user-defined function, or for other reasons. In this case, you can still take advantage of Confluent Cloud to manage the Kafka cluster used by your ksqlDB applications.
ksqlDB consumes and produces from Kafka topics, using SQL declarations to build and execute Kafka Streams topologies.
RocksDB is used internally to store state for tables and aggregations, with Kafka topics providing a changelog for RocksDB.
Multiple ksqlDB servers can be grouped together to form a cluster, for parallel processing and resilience.
Organizations will usually deploy multiple clusters of ksqlDB servers, forming a topology much more akin to application deployments than to a behemoth RDBMS.
We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.