Streaming Database Systems

Streaming Database Systems for an "always-on" world, where data never rests

A streaming database flips a traditional database on its head. In a traditional database, when you write data into a table, it’s integrated into storage and nothing else happens, and you don't know what happens to your data between two queries invocations. But in an always-on world, you need to always know your latest data.

This is where streaming database systems come in. When data flows into a streaming database, it’s processed and immediately used to update the results of any registered queries. When applications want to read query results, they can look at all the ways that it has changed over time.

Confluent has spent many years working on a new kind of streaming database called ksqlDB. In the following blog series, we're exploring several key properties and fundamentals of streaming database systems and why they are so important for the database community.

If you have any questions or suggestions along the way, please head over to the Confluent Community Forum.

The Future of SQL: Databases Meet Stream Processing

To have the right abstractions for streaming database systems that let you query data in motion, it requires several enhancements to traditional SQL. These include a new STREAM data collection, new query types, and extended semantics particularly with regard to time. We take a deep dive into key technical areas and illustrate what Streaming SQL looks like at the example of ksqlDB.

Read More: (Article Coming Soon)

Design Principles and Guarantees of Streaming Databases

Modern streaming databases must be designed to provide correct results in real-time as a source of truth for long-lived queries, even with large-scale data streams. And when you're not managing your own infrastructure, like with a cloud database, you want the confidence that it auto-recovers from failures, guarantees exactly-once semantics, transparently handles out-of-order events, and provides consistent query results. Learn about the design principles of correctness and how ksqlDB gives you strong guarantees for your data.

Read More: (Article Coming Soon)

Data in Motion Never Rests: Managing Change without Downtime

In a streaming database, you run queries over data in motion, and these queries never terminate. Since the data never stops, the database needs to continue processing data even if you may need change the query itself. Learn how you can manage this change, without downtime.

Read More: (Article Coming Soon)