Streaming Database Systems
Streaming database systems for an "always-on" world, where data never rests.
A streaming database flips a traditional database on its head. In a traditional database, when you write data into a table, it’s integrated into storage and nothing else happens, and you don't know what happens to your data between two queries invocations. But in an always-on world, you need to always know your latest data.
This is where streaming database systems come in. When data flows into a streaming database, it’s processed and immediately used to update the results of any registered queries. When applications want to read query results, they can look at all the ways that it has changed over time.
Confluent has spent many years working on a new kind of streaming database called ksqlDB. In the following blog series, we're exploring several key properties and fundamentals of streaming database systems and why they are so important for the database community.
If you have any questions or suggestions along the way, please head over to the Confluent Community Forum.
Readings in Streaming Database Systems
The next important category of databases won't look like those that came before it. The demand for real-time data means working with data in motion instead of data at rest, and this calls for a streaming database. But what exactly does this look like? Let's talk about ksqlDB and the fundamental concepts behind it.
Read more: Readings in Streaming Database Systems
The Future of SQL: Databases Meet Stream Processing
To have the right abstractions for streaming database systems that let you query data in motion, it requires several enhancements to traditional SQL. These include a new STREAM data collection, new query types, and extended semantics particularly with regard to time. We take a deep dive into key technical areas and illustrate what Streaming SQL looks like at the example of ksqlDB.
Read more: The Future of SQL: Databases Meet Stream Processing
4 Key Design Principles and Guarantees of Streaming Databases
Modern streaming databases must be designed to provide correct results in real-time as a source of truth for long-lived queries, even with large-scale data streams. And when you're not managing your own infrastructure, like with a cloud database, you want the confidence that it auto-recovers from failures, guarantees exactly-once semantics, transparently handles out-of-order events, and provides consistent query results. Learn about the design principles of correctness and how ksqlDB gives you strong guarantees for your data.
Read more: 4 Key Design Principles and Guarantees of Streaming Databases
How Do You Change a Never-Ending Query?
In a streaming database, you run queries over data in motion, and these queries never terminate. Because the data never stops, the database needs to continue processing data even if you may need change the query itself. Learn how you can manage this change without downtime.
Read more: How Do You Change a Never-Ending Query?