Storing Data as Events

Storing Data as Events

This module introduces event sourcing and shows how it shapes the events you are collecting in your application.

Classic CRUD: Create, Read, Update, and Delete

Imagine an e-commerce application. A user, Sanjana, has added a t-shirt, pants, and a hat to her shopping cart. The cart is stored in a database table, which looks exactly like the cart at this particular moment: it has rows and columns, and it holds exactly 1 pair of pants, 1 t-shirt, and 1 hat:

Adding a completely new item would add a new row to the screen and a new row to the database table. But if Sanjana adds more than one of an already-listed item, the item count simply increases on the screen and is saved to the database. This is a classic CRUD approach: you can create, read, update, or delete any item in your table. (You will find this quite familiar if you've used databases in the past.)

Event Sourcing: Create and Read Only

When event sourcing is applied, the same goal is achieved—the shopping cart is safely stored to disk—but the process is quite different. As with CRUD, you can create and read values, but unlike CRUD, you never update a value and you never delete a value; these two destructive operations are simply not allowed with event sourcing.

Every action a user makes is preserved forever. So adding a t-shirt to the shopping cart is one event, adding another is a second event, and removing one is a third event. Checking out is also an event. As the events accumulate, they create a timeline of the user's activity, a kind of customer journey showing exactly what the customer did. In fact, this lack of destructive options makes event sourcing systems more like the version control systems in which you store code.

Getting Current State

A stream of events by itself, however, doesn't reflect current state. Say you need to determine how many pairs of pants are in the shopping cart right now. As you can see in the diagram below, that information is spread over multiple events—two "add" events and one "remove" event, each created at different times. Event sourcing systems solve this problem by reading all of the events into the client and then performing a computation that derives the current state. So getting the current pants count would involve a chronological reduce, i.e., the image on the left would be transformed into the one on the right, letting you display the current contents of the cart.

Event Sourcing Preserves Everything

Notice how the event-based view on the left side of the diagram above has quite a bit more information than the table view on the right. The transformation is unidirectional: you can go from the events view to the table view, but not vice versa, because information is lost as you move from left to right.

This is exactly why event sourcing is so powerful: it retains extra data about what has happened in the world, data that the CRUD approach simply throws away.

Event Sourcing in Practice

Event sourcing with our cart example is practically accomplished as follows:

• Events are stored in a table in a database, or alternatively in Apache Kafka; basically, you create a table and append events in the order that they occur.
• When you need to query for the latest state of the cart, you run a query that returns specific events, most likely aggregated by customer ID or session ID.
• You perform a chronological reduce to filter the events that are relevant for the view you'd like to serve, so a stream of five events is turned into the three rows of a more traditional database table: