Event Sourcing

Thinking in Events

In this course, you'll learn about how events can be used as the storage model for your applications, whether they're individual applications or an ecosystem of microservices. This first module introduces a way of thinking about system design that is likely different from the more traditional one you are used to. You’ll learn about event modeling using data at rest and data in motion techniques: event sourcing, CQRS, and event streaming.

After you finish this course, you can learn more about its core ideas in the O'Reilly publication Designing Event-Driven Systems.

The Real World Modelled in Software

When you design and build a software system, you're creating a model that corresponds to a real-world system, whether it's for booking taxis, buying hightops, or completing financial transactions:

To build your model, you need some kind of coordinate system that you can use to map the real-world system to its software equivalent, analogous to the way that science maps the outside world using mathematics.

In mathematics, the two dominant systems are polar coordinates and rectangular, or Cartesian coordinates. With polar coordinates, you model a position by an angle and a distance, much like how a fighter pilot calls out a bogey as being "a half-mile up at 4 o'clock." Conversely, with Cartesian coordinates, you model a position using two perpendicular dimensions, similar to making a point on a map using longitude and latitude.

All future problems you might face can in fact be solved using either coordinate system, but some problems are a much better fit for one than for the other. For example, modeling the motion of planets around the sun is much simpler using the polar coordinates system—the math just works out better.

State-Based vs. Event-Based Systems

Similarly, software has two coordinate systems: state-based and event-based. A state-based model of a taxicab system uses a database and synchronous network calls to coordinate state in the system. An event-based model, however, uses continuous and asynchronous streams of immutable events. As with mathematics, you can solve nearly any problem you might face with either software system, but some problems are better solved with one instead of the other.

Illustrating Event-Based vs. State-Based Systems with Chess

Modeling the current state of a chess game is a problem that can help to demonstrate the differences between the state-based and event-based approaches. To describe a point in a chess game using a state-based approach, you would need to go through all of the chess pieces one by one, and record their positions as X and Y coordinates:

In this approach, you are essentially taking a snapshot of the board that can be saved for later. This is exactly how state-based modelling works using a database: you record the current position of all of the actors in the system, and store those positions in a database table.

Event modeling is quite different when applied to a chess match. To reach the current state of the board, rather than taking a snapshot, you would replay each move from the start to the present, one at a time:

You can recreate any point in the game using this method. Each chess move is an event, a transition from one state to the next. The event doesn't need to contain information about the other pieces on the board (although it could), but unlike with the state-based approach, you need the entire stream of events to derive the current position of all of the pieces on the board. (Note that this process can be accomplished with a traditional database approach, but it's far less natural to do so.)

Generally, events and transitions work best when you're modeling how something evolves over time. Additionally, the event-modeling approach is better suited to today's larger systems than the state-based approach, since they tend to include a greater number of separate parts.

Setting Up Confluent Cloud

Later in the course, you'll get to try out event sourcing yourself, with a client of your choice and ksqlDB. If you provision your cluster now, you can be sure that it will be up and running when you reach the exercise.

You can get extra free cloud time using the promo code EVENTS101. To get started, visit https://cnfl.io/confluent-cloud (details) and click Try Free. Enter your name, email address, and password. You will use these later to log in to Confluent Cloud. Select the Start Free button and watch your inbox for a confirmation email to continue.

The link in your confirmation email will lead you to the next step for creating a Kafka cluster, where you should choose a Basic cluster. Click Begin Configuration to choose your preferred cloud provider, region, and availability zone and click Continue. Review your selections, give your cluster a name, and then click Launch cluster. This might take a few minutes.

Finally, from your new cluster, create a new ksqlDB application using the Global Access control option along with a size of 4 Confluent Streaming Units (CSUs), and name the application shopping_cart_database. Starting the process of provisioning the ksqlDB application now will ensure that it is ready for hands-on modules later in the course.

Note that you should be careful to delete your cluster and ksqlDB application when you are finished with them or if you aren't using them for a long period of time. Instructions for deleting are given in Hands-On: Trying Out Event Sourcing.