Event Sourcing

Welcome to this course on Event Sourcing and Event Storage with Apache Kafka. In it, we'll learn about how events can be used as a storage model for the applications we build, be they individual applications or an ecosystem of microservices. In this first module, we're going to introduce a different way of thinking about system design than the more traditional one you may be used to. This course accompanies the book, "Designing Event Driven Systems." If you'd like to look further into the subject matter, there's a link below where you can download the book for free. When you design and build a software system, you're creating a model that describes a real-world system. The real-world system can be anything. Someone booking a taxi, buying a new pair of high-tops, completing a financial transaction. In fact, just about anything. To build your model, you need some kind of coordinate system, which you can use to map the real world into your software. This is analogous to the way science models the outside world using mathematics. In mathematics, the two dominant coordinate systems are polar coordinates and rectangular or Cartesian coordinates. In polar coordinates, you model a position by an angle and a distance, much like a fighter pilot might communicate a bogey being at a half a mile off four o'clock. In Cartesian coordinates, you model position using two perpendicular dimensions, like making a point on a map using longitude and latitude. All problems you face can be solved in either coordinate system, but some problems are a much better fit for one versus the other. For example, modeling the motion of the planets around the sun is much simpler using the polar coordinate system. The math just works out better. Similarly, the design of software system has two coordinate systems: state-based and event-based. For example, a state-based model of a taxi cab system would use a database and synchronous network calls to coordinate the state and the system. The event-based model would use continuous and asynchronous streams of immutable events. You can solve pretty much any problem you might face with either type of model, but just like in mathematics, some problems suit one modeling approach better than the other. In this course, you're gonna get familiar with a second form, modeling with events using data at rest and data in motion techniques, event sourcing, CQRS, event streaming, et cetera. A good place to start this process is to think about a chess game. Say we are mid game and we want to save the position of all the pieces. Chess is another good example of something that offers us two coordinate systems that achieve the same result, the positions of all the pieces, but achieved in wildly different ways. In the first method, you go through all the pieces on the chess board one by one and record their position as X and Y coordinates. So white king is at coordinator E1, white rook is at coordinate E2 and so on. We're essentially taking a kind of snapshot of the state of the board, which can be saved for later, and this is exactly analogous to how state-based modeling works using a database. We record the current position of all the actors in the system and store those positions in a database table. Event modeling is different. Rather than taking a snapshot of the whole board, we noticed that the game of chess has a known starting state, the starting position. From there, a number of moves are made. So if we replay all the moves from the starting position, we can recreate any point in the game. Each chess move is an event. A transition from one state to the next. The event doesn't need to contain information about the positions of other pieces on the board, although there's nothing to stop it from doing so, but unlike the state-based approach, the whole stream of events is required to derive the current position. So in summary, we have two coordinate systems for modeling real-world problems and software. One uses state and is coordinate based, the other uses events and is transition based. You are likely very familiar with the state-based approach if you're familiar with database development, but the event modeling approach, although less popular is better suited to many of today's problems, particularly as systems get larger and incorporate a greater number of separate parts. As a final point, we should ask ourselves this question, "Why do I need this other approach at all?" The answer lies in the chess game itself. If you use events, you will naturally map out the game as it is being played. This can be done with a database, but it's far less natural to do so. Transitions and events work best when you're modeling how something evolves over time. Later in the course, you'll get the opportunity to try out event sourcing yourself with a client of your choice in ksqlDB. If you start your cluster now, it'll ensure it's up and running, ready for you when you get to the exercise module. The promo code will provide you with an extra free cloud time. So make sure you enter it when you log in. To get started, go to the URL on the screen and click the Try Free button. Then, enter your name, email and password. This email and password will be used to log into Confluent Cloud later. So be sure to remember it. Or put it on a post-it note on your monitor. Okay, maybe don't do that. Click the Start Free button and watch your inbox for a confirmation email to continue. The link in your confirmation email will lead you to the next step where you can choose between a basic, standard or dedicated cluster. The associated cost are listed, but the startup amount freely provided to you will more than cover anything you need for this course. Click Begin Configuration to choose your preferred cloud provider, region and availability zone. Cost will vary with these choices, but they're clearly shown on the bottom of the screen. Continue to set up billing info. Here you'll see that you receive $200 of free usage each month for your first three months. Also by entering the promo code, EVENTS101, you receive an additional $101 of free usage to give you plenty of room to try out the things we'll be talking about. Click Review to get one last look at the choices you've made. Then launch your new cluster. While your cluster is provisioning, join me for the next module in this course. In that module, we'll dive right into event sourcing and explain what it is with a more practical and realistic example.