Dimension 4: Modeling As Discrete vs. Continuous Event Flows
Adam Bellemare
Staff Technologist, Office of the CTO (Presenter)
The final major dimension of event design pertains to the relationships between events, and how they’re used and interpreted by the consumer.
A Discrete Event Flow
A discrete event flow drives state changes within the consumer application’s state machine, and results in either the completion, or the abandonment, of the consumer’s workflow. For example, an order is eventually shipped, a trade is eventually booked, and a taxi-cab rider eventually makes it their destination. Discrete event flows are composed not only of the input event streams, but also the state machine logic of the consumer that processes and completes the workflow.
A Continuous Event Flow
In contrast, a continuous event flow is most commonly represented as a series of independent events, such as measurements taken at a point in time, e.g. compute resource metrics, sensor measurements, and periodic reporting from internet of things devices. Continuous event flows are often simpler than discrete event flows, as they do not necessarily drive any complex state changes in the consumer application. Instead, they’re often (but not exclusively) used for generating analytics, measuring operational performance, and creating aggregates and summaries for other applications to use.
Let’s take a closer look at both of these with some examples.
Discrete Event Consumer Workflow
In this example, an ecommerce flow progresses through several states.
- The work starts when an order is received, and remains in the ORDERED state
- Upon receiving a payment for that order, the state machine can then move to the PAID state
- And lastly, a shipping event will transition the state machine for that entity to the final state of SHIPPED
This discrete event flow requires three distinct events, to be received and processed in a specific sequence, to get an order paid for and shipped out to the customer. The state machine of this event flow is tightly coupled to not only the structure of the events, but also the temporal order that they’re received and processed.
A discrete event flow may also contain multiple start or end states. The event flow designers may decide that it’s reasonable for the intermediate stages to time out after a period of inactivity - for example, an order that is not paid for within 30 days is simply cancelled, terminating the state machine for that specific order.
The termination of a state machine is also often the cause of its own new discrete event flow - issuing either an Order_Shipped event or an Order_Cancelled event is itself the conclusion of one event flow, but could be the start of a new event flow for notifying the customer of their incoming order, or sending them an apology email for the cancellation.
Discrete workflows have well-specified steps that lead to a final state of completion. Parallelization is important, as the consumer workflow often processes multiple independent entities simultaneously. They must make sure that they’re correctly resolving the progression of each entity through the workflow. After all, we wouldn’t want someone’s order payment going to someone else’s order. Businesses are full of discrete workflows. Some remain within a single application’s boundary, while some may span multiple applications and multiple event streams.
But what about continuous event flows? How do these work in practice?
Operational Metrics
Continuous workflows can be exemplified by streams of metric events. For example, CPU and disk metric usage are simply quick snapshots of resource usage at a point in time. In the case of metrics generated by sensors, it’s possible that a damaged or malfunctioning sensor may miss publishing some event data. Most continuous use cases are not significantly impacted by missing or duplicate events—metrics and other measurements may be slightly inaccurate, but as long as the bulk of the data is obtained computations based on the metrics will be fairly accurate. In contrast, discrete event flows may arrive at a very different termination point if even just a single event goes missing.
Alerting on Operational Metrics
Now, one of the neat things about metric-style event streams is that you can also use them to power discrete use cases. Consider how metric tracking systems are commonly paired with alerting capabilities. You can create little consumer state machines in the form of alerts that process the metric events and compute the present state. Changes in the state machine represent discrete occurrences, though they are derived entirely from a continuous stream of granular facts. In this example, there is an alert that notifies the disk_alert_mailing_list whenever server disk usage exceeds 80%. Depending on the configuration, the alert may simply trigger once and then remain silent until it drops back below 80%, or it may continually trigger every few minutes. However you do it, the result of this specific continuous event flow is that of a discrete event: a very distinct point-in-time notification that a certain state machine requirement is met, which can then go on to be input for other applications.
Continuous and discrete event flow use cases depend on both the event stream inputs and the business logic of the consumer. The very same event stream may be used by different consumers for both discrete and continuous purposes.
Let’s take a look at one more example, this time focusing on the consumer’s point of view, to see this dichotomy in action.
Sales Events
Let’s go back to the order of facts that we were looking at before. In this example, the accounts receivable consumer is reading order events and payment events to compute the amount of money that we’re still waiting to receive. This service needs to handle user account data, orders, and payments, and apply its own discrete state machine logic to ensure that it correctly computes account status. Additionally, the accounts receivable state machine needs to emit new facts when the account is 30 days, 60 days, and 90 days past due—discrete steps that are based on wall-clock time expiry.
The same order facts can be consumed by another team for a continuous use case—in this case, consuming the order events, composing them into Parquet files, and writing them to cloud storage for batch processing jobs.
And finally, an email notification service consumes order events and sends out a confirmation email, letting the customer know that their order has been received and will be processed shortly. Each email notification is completely independent of the other order facts, and any missed event processing, though inconvenient for the customer, will not get the email service into an incorrect state. Each of these three use cases reads from the very same orders fact stream, but treats the data as either continuous or discrete depending on the consumer’s unique business use case.
Discrete Event Flow Properties
Let's review some of the discrete event flow properties. Usually a discrete event flow will have multiple input event types and consumers will use a state machine to transition between multiple states. Discrete event flows typically cannot tolerate loss, missing even just a single event can get you into a bad state. Similarly, duplicate events can also cause problems. With discrete event flows, there's a significant relationship between the events, the event streams, and the consumer processing logic.
Continuous Event Flow Properties
And now let's look at the continuous event flow properties. Continuous event flows typically use just a single input event type. The consumer may or may not use a state machine, depending on the requirements, and consumers can usually tolerate lost or duplicate events, but it does depend on your use case.
Module Summary
Event streams cover a wide range of possibilities for both discrete and continuous use cases. The order shipping workflow use case illustrated the use of order, payment, and shipment events to compute the progress of each user's order through the system. Discrete changes in the consumer state machine map directly to the associated input event.
Alternatively, metric events, recording point in time measurements can be used for continuous use cases such as a visualization dashboard. However, they can also be used to power discrete state changes, such as when using threshold alerts.
When designing event streams, think about how your consumers may use the events to build either discrete or continuous workflows. Talk to your consumers to find out what their main use cases are so that you can ensure that their needs are met.
Use the promo code EVENTDESIGN101 & CONFLUENTDEV1 to get $25 of free Confluent Cloud usage and skip credit card entry.
Be the first to get updates and new content
We will only share developer content and updates, including notifications when new content is added. We will never send you sales emails. 🙂 By subscribing, you understand we will process your personal information in accordance with our Privacy Statement.
Dimension 4: Modeling As Discrete vs. Continuous Event Flows
Hi, I'm Adam from Confluent. And in this module, we're gonna be talking about modeling as discrete and continuous event flows. The final major dimension of event design pertains to the relationship between events and how they're used and interpreted by the consumer. A discrete event flow drives state changes within the consumer application state machine and results in either their completion or the abandonment of the consumer's workflow. For example, an order is eventually shipped, a trade is eventually booked, and a taxi cab rider eventually makes it to their destination. Discrete event flows are composed, not only of the input event streams, but also the state machine logic of the consumer that processes and completes the workflow. In contrast, a continuous event flow is most commonly represented as a series of independent events, such as measurements taken at a point in time, for example, compute resource metrics, sensor measurements, and periodic reporting from Internet of Thing devices. Continuous event flows are often simpler than discrete event flows as they do not necessarily drive any complex state changes in the consumer application. Instead, they're often, but not exclusively, used for generating analytics, measuring operational performance, and creating aggregates and summaries for other applications to use. Let's take a closer look at both of these with some examples. In this example, an eCommerce flow progresses through several states. The work starts when an order is received and remains in the ordered state. Upon receiving a payment for that order, the state machine can then move on to the paid state. And lastly, a shipping event will transition the state machine for that entity to the final state of shipped. This discrete event flow requires three distinct events to be received and processed in a specific sequence to get an order paid for and shipped out to the customer. The state machine of this event flow is tightly coupled to not only the structure of these events but also the temporal order that they're received in. A discrete event flow may also contain multiple start and end states. The event flow designers may decide that it's reasonable for the intermediate stages to time out after a period of inactivity. For example, an order that is not paid for within 30 days is simply canceled, terminating the state machine for that specific order. The termination of a state machine is also often the cause of its own new discrete event flow, issuing either a order shipped event or a order canceled event is itself the conclusion of one event flow, but could be the start of a new event flow for notifying the customer of their incoming order or sending them an apology email for the cancellation. Discrete workflows have well specified steps that lead to a final state of completion. Parallelization is important as the consumer workflow often processes multiple independent entities simultaneously. They must make sure that they're correctly resolving the progression of each entity through the workflow. After all, we wouldn't want someone's order payment going to pay someone else's order. Businesses are full of discrete event flows. Some remain within a single application's boundary, while some may span multiple applications and multiple event streams. But what about continuous event flows? How do those work in practice? Continuous event flows can be exemplified by streams of metric events. For example, CPU and disk metric usage are simply quick snapshots of that resource usage at that point in time. In the case of sensor generated metrics, it's possible that a damaged or malfunctioning sensor may result in a failure to publish some data. Continuous use cases tend to not be significantly impacted by missing or duplicate events. Metrics and other measurements may be slightly inaccurate, but as long as the bulk of the data is obtained, computations based off the metrics will be fairly accurate. In contrast, discrete event flows may arrive at a very different termination point if even just a single event goes missing. Now, one of the neat things about continuous event streams is that we can also use them to power discrete use cases. Consider how metric tracking systems are commonly paired with alerting capabilities. We can create little consumer state machines in the form of alerts that process the metric events and compute the present state. Changes in the state machine represent discrete occurrences, though they are derived entirely from a continuous stream of granular facts. In this example, we have an alert that notifies the @disk_alert_mailing_list whenever server disk_usage exceeds 80%. Depending on your configurations, the alert may simply trigger once and then remain silent or it may continually trigger every few minutes. However you do it, the result of this specific continuous event flow is that of a discrete event, a very distinct point in time notification that a certain state machine requirement is met. This event can then go on to be the input for other applications. Continuous and discrete event flow use cases depend on both the event stream inputs and the business logic of the consumer. The very same event stream may be used by different consumers for both discrete and continuous purposes. Let's take a look at one more example, this time focusing on the consumer's point of view to see this dichotomy in action. Let's go back to the order facts that we were looking at before. In this example, the accounts receivable consumer is reading order events and payment events to compute the amount of money that we're still waiting to receive. This service will need to handle user account data, orders, and payments, and apply its own discrete state machine logic to ensure that it correctly computes the account status. Additionally, the accounts receivable state machine will need to emit new facts when the account is 30 days, 60 days, and 90 days past due, discrete steps that are based on the wall clock expiry time. The same order facts can be consumed by another team for a continuous use case, in this case, consuming the order events, composing them into Parquet files, and writing them to cloud storage for batch processing jobs. And finally, an email notification service consumes order events and sends out a confirmation email, letting the customer know that their order has been received and will be processed shortly. Each email notification is completely independent of the other order facts, and any missed event processing, though inconvenient for the customer, will not get the email service into an incorrect state. Each of these three use cases reads from the very same orders facts stream but treats the data as either continuous or discrete, depending on the consumer's unique business case. Let's review some of the discrete event flow properties. Usually a discrete event flow will have multiple input event types and consumers will use a state machine to transition between multiple states. Discrete event flows typically cannot tolerate loss, missing even just a single event can get you into a bad state. Similarly, duplicate events can also cause problems. With discrete event flows, there's a significant relationship between the events, the event streams, and the consumer processing logic. And now let's look at the continuous event flow properties. Continuous event flows typically use just a single input event type. Now the consumer may or may not use a state machine, depending on the requirements, and consumers can usually tolerate lost or duplicate events, but it does depend on your use case. Event streams cover a wide range of possibilities for both discrete and continuous use cases. The order shipping workflow use case illustrated the use of order, payment, and shipment events to compute the progress of each user's order through the system. Discrete changes in the consumer state machine map directly to the associated input event. Alternatively, metric events recording point in time measurements can be used for continuous use cases such as a visualization dashboard. However, they can also be used to power discrete state changes, such as when using threshold alerts. When designing event streams, think about how your consumers may use the events to build either discrete or continuous workflows. Talk to your consumers to find out what their main use cases are so that you can ensure that their needs are met.