Messaging and Event Streaming Use Cases

 View Only

Messaging and Event Streaming Use Cases 

Wed March 04, 2020 02:47 PM

Messaging and Event Streaming Use Cases


Callum Jackson
Published on 18/05/2018
 / Updated on 22/05/2018

 

Within an organization there are many different types of messages flowing, for instance:

  • Request messages representing a task to be completed on an application or system.
  • Responsemessages that communicate the completion of a request.
  • Loggingthat informs us on the processing of a request
  • Monitoringthat informs us on the state of a system

Many additional examples could be included, but regardless of the number, they will fall into two categories. Messages that are focused on:

  • Operations: Messages that represent some current or future processing. For instance, a retailer may send an order request to their payment provider for processing. This message is targeted to the payment processing system with an explicit intent to be actioned.
  • Events: Messages that represent the state of the system. For instance, a retailer may emit an event for each order completion. These events represent facts and are emitted with no knowledge on how they will be used.

Considering these two categories, what capabilities does a solution require to support these categories of messaging? Although we could list hundreds of capabilities, we will focus on three key aspects, to focus our minds on the differences.

Operations

  • Transient Data – data is only stored until a consumer has processed the message, or it expires. The data does not need to be persisted longer than required, and it is actually beneficial from a system resource point of view that it does not.
  • Request / Reply– operations can be fire and forget, however often a request / reply interaction is desirable. Therefore, any solution should inherently support this pattern of interaction.
  • Targeted Reliable Delivery – when a message is sent, it is deliberately targetedto the entity that will process the request or receive the response. This targeting can use logical addressing and, in many instances, it is preferable. Messages should also be delivered using a suitable level of reliability. Different messages will have different levels, but often once-and-once-only, assured and transactional behavior is critical.

Events

  • Stream history: when retrieving events, often consumers are interested in the historical events, not only the most recent. There are many instances when this is valuable, such as retrieving the historical trends of a systems availability. Therefore, every event needs to be appended, and made available to consumers, and depending on the configuration, a certain number or volume of events will be stored prior to removal.
  • Scalable Consumption – multiple events are likely to be emitted for a single operation as it passes through the network. Therefore, the scaling requirements to handle events is likely to be orders of magnitude larger than that required to support operational messages. The scalability requirement is not only limited to the producers creating messages, but also to the consumption. A single event should be available to be consumed by many consumers with limited impact as the number of consumers is increased.
  • Immutable Data– when an event is emitted, it is a statement of fact, and can be considered immutable data. This enables consumers to rely on the assumption of consistent replay, and reduces the impact in replicating the data from a consistency point of view.

Messaging Technologies

Immediately you can see that the capabilities required to handle event and operation message types are significantly different. Therefore, it should not be surprising that different products exist for handling these categories of data. For instance, event streaming technology specializes in handling events, while brokered messaging focuses on reliably handling operation messages. This is not saying that either technology is limited to a particular use case, however it is useful to understand the use cases each technology has been specifically built to support in order to match the right solution to your requirements.

 

Entry Details

Statistics
0 Favorited
41 Views
1 Files
0 Shares
8 Downloads
Attachment(s)
pdf file
Messaging and Event Streaming Use Cases.pdf   192 KB   1 version
Uploaded - Wed March 04, 2020

Tags and Keywords

Related Entries and Links

No Related Resource entered.