Learning path: Building reactive Kafka applications

Apache Kafka is one of the most popular and commonly used tools for data streaming. It provides a unified, high-throughput, low-latency platform for handling real-time data feeds, which is particularly useful in cloud-native applications. The asynchronous and data-driven nature of Kafka is a great fit for the characteristics required to create reactive systems.

This learning path will introduce you to how you can best optimize Kafka-based applications to become reactive and highly responsive systems.

Objectives

Upon completion of this learning path, you will:

  • Gain a basic understanding of what reactive systems are
  • Gain a basic understanding of what Apache Kafka is and the types of communication it enables
  • Understand where Kafka fits within the architecture of reactive systems
  • Learn how best to use the configuration options available within Kafka to optimize it for reactive systems
  • Discover the potential benefits of re-architecting existing Kafka-based applications to be more reactive and responsive
  • Get hands on learning how to design and create reactive applications that use Kafka

Prerequisite knowledge

This is an introduction to using Kafka within reactive systems, so you will not need in-depth knowledge or experience, but a basic understanding of Kafka and Reactive would be helpful. However, we have also included useful resources within this learning path if you are not familiar with these technologies or terms.

Skill level

The skill level of this learning path is for a beginner.

Estimated time to complete

It will take you approximately 3 hours to complete this entire learning path.

Modules

This learning path includes two sets of information:

Using Kafka in a reactive application

  1. Advantages of event driven architecture (article) This article helps to lay out 6 key advantages gained by using an event-driven architecture, including decoupling of application components, increased resiliency, and providing real-time event streams for data science.

  2. Configuring Kafka for Reactive applications (article) This article details the configuration options that need to considered when using Kafka in a reactive application, including consumer offsets, retry policies, and so on. It also introduces some of the Kafka connectors in reactive frameworks that are available.

  3. Where does Kafka fit in a reactive architecture? (article) This article shows where Kafka fits within a reactive architecture style and how it enables the key characteristics of the reactive manifesto. It also helps to explain how reactive patterns can be enabled through Kafka-based applications like back-pressure and how these approaches can vary depending on the reactive framework or toolkit.

  4. Reacting to an event-driven world (video) This video introduces how Apache Kafka can be used and optimized for use within a reactive system. It introduces some of the open source reactive frameworks and the Kafka connectors they offer. A demo application is also shown, which has been converted into a reactive Kafka application.

Getting hands on with Kafka and Reactive technologies

  1. Experiences writing a reactive Kafka application (tutorial) This tutorial details the experiences and benefits gained from transforming a previously synchronous, non-reactive Kafka-based application into a more asynchronous, reactive Kafka-based application. It also details how to deploy the application yourself to have a play with these reactive API's and the demo Kafka app.

  2. Creating Reactive Java microservices with Open Liberty and MicroProfile (Open Liberty guide) In this guide, you will learn how to build reactive microservices that can send requests to other microservices, and asynchronously receive and process the responses. You will use an external messaging system (Apache Kafka) to handle the asynchronous messages that are sent and received between the microservices as streams of events. MicroProfile Reactive Messaging makes it easy to write and configure your application to send, receive, and process the events efficiently.

  3. Reactive and event streaming app development with Apache Kafka and Java (labs) This hands-on lab uses the same system/properties demo application used by the guide above; however, in this lab you'll be using our online environment and will be deploying this application to Red Hat OpenShift. There is also a second lab within this link that enables you to use the same virtual environment to try out and deploy a Quarkus-based reactive application to OpenShift.

Summary and next steps

By completing this learning path, you should now have a good understanding of how Apache Kafka can be used when building reactive applications and how it can be used in conjunction with other reactive tools.

To learn more about how to re-architect a Kafka application to be reactive, check out this series: Reactive in practice: A complete guide to event-driven systems development in Java.

To get more hands-on experience with Kafka, check out some of our Java Kafka tutorials: