This is the 37th edition of the Kafka Monthly Digest! In this edition, I’ll cover what happened in the Apache Kafka community in February 2021.

For last month’s digest, see Kafka Monthly Digest: January 2021.

Releases

2.8.0

The 2.8.0 minor release reached feature freeze on February 3, and according to the plan the 2.8 branch was cut on February 18. We are now in the stabilization period, and the first release candidate should be available in the next few days. For more details, see the release plan.

2.7.1

On February 18, I volunteered to run the 2.7.1 bugfix release. Since 2.7.0, 34 issues have been addressed including a few blockers. See the release plan for details about this release.

2.6.2

The release process of 2.6.2 continued. A few blocker issues in Connect (KAFKA-12326, KAFKA-12339, KAFKA-12340, KAFKA-12343) have delayed the first release candidate but they have now all been fixed. So we’re expecting RC0 in the next few days. All the details are in the release plan.

KIPs

Last month, the community submitted 6 KIPs (KIP-711 to KIP-716), and these are the ones that caught my eye.

• KIP-712: Shallow Mirroring. This KIP is for MirrorMaker1. When a record is mirrored, it is essentially consumed from a cluster and produced to another. This creates a number of overheads as bytes consumed are converted into ConsumerRecord objects (and potentially decompressed) before being converted back to bytes (and recompressed) by the producer. This KIP proposes adding a new consumer configuration, fetch.raw.bytes, to keep the data as ByteBuffer. The Produce API will also be updated to accept raw bytes as input.

• KIP-713: Validation of Enums in configuration. This KIP aims at improving error messages when configurations (client and server side) are not set correctly. The new messages should clearly state the issues and include valid values.

• KIP-716: Allow configuring the location of the offset-syncs topic with MirrorMaker2. In order to translate offsets between clusters, MirrorMaker2 uses an offset-syncs topic. When mirroring data from a source cluster to a target cluster, MirrorMaker2 currently creates the offset-syncs topic on the source cluster. This prevents granting only Read ACLs on the source cluster. This KIP proposes making the location of that topic configurable so it can be moved to the target cluster instead.

Community Releases

In this section, I will cover releases of some community projects. This only includes projects that are Open Source.

• Sarama 1.28.0. Sarama is a pure Golang Kafka client. This new release adds support for Kafka 2.7.0. Consumers can now take advantage of KIP-392 and fetch records from their closest replica. Finally, the roundrobin partition assignment strategy has been updated to match the Java client RoundRobinAssignor strategy.

Blogs

• Saga Orchestration for Microservices Using the Outbox Pattern
• Real-time monitoring of Formula 1 telemetry data on Kubernetes with Grafana, Apache Kafka, and Strimzi

Get started with Kafka

IBM Event Streams for Cloud is Apache Kafka-as-a-Service for IBM Cloud. Get started with IBM Event Streams today.