Kafka Monthly Digest – July 2020
See what's going on in the Kafka community this month
In this 30th edition of the Kafka Monthly Digest, I’ll cover what happened in the Apache Kafka community in July 2020.
For last month’s digest, see Kafka Monthly Digest: June 2020.
2.6.0: Randall Hauch opened a vote on RC0 on July 14, but an issue related to KIP-546 was found. A few days later a new vote on RC1 started, but again a couple of blockers ( KAFKA-10287 and KAFKA-10306 ) were found. On July 28, the vote on RC2 started, and it is still on-going. If no other issues are found, 2.6.0 should be ready in the next few days.
2.5.1: The release process of 2.5.1 continued. On July 24, John Roesler started the vote on RC0, and it is currently still on-going.
Last month, the community submitted 16 KIPs (KIP-635 to KIP-650), and these are the ones that caught my eye.
KIP-636: Make RPC error codes and messages tagged fields. Since 2.4.0 (KIP-482), the Kafka protocol supports optional fields. This KIP proposes making error code fields optional in order to reduce the size of responses when no errors have occurred.
KIP-642: Dynamic quorum reassignment. This KIP is part of the effort to remove ZooKeeper (KIP-500). In post KIP-500 Kafka, a metadata quorum that consists of Kafka brokers will provide the distributed coordination required by Kafka. This KIP introduces a mechanism to dynamically resize the metadata quorum. It also describes the bootstrapping process that brokers will follow when starting up to form a cluster.
KIP-649: Dynamic Client Configuration. At the moment, there is no way to change the configuration of a Kafka client once it’s started. If a different configuration is needed, the current client has to be closed and a new one created. This KIP suggests a mechanism for updating selected settings at runtime reusing the APIs (
IncrementalAlterConfigs) to reconfigure brokers.
In this section, I will cover releases of some community projects. This only includes projects that are Open Source.
Strimzi 0.19.0. Strimzi is a Kafka Operator for Kubernetes. The scaling of Connect resources including MirrorMaker2 has been simplified. In addition, the
KafkaRebalanceresource has received a number of updates making it more configurable.
Librdkafka 1.5.0. Librdkafka is a third party client in C/C++. This new release does not include big new features but instead a lot of usability improvements, enhancements, and fixes. This includes better error reporting and improved handling of consumer subscriptions.
Due to Covid-19, Kafka Summit will be a virtual conference this year. It is happening on August 24 and 25, and you can register for free on the website. There are over 50 sessions covering a wide variety of topics, so whether you are a developer, an administrator, or a manager, or whether you are a Kafka expert or a newcomer, you’ll find something interesting.
- Implementing the Transactional Outbox Pattern with Debezium in Quarkus
- Improved Robustness and Usability of Exactly-Once Semantics in Apache Kafka
Get started with Kafka
IBM Event Streams for Cloud is Apache Kafka-as-a-Service for IBM Cloud. Get started with IBM Event Streams today.