In this 29th edition of the Kafka Monthly Digest, I’ll cover what happened in the Apache Kafka community in June 2020.

For last month’s digest, see Kafka Monthly Digest: May 2020.

Releases

2.6.0: This release entered code freeze on June 10. The community identified a handful of blocker JIRAs and is currently working on fixing them. For the full details, see the release plan in the wiki.

2.5.1: The release process of 2.5.1 continued. At the moment there are a few blocker JIRAs left. The release is now expected for the first half of July.

KIPs

Last month, the community submitted 15 KIPs (KIP-620 to KIP-634), and these KIPs are the ones that caught my eye.

• KIP-621: Deprecate and replace DescribeLogDirsResult.all() and .values(). When the describeLogDirs() Admin API was introduced, some of the classes were not properly exposed and thus are not part of the public API. This is the reason why some types are not visible in the DescribeLogDirsResult Javadocs. This KIP will address this issue and expose new classes as part of the public API.

• KIP-627: Expose Trogdor-specific JMX Metrics for Tasks and Agents. Trogdor is Kafka’s fault injection framework. This KIP proposes adding a few metrics to track the state of Trogdor tasks. This will ease the monitoring of Trogdor clusters as currently administrators have to rely on the REST API to retrieve that information.

• KIP-629: Use racially neutral terms in our codebase. The Kafka project and community strives to be inclusive. This KIP aims at replacing all racially charged terms in the project. For configuration settings, old names will still be supported but they will be deprecated and removed in the next major release.

• KIP-634: Complementary support for headers in Kafka Streams DSL. At the moment, Streams can only work on record headers via the Processor API. This KIP proposes extending the DSL grammar to allow accessing, setting, and removing headers on a KStream.

Community Releases

In this section, I will cover releases of some community projects. I cover only those community projects that are Open Source.

• node-rdkafka 2.9.0. This new release of node-rdkafka is now based on librdkafka 1.4.2 and offers access to the transactional producer API. The Typescript API has also been improved and as always a number of issues have been fixed.

• Debezium 1.2.0. This release is now based on Kafka 2.5.0 and can make use of KIP-585 for content-based event routing and filtering. In addition to improvements to connectors, Debezium 1.2.0 introduces a new runtime, the Debezium Server that is compatible with messaging infrastructure like Amazon Kinesis, Google Cloud Pub/Sub, or Apache Pulsar.

Blogs

• Kafka Streams, co-partitioning requirements illustrated
• How to build a real-time analytics platform using Kafka, ksqlDB, and ClickHouse

Get started with Kafka

IBM Event Streams for Cloud is Apache Kafka-as-a-Service for IBM Cloud. Get started with IBM Event Streams today.