IBM Cloud Satellite: Build faster. Securely. Anywhere. Read more

IBM Developer Blog

Follow the latest happenings with IBM Developer and stay in the know.

From releases, to Kafka Improvement Proposals (KIPs), to blogs, check out what's going on in the Apache Kafka community for January 2021.


This is the 36th edition of the Kafka Monthly Digest! This series has now been running for 3 full years! Thanks for all the great feedback. Let’s keep going! If you are interested in a bit of history about this series, I’ve shared a few details on Twitter.

In this edition, I’ll cover what happened in the Apache Kafka community in January 2021.

For last month’s digest, see Kafka Monthly Digest: December 2020.

Releases

2.6.1

I released 2.6.1 on January 11. This bug fix version contains 41 fixes and improvements including 5 blocker JIRAs. You can check the release notes and the release plan for the full details.

2.8.0

On January 7, John Roesler volunteered to run the 2.8.0 release. KIP freeze happened on January 27. The next step will be feature freeze on February 3. As always, the release plan is available on the wiki.

2.6.2

A regression in Kafka Streams was discovered in Kafka 2.6.1. This issue, KAFKA-12190, causes crashes for users running on Windows. Sophie Blee-Goldman volunteered to run this new bugfix release. All the details are in the release plan.

KIPs

Last month, the community submitted 9 KIPs (KIP-702 to KIP-710), and these are the ones that caught my eye.

  • KIP-707: The future of KafkaFuture. The Admin API makes extensive use of KafkaFuture to provide sync and async functionalities. This class was designed when Java 7 was still supported. With Java 8, CompletableFuture became the new preferred API as it’s richer and many 3rd party tools or APIs support it directly. This KIP proposes extending KafkaFuture to be able to retrieve CompletableFuture objects.

  • KIP-708: Rack aware Kafka Streams with pluggable StandbyTask assignor. Kafka Streams applications can set up standby tasks to provide fast recovery times in case a task fails. This KIP aims at improving this mechanism even further by automatically distributing standby tasks across racks if possible.

  • KIP-710: Full support for distributed mode in dedicated MirrorMaker 2.0 clusters. MirrorMaker 2 runs in Kafka Connect. However, when it’s run in dedicated mode, it does not expose Kafka Connect’s REST API. The REST API is key for administrators to efficiently monitor and manage Kafka Connect deployments, but it’s also used by Connectors to dynamically react to topic and group changes. The goal of this KIP is to address these issues and expose all Kafka Connect capabilities when MirrorMaker 2 is run in dedicated mode.

Community Releases

In this section, I will cover releases of some community projects. This only includes projects that are Open Source.

  • Librdkafka 1.6.0. Librdkafka is a Kafka client in C/C++. This new version adds support for incremental rebalancing for consumers. The Admin client now has APIs to delete records, consumer groups and consumer group offsets. Finally, the producer can now use sticky partitioning to achieve higher throughput.
  • Debezium 1.4.0. Debezium is a popular Change Data Capture platform. 1.4.0 includes a brand new connector for Vitess and also includes improvements to the Oracle, PostgreSQL, and MySQL connectors.
  • strimzi-kafka-operator 0.21: Strimzi is a Kubernetes Operator for running Kafka. Kafka 2.7.0 is now supported, managing the configuration of Connectors has been simplified and overall most components have received improvements.

Blogs

Get started with Kafka

IBM Event Streams for Cloud is Apache Kafka-as-a-Service for IBM Cloud. Get started with IBM Event Streams today.