Kafka Monthly Digest – February 2020
See what's going on in the Kafka community this month
In this 25th edition of the Kafka Monthly Digest, I’ll cover what happened in the Apache Kafka community in February 2020.
In this edition, I am trying out a new section, Community Releases. If you have any feedback or would like a project to be covered, please contact me via Twitter.
For last month’s digest, see Kafka Monthly Digest: January 2020.
The release process for 2.5.0 continued and code freeze happened on February 12. A few issues — KAFKA-9601, KAFKA-9556 — were found and delayed the release a bit. Finally, on February 29, David Arthur published the first Release Candidate. The vote for RC0 is currently on-going. If you have time, please check the release and vote.
Last month, the community submitted 5 KIPs (KIP-569 to KIP-574) and these are the ones that caught my eye:
KIP-571: Add option to force remove members in StreamsResetter: In order to use the StreamsResetter tool for a Kafka Streams application, not only all Streams instances of the application have to be stopped but their sessions must also have expired. This KIP proposes adding a way to clear members to avoid having to wait for timeouts to expire before resetting a Streams application.
KIP-572: Improve timeouts and retries in Kafka Streams: The goal of this KIP is to improve the way timeouts and retries are handled in Streams applications. Currently Streams relies on the underlying clients (Producer, Consumer and Admin) default timeouts and retries logic. This can sometimes lead to tasks being blocked and it can be hard to troubleshoot failure scenarios. The proposal is to re-use the existing
retriessetting, apply it per task, and only stop processing once all retries have been used.
KIP-573: Enable TLSv1.3 by default: From Kafka 2.5, only TLSv1.2 will be enabled by default (KIP-553). Since support for TLSv1.3 is also added in 2.5, the KIP proposes adding it to the enabled SSL protocols, so
ssl.enabled.protocolswould default to
In this section, I will cover releases of some community projects. This only includes open source projects.
Debezium 1.1.0.Beta: Debezium is a popular Change Data Capture platform for Kafka. The 1.1 release will bring a number of new features such as Cloud Events support, improved handling for transaction boundaries and a brand new connector for IBM Db2.
Sarama 1.26: Sarama is a pure Golang Kafka client. It recently enabled support for
zsdtcompression as well as support for custom consumer assignment strategies.
- Kafka Topics: Pitfalls and insights
- Integration scenarios with Debezium and Apache Camel
- 99th percentile latency at scale with Apache Kafka
Get started with Kafka
IBM Event Streams for Cloud is Apache Kafka-as-a-Service for IBM Cloud. Get started with IBM Event Streams today.