Kafka Monthly Digest – January 2020
From releases, to Kafka Improvement Proposals (KIPs), to blogs, check out what's going on in the Apache Kafka community for January 2020.
This is the 24th edition of the Kafka Monthly Digest. This series has now been running for 2 full years! We’ve covered all releases since Kafka 1.0.1, hundreds of KIPs, and many blog articles. When I started the series, I had no idea it would still be going 2 years later. I’m very grateful for all the amazing feedback I have received, and I hope to keep it alive for as long as possible. Thank you!
Without further ado, let’s cover what happened in the Apache Kafka community in January 2020.
The release process for 2.5.0 started when David Arthur volunteered to be the Release Manager on December 20, 2019. The release target is currently at the end of February. Feature freeze happened on January 29, and the next step is code freeze on February 12. As always, the release plan is available on the wiki.
Last month, the community submitted 14 KIPs (KIP-555 to KIP-568, KIP-554 was skipped) and these are the ones that caught my eye:
KIP-555: Deprecate Direct Zookeeper access in Kafka Administrative Tools: This KIP is another step towards removing the ZooKeeper dependency (KIP-500). All tools have been updated to not rely on ZooKeeper so this KIP proposes deprecating the
--zookeeperflag to prepare for its removal in the future.
KIP-558: Track the set of actively used topics by connectors in Kafka Connect: At the moment, there is no easy way to quickly know which topics a Connector is using. This information can be useful for several reasons including debugging applications and auditing purposes. This KIP’s goal is to track that information using the Connect status topic and expose it via the Connect REST API.
- KIP-559: Make the Kafka Protocol Friendlier with L7 Proxies: As service proxies, such as Envoy, are gaining in popularity, it’s important to ensure Kafka works well alongside them. This KIP aims at making the Kafka protocol friendlier with service proxies by allowing all requests and responses to be parsed without ambiguity and independently.
- KIP-560: Auto infer external topic partitions in stream reset tool: When resetting a Kafka Streams application using the
kafka-streams-application-reset.shtool, the user has to specify all input and intermediate topics. For complex topologies, this can be dozens of topics. This KIP introduces a new flag
--reset-all-external-topicsto automatically detect these topics, using the consumer group information of the Streams application, and reset them all.
Get started with Kafka
IBM Event Streams for Cloud is Apache Kafka-as-a-Service for IBM Cloud. Get started with IBM Event Streams today.