Kafka Monthly Digest – September 2020
See what's going on in the Kafka community this month
In this 32nd edition of the Kafka Monthly Digest, I’ll cover what happened in the Apache Kafka community in September 2020.
For last month’s digest, see Kafka Monthly Digest: August 2020.
2.7.0: On August 14, Bill Bejeck volunteered to run the 2.7.0 release. KIP freeze happened on September 30. The next milestone is feature freeze, and it is expected for October 7. The final release is planned for early November. You can check the release plan on the wiki for all the details.
Last month, the community submitted 7 KIPs (KIP-668 to KIP-674), and these are the ones that caught my eye.
KIP-670: Add ConsumerGroupCommand to delete static members. Since Kafka 2.4.0, the Admin API enables removing static members from a consumer group. This KIP proposes adding this functionality in the
kafka-consumer-group.shtool as this script is often used by administrators.
KIP-671: Add method to Shutdown entire Streams Application. Kafka Streams applications are by design working on never ending streams of data. However, it can still be desired sometimes to shut them down. At the moment, the Streams API does not offer a way to gracefully shutdown an application. This KIP aims at addressing this limitation by providing a new method
KIP-673: Emit JSONs with new auto-generated schema. When investigating complex issues, it’s often interesting to log all requests/responses that brokers handle. You can do this by setting the logger level of
DEBUG. Today, log lines are emitted in a custom JSON-like format. The goal of this KIP is to switch the format to standard JSON to allow loading and parsing by external tools.
In this section, I will cover releases of some community projects. This only includes projects that are Open Source.
- Azkarra Streams 0.8. Azkarra is a framework to help build streaming microservices using Kafka Streams. This release adds support for Server Sent Events to directly send updates to external systems. It also contains a number of fixes and improvements, and it now uses Kafka Streams 2.6.
- Eliminating Task Processing Outages by Replacing RabbitMQ with Apache Kafka Without Downtime
- Auto-creating Debezium Change Data Topics
- How to Use Kafka, Schema Registry and Avro with Quarkus
Get started with Kafka
IBM Event Streams for Cloud is Apache Kafka-as-a-Service for IBM Cloud. Get started with IBM Event Streams today.