In this 34th edition of the Kafka Monthly Digest, I’ll cover what happened in the Apache Kafka community in November 2020.

For last month’s digest, see Kafka Monthly Digest: October 2020.

Releases

2.7.0: The release process for the next minor version continued. A few regressions and blockers were found: (KAFKA-10669, KAFKA-10687, KAFKA-10755 and KAFKA-10758). These required a few Release Candidates. Once all issues were fixed, Bill Bejeck built RC3 on November 25 and started a vote. If you have time, test this Release Candidate and vote in the thread. All the details about this release are in the release plan.

2.6.1: The release of this bugfix version has been affected by the same issues as 2.7.0. After all fixes were back-ported to 2.6, I built RC2 on November 25 and like 2.7.0, a vote is currently in progress. The release plan on the wiki lists all changes in this bugfix.

KIPs

Last month, the community submitted 9 KIPs (KIP-680 to KIP-688), and these are the ones that caught my eye.

• KIP-681: Rename master key in delegation token feature. This KIP follows on from KIP-629 and will replace the last racially charged terms in Apache Kafka by neutral terms. Old names will still be supported but they will be deprecated and removed in the next major release.

• KIP-684 – Support mutual TLS authentication on SASL_SSL listeners. Today Kafka only supports Mutual Authentication (mTLS) on SSL listeners. This KIP proposes adding support for mTLS on SASL_SSL listeners too. Mutual Authentication enables brokers to authenticate users via their TLS certificates. When enabled on SASL_SSL listeners, users will be authenticated using both their TLS certificates and the selected SASL mechanism. This offers increased security which is often mandatory in highly secured or regulated environments.

• KIP-686: API to ensure Records policy on the broker. This KIP proposes adding an interface, RecordsPolicy, to validate records before brokers append them to topics. The main use case is for administrators to ensure all records on a topic have the similar format. For example, this could be the same schema or the same serialization. Such validation is helpful to avoid poison pills and enforce stronger data governance.

Community Releases

In this section, I will cover releases of some community projects. This only includes projects that are Open Source.

• kafkajs 1.15.0. Kafkajs is a Kafka client for Node.js. This latest release added support for static group membership for consumers as well as a number of features to the admin client, such as APIs that manage ACLs and support for the delete records API.

Blogs

• Analyzing historical and live data with ksqlDB and Elastic Cloud
• If You’re Using Kafka With Your Microservices, You’re Probably Handling Retries Wrong

Get started with Kafka

IBM Event Streams for Cloud is Apache Kafka-as-a-Service for IBM Cloud. Get started with IBM Event Streams today.