In this post, I will cover what happened in the Kafka community in August 2018.
No new releases this month.
Once again, the community has been very active and has submitted 22 KIPs (KIP-348 to KIP-369) since last month. These are the ones that caught my eye:
KIP-349: Priorities for Source Topics
It is common to have consumers subscribed to several topics and this KIP proposes adding a mechanism to specify the priority of each topic. That way, consumers will consume topics with the highest priority first.
KIP-352: Distinguish URPs caused by reassignment
Currently when reassigning partitions, they are counted as under replicated until the reassignment completes. This is inconvenient for administrators that often have alerts for under replicated partitions and it can also mask replication issues. This KIP’s goal is to change this behaviour and instead track partitions being reassigned under a new metric called “ReassignedCount”.
KIP-357: Add support to list ACLs per principal
Using the kafka-acl tool it is only possible to list all ACLs or to list them by resource. So this KIP’s objective is to allow listing them for specific users as this is a really common use case for administrators.
KIP-368: Allow SASL Connections to Periodically Re-Authenticate
At the moment, once a authenticated (SASL) session is established between a client and broker it is never revalidated. This makes it hard for administrators to block a user as any active session can just stay connected. This KIP targets specifically the OAUTH SASL mechanism and aims at providing a method to re-authenticate clients periodically.
KIP-369: Alternative Partitioner to Support “Always Round-Robin” Selection
Kafka Producers use a Partitioner to select the partition each message is sent to in case the user has not specified it explicitly. The default Partitioner has different behaviours depending if messages have keys or not. This KIP proposes adding a new Partitioner that always assigns partitions using round robin even for messages with keys.
IBM Message Hub is Apache Kafka as a service for IBM Cloud. You can get started at https://console.bluemix.net/docs/services/MessageHub/index.html#messagehub.