In this post, I will cover what happened in the Kafka community in May 2018.
The release process of 2.0.0 continues. KIP freeze happened on the 23rd of May. With over 40 KIPs targeted for 2.0.0, it already looks like it’s going to be a huge release. We’re now quickly approaching code freeze and the first release candidate is planned for mid-June. Finally, the full release plan for 2.0.0 is available on the wiki.
Once again, the community has been very active. Since last month, contributors have submitted 19 KIPs (KIP-290 to KIP-308). These are the ones that caught my eye (the top 3 should be in 2.0.0):
KIP-290: Support for Prefixed ACLs
Currently Kafka requires users to specify the exact resources names or the match all (*) special name when defining ACLs. This KIP aims to add support for some wildcards in ACLs to simplify the process of creating them.
KIP-297: Externalizing Secrets for Connect Configurations
This KIP’s goal is to enable Connect to integrate with secret management systems like Vault or AWS Secrets Manager. As a result, it will avoid having to store passwords in plaintext in Connect’s configuration files. This is an improvement that should simplify the deployment of Connect in secured environments.
KIP-305: Add Connect primitive number converters
Kafka has supported serializers and deserializers of primitive types for some time now. Connect however did not have those and users had to use the String, JSON or Avro converters. This KIP aims at fixing this gap by aligning Connect with Kafka.
KIP-306: Configuration for Delaying Response to Failed Client Authentication
When a user configures a client with invalid credentials, whether it’s malicious or simply a mistake, the client can retry authenticating against the cluster pretty quickly and authentication is an expensive operation. Therefore this KIP proposes adding a configuration to delay the closing of a connection if it has failed authentication so clients with bad credentials are slowed down and denial of service like situations are mitigated.
KIP-307: Allow to define custom processor names with KStreams DSL
This KIP is a quality of life improvement for Streams. By default, Streams generates names for Processors. In topologies with a large number of Processors it can be hard to quickly identify what each Processor is doing so this KIP would provide the ability to name them which can significantly ease understanding or debugging of a topology.
IBM Message Hub is Apache Kafka as a service for IBM Cloud. You can get started at https://console.bluemix.net/docs/services/MessageHub/index.html#messagehub.