We’ve recently updated Message Hub to run Kafka 0.10.2.1. This release added a number of new features along with many fixes and improvements. As always, we recommend users to migrate their clients to this new version.

SASL is now simpler to use and more flexible

Kafka 0.10.2.1 introduces a new client configuration, sasl.jaas.config, which can be used instead of a JAAS file. To connect to Message Hub, set it to:

sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="USERNAME" \
password="PASSWORD";

Where USERNAME and PASSWORD are the values from your Message Hub service Credentials page in Bluemix.
Also, using sasl.jaas.config, clients running in the same JVM can now use different credentials.
For the details, see http://kafka.apache.org/documentation/#security_sasl_plain_clientconfig

Topic Admin APIs

This release also introduced new APIs to create and delete topics. We’ve put some restrictions on the settings allowed when creating topics. Currently the only settings allowed are:
cleanup.policy: Can be set to delete (default), compact or delete,compact
retention.ms: Default is 24 hours. Minimum is 1 hour. Maximum is 30 days. It must be in muliples of hours.
Note:
If the cleanup policy is only compact, we automatically add delete but disable deletion based on time. Messages in the topic will be compacted up to 1GB before being deleted.

Kafka Streams support

The updated Streams library takes advantage of the topic APIs and now works out of the box with Message Hub. All you need to do is specify your SASL credentials via sasl.jaas.config or a JAAS file and set replication.factor to 3.

For example:

props.put(StreamsConfig.REPLICATION_FACTOR_CONFIG, "3");
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, BOOTSTRAP_SERVERS);
props.put("security.protocol","SASL_SSL");
props.put("sasl.mechanism","PLAIN");
props.put("ssl.protocol","TLSv1.2");
props.put("ssl.enabled.protocols","TLSv1.2");
props.put("sasl.jaas.config","org.apache.kafka.common.security.plain.PlainLoginModule required username=\"USERNAME\" password=\"PASSWORD\";");

Where BOOTSTRAP_SERVERS, USERNAME and PASSWORD are the values from your Message Hub service Credentials page in Bluemix.

If you have any questions, let us know in the comments or on StackOverflow using the message-hub tag.

1 comment on"MessageHub now runs Kafka 0.10.2.1 and supports Streams"

  1. Awesome, look forward to taking advantage of changes.

Join The Discussion

Your email address will not be published.