In this post I will summarize the latest work being done in the github organization IBMStreams. As of today there are 16 toolkit repositories within the IBMStreams organization. There is a samples repository, 2 demo repositories and a utilities reposository.  This has exceeded our initial expectations and continues to grow.  Get involved and contribute, try out the toolkits or one of the samples, we need people to use these and provide feedback. If you find any problems open an issue. If you have ideas for samples, operators or new toolkits open an issue in the administration repository.  If there are enhancements to an existing repository you want, open an issue against that repository.

Toolkit Repositories

New integrations

The following are the latest repositories that provide new integrations for Streams:

Updated Toolkits

These repositories were based on existing function in Streams and have been updated and extended.

  • streamsx.inet
    The website for this toolkit is here. This contains the SPLDoc for the Internet toolkit. Additions to the toolkit include:

    • com.ibm.streamsx.inet.http — Operators that interact with external HTTP servers.
    • com.ibm.streamsx.inet.rest — Operators that embed an HTTP server to provide REST style API to streaming data
    • com.ibm.streamsx.inet.wsserver — Operators that embed a WebSocket server to expose streaming data as WebSocket messages
  • streamsx.hdfs
    This repository contains the operators that allow Streams to interact with an Hadoop distributed file system. These were migrated from the Streams BigData toolkit and updated. There are some samples included in the repository that demonstrate using the operators.
  • streamsx.messaging
    This has been updated to add support for Apache Kafka. Updates have been made to the MQTT, JMS and XMS operators.¬† Looking for someone to add support for RabbitMQ — if you can help out go here.¬† Do you need support for Kestrel?¬† If yes, go here and indicated you need this support.

New toolkits

These provide new and improved capabilities. Feel free to open issues on these suggesting new functions or operators. We are always looking for contributions to add more capability to these toolkits.

  • streamsx.json
    Provides parsing and formatting for JSON data.
  • streamsx.bytes
    Provides operators and functions for parsing and formatting raw data.
  • streamsx.datetime
    Functions for getting the current time as well as conversion functions.
  • streamsx.math
    Contains various math based functions.

New toolkit repositories — looking for contributors

The following have been created and will have content coming soon.  We need contributors to work on these toolkits.

Samples and Demos

There is a generic samples repository that currently contains 4 samples. This repository will be continually added to as new samples and examples are created. We are expecting to migrate and consolidate some existing samples into this repository over the summer months. The current samples are:

  • streamsx.demo.accelerometer – A collection of demos streaming smartphones’ accelerometers / gyroscope data to InfoSphere Streams applications.
  • streamsx.demo.logwatch – The purpose of the LogWatch family of applications is to teach concepts in SPL and Streams through a self-contained, small application with a well-defined problem statement.

Benchmarks

The benchmark repository contains benchmarks used in the comparison of Streams and Storm.

Administration

The administration repository is used to manage IBMStreams on github. There is where new proposal are made, etc… If you want to contribute, have a proposal this repository is where to start.

In future posts we will provide details about the specific toolkits. In the meantime, please download the toolkits and try them out.

Join The Discussion