How to extract text using the IBMStreams Natural Language Processing (NLP) Toolkit RutaText operator?

Text extraction is one means to get insights to unstructured data like text or speech transformed into text. There are different methods to write text extraction rules. One of them is the UIMA Ruta language. The RutaText operator extracts data from streaming text according to predefined UIMA Ruta rules. It is part of the IBM Streams Natural Language Processing (NLP) Toolkit. This post describes how to use it. Continue reading How to extract text using the IBMStreams Natural Language Processing (NLP) Toolkit RutaText operator?

Rules based processing in real-time streaming applications

Introduction In IBM Streams 4.2, we have added support for authoring rules compatible with the Operational Decision Manager (ODM) product in Streams Studio, converting them to an SPL composite and using them for real-time analysis in IBM Streams. In this tutorial, we will develop a sample application that will demonstrate each of the steps and walk […] Continue reading Rules based processing in real-time streaming applications

IBM Streams Network Toolkit Overview

IBM Streams 4.2 brings many exciting new capabilities to the customer especially in the area of specialized toolkit. One of the new toolkits that was released along with IBM Streams 4.2 is the streamsx.network toolkit. The network toolkit enables SPL applications to analyze low level network packets such as parsing DHCP,DNS,Netflow,IPFIX messages, enriching IPV4 and […] Continue reading IBM Streams Network Toolkit Overview

Cybersecurity Toolkit – What’s New!

Cybersecurity Toolkit – What’s New! The Cybersecurity Toolkit provides operators that are capable of analyzing DNS response records. The operators in this toolkit use machine learning models to analyze DNS traffic and report on suspicious behaviour. The Cybersecurity Toolkit v2.0.0 includes new operators to further allow users to detect suspicious behaviour in their network as […] Continue reading Cybersecurity Toolkit – What’s New!

Hourly Moving Average — Streams makes it simple

In a recent discussion around using Streams, the following use case was considered problematic for an existing system. Given a set of devices that produce metrics, calculate the hourly moving average of the metric per device. In Streams this is very simple and a sample application took about 15 minutes to construct. Continue reading Hourly Moving Average — Streams makes it simple

Streaming Analytics Airport Sentiment Demo

This article describes a demo application that runs on the Bluemix Streaming Analytics Service in the cloud. It uses a Streams Application to read from the FAA website to get airport weather and delay information. It retrieves tweets from the IBM Insights for Twitter Bluemix service. It uses Streams text analytic capabilities to categorize the area the tweets are related to such as "baggage" or "maintenance". <a href="https://developer.ibm.com/streamsdev/wp-content/uploads/sites/15/2015/10/Main2.jpg"><img class="alignnone size-full wp-image-8454" src="https://developer.ibm.com/streamsdev/wp-content/uploads/sites/15/2015/10/Main2.jpg" alt="Main2" width="917" height="521" /></a> Continue reading Streaming Analytics Airport Sentiment Demo

Detect Active Threats in Real-time: Streams Cybersecurity Toolkit

Streams is an ideal platform for providing cybersecurity analytics, a new toolkit has been added to the Streams 4.1 release called the Cybersecurity Toolkit. This toolkit will provide the building blocks to enable developers and cybersecurity analysts to gain insight into their networks in real-time. Continue reading Detect Active Threats in Real-time: Streams Cybersecurity Toolkit

Introduction to the Spark MLLib Toolkit in IBM Streams V4.1

Ankit Pasricha is the team lead of the IBM Streams Toolkit development team. In his presentation, Ankit will introduce the new Spark MLLib Toolkit that is available in IBM Streams V4.1. This toolkit combines the power of Spark MLLib and the real-time streaming capabilities of Streams. Continue reading Introduction to the Spark MLLib Toolkit in IBM Streams V4.1

Predicting the Future in a Streams Application

Time series forecasting is a very broad subject. The ability to forecast future values is applicable in areas such as sales forecasting, stock market analysis and utilities forecasting (i.e. energy consumption). Forecasting can be a complicated subject as there many different forecasting algorithms, with each algorithm having certain properties that only makes it useful in specific circumstances. This article demonstrates how to easily introduce forecasting into an application using the AutoForecaster operator. Continue reading Predicting the Future in a Streams Application

Text Analytics To Go

In this article, I'm going to give you two simple applications that can serve as starting points for Text Analytics applications on Streams. The first example will use BigInsights Text Analytics to do normalization of terms. The second example will show how to tokenize, both with the simple SPL function, and using the more full-featured BigInsights Text Analytics. Continue reading Text Analytics To Go

Geospatial Toolkit Hands-on Lab Solution: Part 2

Application graph SPL code The code below is formatted as generated by the graphical editor. I only changed the following preferences (in SPL editor, right-click > Preferences…): In General > Editors > Text Editors: set Displayed tab width: to 3 and check Insert spaces for tabs. In InfoSphere Streams > SPL > Formatter: set Maximum line width (in characters): […] Continue reading Geospatial Toolkit Hands-on Lab Solution: Part 2

Geofence – Smart Marketing

In Streams 4.0, the geospatial toolkit introduced a new operator called Geofence.  The Geofence operator allows you to dynamically add or remove geographical regions of interest.  As entities move in and out of these regions, the operator will provide entry and exit events. This video demonstrates how you can use the Geofence operator to run […] Continue reading Geofence – Smart Marketing

Bandpass and bandstop filters using the DSPFilter operator

The DSPFilter operator implements a butterworth filter and can be used to isolate frequencies in a time series. For example, a low pass filter can be used to reject all frequencies above a certain point (this point is referred to as the cut-off frequency). Likewise, a high pass filter can be used to reject frequencies […] Continue reading Bandpass and bandstop filters using the DSPFilter operator

Example: Analyzing Weather Data using Windowing

My goal with this article is to provide a working example of how to build a Streams Application. In this example, I will be building a Streams application to calculate the average surface temperature and wind speed over a period of time. I will be using a CSV file that contains the temperature and wind […] Continue reading Example: Analyzing Weather Data using Windowing