This article demonstrates capturing real-time events from SmartPhones (iPhone, Android), injecting them in to InfoSphere Streams, and streaming the processed data out to web browsers where the data is rendered.
This demonstration can be extended to create applications that perform:
- live polling
- monitor your walking gait
- monitor the acceleration of building’s elevators
- verify¬† Galileo’s Tower of Pisa hypothesis – (voids SmartPhone’s warranty)
The effort involves locating,¬† configuring and accessing real-time data which are significant ‘barriers to demonstrate’.¬† The goal of this article is to remove the barriers by providing the components to stream live data.
The demonstration is depicted in the following diagram.
The steps are :
- SmartPhones’ accelerometers generates data.
- Events are received by¬† com.ibm.streamsx.inet.wserve on Streams toolkit.
- The toolkit’s¬† WebSocketInject() operator injects data into stream.
- Data is processed
- The toolkit’s WebSocketSend(), operator sends the processed stream data via WebSocket protocol.
- HTML5 receives events with the data from InfoSphere Streams.
¬† ¬†Software Download links :
- VMWare Quick Start Edition 3.2 : Streams execution environment that the application runs within, using version 3.2.1 will work as well.
- streamsx.inet : A toolkit that support web protocols.
- streamsx.demo.accelerometer :¬† The demonstration application that streams SmartPhone data.¬† se the ‘Download ZIP’ button in the lower right hand corner.
¬† Hardware :¬†
- I’ve run this on VMWare hosted on PC (Workstation) as well as Mac (Fusion)
- SmartPhone ¬†(sorry you’ve got to provide your own).¬† I’ve tried the application on the following hardware
- IPhone4 IOS7 : Safari or Chrome for Mobile browsers
- Nexus7/Android 4.4 ¬†: ¬†Chrome for Mobile
Components – provided
com.ibm.streamsx.inet.wsserve: Toolkit that implements WebSocket server protocol, within the streamsx.inet download.
com.ibm.streamsx.inet.rest : Toolkit that provides http access, located in the¬† streamsx.inet download.
basic.accelerometer : ReceiveToSend application provided in the streamsx.demo.accelerometer download.
ReceiveToSend.spl :¬† Streams application that utilizes¬†com.ibm.streamsx.inet.wsserve toolkit. The application receives¬†WebSocket messages in a XML format and transforms them into JSON¬†messages as they are sent out.
index.html :¬†Web page that acts as entry point to collect and display data.¬†The web pages utilize the com.ibm.streamsx.inet.rest toolkit.
data/html/displayWebSocket.html :¬†This is accessed from index.html. It displays the current accelerometer and gyroscopic data. Selecting the ‘Blue’ button¬†will connect to server and start transmitting data (after putting up two alerts). It transmits XML messages to the ReceiveToSend.spl¬†via WebSocket protocol.
data/html/sinkWebSocket.html :¬†This is accessed from the index.html page, receives WebSocket messages from the RecieveToSend.spl. The data is¬†displayed in a table and scrolling graph.
Setup : Configuring VM’s network adapter
The Streams application gets data from a SmartPhone and displays a graphic on a web page. Both of these facilities need the IP address of the Streams instance running within the VM. Before you install the application you’ll¬† need to change the network configuration and get the IP address. If the network is not setup or the IP address is invalid, this demonstration will fail.
When using a VM for development by default you share the IP address with the host machine. Since the SmartPhone needs to communicate with the wsserve toolkit you’ll need to expose an IP address for the VM.
Configure the VM image so that the IP address is exposed on the network by setting the VM Network connection to ‘Bridged:..” Below is a screen capture of the settings VMWorkStation (win) and VMWare’s Fusion (mac) products.
Setup when hosted on Windows
Setup when hosted on Mac
NOTE : I have found that rebooting the host VM is necessary after making these changes. I never really ‘grok’ how the physical and virtual hardware interact.
NOTE:¬†This document assume that you are using Streams on the ‘vanilla’ QSE, if you are not or¬†encounter communications issues you will want to refer to ‘Appendix – Firewall configuration’.
Get the host’s IP address.
Get the IP address of where the Streams application is running. Use the ‘/sbin/ifconfig’ command to retrieve the IP address. Below is and example of running the command from the terminal.
The IP address of the system that is hosting the application in this example is 184.108.40.206, we’ll use this value to connect from the SmartPhone and desktop browser to the Streams application.
NOTE: If you have a ‘ping’ application on your SmartPhone, you should attempt to ping the host IP address to verify connectivity.
NOTE : To quote one of the initial testers, “The biggest stumbling block is the network, WIFI, VM firewalls … many possible roadblocks since everyone’s environment is different”
The schematic above (Diagram A) depicts the application, drilling into the application we see:
Data from the SmartPhone(s)¬† flows into the left most operator WebSocketInject() and flows out the right most operator ‘ WebSocketSend()’.
This document walks through the configuring, installing and running of the ReceiveToSend application. The components of the application are:
- Toolkits used to read/write web messages.
- ReceiveToSend.spl that receives data consolidates/processes and sends it out.
Download and unpack
Download streamsx.inet application and streamsx.demo.accelerometer toolkit bundle using the links in the ‘Resources’ section above. After downloading, use the ‘Archive Manager’ to extract the files.
I downloaded and unarchived the files to the desktop.
Load the toolkits
Within the Streams explorer pane,¬† go to the ‘Toolkit Location’ element, right-click and select ‘Add Toolkit Location…’ set the directory to the location that the bundle was unpacked to. Here is a capture of my screen:
Upon completion your ‘Toolkit Locations’ should include the imported toolkits as depicted below:
Import the Application
Import¬† project basic.accelerometer:¬† from the¬† ‘File’ menu ‘Import …’, browse to ‘Infosphere Streams Studio’ | ‘SPL Project’ where you will be prompted for the location of the project. Open the location where the application was unpacked, select the ‘Project Name’ of¬† ‘basic.accelerometer’, then select the ‘Finish’ button.
Create the Application
To create (compile) the application in standalone mode: Expand ‘basic.accelerometer’ to ‘RecieveToSend’, right-click and select ‘New’ then¬† ‘Standalone Build’.
On the next page select ‘Apply’ and ‘Continue’ to create and build the application.
Run the Streams Application
After the ‘ReceiveToSend’ application builds, launch it:
We’re running the application standalone so that we can monitor the processing via the console. The running application ‘Console’ will look like:
The application is waiting for a web socket connection a well as¬† an HTTP connection, the HTTP communication; will occur on port 8080.
Connecting the SmartPhone(s) to Streams
We’re ready to connect to the Streams application with the SmartPhone. Using the IP address retrieved with the ‘/sbin/ifconfig’ command (above) connect to the application via the SmartPhone’s browser. In this example I’m connecting to host’s IP 220.127.116.11¬† port 8080 using a Nexus7.
Select the blue ‘ Transmit Devices Accelerometer Data’ button to enter the data collector page, which looks like:
On entry into the page the Alpha, Beta, Gamma, X, Y and Z values will be updating. If you set the device on flat surface the Beta will go to approximately zero.¬† Rotating it toward you, the Gamma will change.
Select the blue ‘Start Transmitting’ button to start sending data to the Streams application.
You’ll get two alerts as depicted in the next two screen shots.
Select ‘OK’ to move on:
Select ‘OK’ to move on.
The alerts are validating your hardware. Refer to the Appendix if you do not get the described alerts.
If you scroll down the page, you’ll find a ‘Communications’ section with a “Message;” indicating that the communications is “OPEN”.¬† The browser is now sending data to the Streams via WebSockets.
Data flowing into Streams via WebSockets
On the ‘Console’ pane you’ll see data scrolling by. It will look something like:
Rotating the SmartPhone device, you should see the values in the windows change significantly.
Data flowing out of Streams via WebSockets
In order to see the data rendered using web sockets, you’ll need a bring up a current¬† version of Firefox or Chrome on a computer (laptop/desktop/SmartPhone). Use the same IP address/port you used to connect from the SmartPhone.
In this example I’m connecting to¬† host’s IP address (retrieved with the ‘/sbin/ifconfig’ command ) 18.104.22.168 port 8080 on a Mac using a Firefox browser, you should see something like:
Select the green ‘Display Streaming WS data’ button. You’ll be presented with the WebSocketServerSink messages:
The three graphs will be scrolling with no data on them. This web page will connect to Streams using WebSockets and render the data streamed from Streams. The streamed data will be displayed on the three graphs and the table below the graph.
Select the green ‘Start Receiving’ button in order to setup the connection to Streams.
You’ll get two alerts as depicted in the next two screen shots.
Select’OK’ to move on.
Select ‘OK’ to move on.
The live graph at the bottom of the screen will display a line. That is the data from the device after it’s routed through Streams. If you rotate the SmartPhone the Beta value (second graph) will update in ‘near’ real time.
The demonstration application will render data from up to nine SmartPhones. Every time a new connection is established between a SmartPhone and Streams a new line appears on the graph.¬† Connecting a second SmartPhone using the procedure described in section ‘Connecting the SmartPhone(s) to Streams’¬† will result in something like:
For details regarding the application and toolkits refer to the SPLDOCS accompanying the components.
Appendix – Issues
Error CDISP0127E ERROR : The following toolkit file is out of date …. :
This error may be encountered during the import or build process, this fix requires that rebuild the inet toolkit. The error will appear on the console, it will look like:
CDISP0127E ERROR: The following toolkit file is out of date: /home/streamsadmin/Desktop/com.ibm.streamsx.inet/toolkit.xml. This file is newer: /home/streamsadmin/Desktop/com.ibm.streamsx.inet/com.ibm.streamsx.inet.http/HTTPGetStream/HTTPGetStream.xml.
- Open an ‘xterm’¬†by double clicking on ”Terminal for streamsadmin’ .
- Rebuild the toolkit by executing¬† the the following command in the xterm:
spl-make-toolkit -i <directory>
Where <directory> is the directory containing the toolkit.xml, as shown in the CDISP0127 error message. For example, in the error message above¬†the directory is ‘/home/streamsadmin/Desktop/com.ibm.streamsx.inet’, the command to issue is :
spl-make-toolkit -i /home/streamsadmin/Desktop/com.ibm.streamsx.inet
The following screen capture illustrates:
After rebuilding the toolkit you should clean and rebuild the application, this is accomplished using the ‘clean..’ command within the ‘Project’ menu, it¬†looks like:
Make sure that the ‘Clean all projects’ is selected and then hit ‘OK’.
After the application rebuilds you can proceed on to ‘Run the Streams Application’.
Hardware Validation Failure :
The code validates the hardware and software capabilities. If issues arise the following messages can appear.
This is indicating that your device does not have an accelerometer that is accessible from the browser. You may want to try another browser and/or upgrade to the latest version. I’ve used this software on the Chrome and Safari browsers and have reports that it works on Firefox as well.
This is indicating that the browser does not have WebSocket access.¬† You should upgrade to the latest version of the browser and/or try another browser.
If you inadvertently start a second application before the first stops you’ll get the following error in the console.
Only one application can allocate the socket at a time. To remedy the situation kill the application¬† by selection the ‘red box’ of the active application’s console.
Appendix – Firewall configuration
Contributed by Edward Pring (email@example.com) – thanks
The Streams ReceiveToSend.spl application uses three network ports to communicate with the SmartPhone:
- TCP port 8084 is used by the SmartPhone to send accelerometer data to the Streams application.
- TCP port 8085 is used by the Streams application to send chart data to the browser
There may be one or more firewalls within the machine running Streams that could block any or all of these network ports. The instructions below can help you configure these firewalls to allow the SmartPhone and browser to communicate with Streams without compromising the security of the machine.
Linux ‘iptables’ firewall
The RHEL or CentOS operating system contains a firewall service called ‘iptables’. By default, it will block access to the TCP ports used by the Streams application. To allow the SmartPhone and browser to access them, do this:
- Log into RHEL or CentOS as ‘root’.
- Launch the Linux ‘Firewall Configuration’ application by selecting ‘System –> Administration –> Firewall’ from the Gnome menu, like this:
- Alternatively, Launch the Linux ‘Firewall Configuration’ application by typing ‘system-config-firewall’ at a Linux command prompt.
- In the ‘Firewall Configuration’ window, select ‘Other Ports’ and then click ‘Add’.
- In the ‘Port and Protocol’ window, select ‘User Defined’, enter ‘8080’ in the ‘Port / Port Range’ field, select ‘tcp’ in the ‘Protocol’ field, and click ‘OK”, like this:
- Repeat the previous step for ports 8084 and 8085.
- Close the ‘Firewall Configuration’ application.
- Confirm that ports 8080, 8084, and 8085 are enabled by typing ‘iptables -L -n’ at a Linux command prompt. Its output should include three lines like these:
ACCEPT tcp — 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:8080
ACCEPT tcp — 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:8084
ACCEPT tcp — 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:8085
- Alternatively, confirm that ports 8080, 8084, and 8085 are enabled by examining the file ‘/etc/sysconfig/iptables’. ¬†It should contain three lines like these:
-A INPUT -m state –state NEW -m tcp -p tcp –dport 8080 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 8084 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 8085 -j ACCEPT
VMware Fusion NAT firewall (for shared IP address)
If you are running RHEL or CentOS in a VMware Fusion virtual machine on Mac OS X, you may have configured its virtual ethernet adapter to ‘Share with my Mac’, like this:
In this configuration, the virtual ethernet adapter will use the IP address assigned to your real ethernet adapter, and VMware Fusion will place a NAT firewall between them, which will block access to the TCP ports used by the Streams application. To allow the SmartPhone and browser to access them, do this:
- Edit the VMware Fusion NAT configuration file. This requires Mac OS X ‘Administrator’ privilege, which you can obtain temporarily by executing an editor with the ‘sudo’ prefix, like this:
sudo emacs /Library/Preferences/VMware\ Fusion/vmnet8/nat.conf
- Enter your password for a Mac OS X password when prompted.
- Near the bottom of the ‘nat.conf’ file, in the ‘[incomingtcp]’ section, insert one line for each TCP port used by the Streams application, following the pattern shown in the comments, like this:
[incomingtcp] # Use these with care - anyone can enter into your VM through these... # The format and example are as follows: #<external port number> = <VM's IP address>:<VM's port number> 8080 = 192.168.199.127:8080 8084 = 192.168.199.127:8084 8085 = 192.168.199.127:8085
- Save your changes to the ‘nat.conf’ file and exit from the editor.
- Suspend your RHEL or CentOS virtual machine, close VMware Fusion.
- Open VMware Fusion and resume your RHEL or CentOS virtual machine.
¬†VMware Fusion (for separate IP address)
If you are running RHEL or CentOS in a VMware Fusion virtual machine on Mac OS X, you may have configured its virtual ethernet adapter for ‘Bridged Networking’, like this:
In this configuration, your virtual ethernet adapter will be assigned a separate IP address than your real ethernet adapter, and VMware Fusion will not use a NAT firewall.
Mac OS X firewall
If you are running VMware Fusion in Mac OS X, then the Mac OS X firewall may block access to the ports used by the Streams application. To allow the SmartPhone and browser to access them, do this:
- In the ‘System Preferences’ window, launch the Mac OS X ‘Security & Privacy’ application, like this:
- In the ‘Security & Privacy’ window, select the ‘Firewall’ tab and unlock it by clicking the Lock icon in the lower left corner, like this:
- Click “Firewall Options…”, then click “+”, add “VMware Fusion.app”, and select ‘Allow Incoming Connections’, like this:
- Click ‘OK’ and close the ‘Security & Privacy’ application.
Linux ‘/etc/hosts’ file
Streams requires that the hostname and IP address of the RHEL or CentOS machine match. ¬†If these configuration changes to any of the firewalls results in a different IP address, you may have to change the ‘/etc/hosts’ file accordingly. ¬†To ensure that the hostname and IP address of the machine match, do this:
eth1 Link encap:Ethernet HWaddr 00:0C:29:D0:36:6D ¬† ¬† ¬† ¬† ¬† inet addr:192.168.199.127 Bcast:192.168.199.255 Mask:255.255.255.0 ¬† ¬† ¬† ¬† ¬† inet6 addr: fe80::20c:29ff:fed0:366d/64 Scope:Link ¬† ¬† ¬† ¬† ¬† UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 ¬† ¬† ¬† ¬† ¬† RX packets:2597 errors:0 dropped:0 overruns:0 frame:0 ¬† ¬† ¬† ¬† ¬† TX packets:2554 errors:0 dropped:0 overruns:0 carrier:0 ¬† ¬† ¬† ¬† ¬† collisions:0 txqueuelen:1000 ¬† ¬† ¬† ¬† ¬† RX bytes:507712 (495.8 KiB) TX bytes:519181 (507.0 KiB)
- Edit the ‘/etc/hosts’ file. ¬†This requires Linux ‘root’ privilege, which you can obtain temporarily by executing an editor with the ‘sudo’ prefix, like this:
sudo emacs /etc/hosts
- Enter the ‘root’ password when prompted.
- Alter the ‘/etc/hosts’ file, if necessary, to map the IP address of the ethernet adapter to the hostname, like this:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.199.127 centos65.localdomain centos65.local centos65
- Save your changes and close the editor.
Appendix – Extended Resources
The Web community has developed the technology that I’ve utilized to deliver realtime data.
- HTML5 provides access to underlying hardware on your SmartPhone. Data includes: accelerometers, geolocation, ¬†camera, lightsensor, ¬†gyroscope, compass, battery status and¬†proximity events.
- WebSockets is a light weight W3C specified protocol for low latency ¬†asynchronous communication between nodes. Sending data from¬†the SmartPhone hardware turns it into a participant in the ‘Internet¬†of Things’.
These resources are bundled in the SmartPhone’s browser(s). It is not necessary to install any software on the SmartPhone in order to run.
The following open source components have been utilized for this demonstration.
- https://github.com/joewalnes/smoothie/¬† : Smoothie charts fro rendering data.
- https://github.com/TooTallNate/Java-WebSocket : WebSocket foundation that operators are built upon.
These components are bundled with the demo.