Streams applications use the Inet toolkit to provide web connectivity, exposing¬†Streams functionality to the web. For example, the
HTTPRequestProcess operator is used to enable HTTP forms and REST calls to a Streams application.
This is¬†fine when the Streams application is running¬† in a VM or bare hardware. Problems arise when running applications on the IBM Cloud. For security reasons, Streams applications are not allowed to expose themselves on the web when running within the Cloud.¬† As discussed in this article, there are multiple ways to work around this issue. These include using a messaging system such as Kafka or MQTT instead of HTTP, or using a secure gateway.¬† You can also use a proxy server, running on the IBM Cloud, to form a bridge between the internet and Streams. This article will discuss using a proxy server.
Problem: You have a web enabled Streams application¬† and you want it to run it in the IBM Cloud.
Solution : You need a proxy server to bridge the gap between the web and Streams. This article provides a self configuring proxy server and the instructions to bridge the gap.
Provided is a self configuring proxy server that enables a Streaming Analytics application running on IBM Cloud to be accessible from the internet.¬† It runs on the¬† Liberty for Java runtime in the IBM Cloud, so all you have to do is deploy the server to the cloud and connect it to the Streaming Analytics service.
I have also provided a sample Streams application that accepts and responds to HTTP requests. You can use this application to test the proxy server.
Steps to form a bridge:
- On the IBM Cloud create a¬†Liberty for Java Foundry App and an instance of the Streaming Analytics¬† service and connect them.
- Upload a Streams application (example ‘UpperRestRH6.sab’ included). The Streams application must be running before the first ProxyServer request is made.
- Build and upload the LibertyStreamsProxy .war file to form a bridge between the web and the Streams application.
The proxy server is self configuring, it locates the Streams application Inet operators and configures the proxy server to bridge the gap.¬† It determines the address and port of the Inet toolkit’s operators running within the IBM Cloud, and then forwards requests from the internet¬† to the Streams application.
Code, Video and binaries.
- A IBM Cloud¬† account where you’ll provision a Server and¬† Streams resource.
- A Streams application that uses the inet toolkit, use the provided UpperRestRH6.sab if you do not have one.
- IBM Cloud CLI,¬† to upload and control your proxy server. Use the tool is upload the .war, start and stop the Web Server.
- Maven, this comes on the Streams Quick Start edition installed.
- SAB file for a sample Streams application that upper cases a string sent via the web. A built version is provided for convenience. The application graph looks like:
The first operator is an instance of the
HTTPRequestProcess operator which¬† receives the lowercase string, and also sends back the result of converting it to uppercase.¬† You can download the sab file, UpperRestRH6.sab from Github.
- ¬†The source code for the Streams application and the Web Server, including pom.xml to build and test the proxy server can be found in the IBM Streams samples repository.¬†¬† Download the project as a zip file.
Walk through video:
The video demonstrates how to bring up the demonstration. The highlights are listed below.
- Provision a ‘Streaming Analtyics‘ service, specify a ‘Service Name’ (ie LibertyRiverAnaytics). You will need the ‘Service Name’ when you connect the Cloud Foundry App to this service.
- Provision a Liberty for Java Cloud Foundry App,¬† specify the ‘App name’ (ie LibertyRiver).
- Connect your Cloud Foundry App’ to your ‘Streaming Analytics’ using the connect button.
- Using the Streams console upload the provided UpperRestRH6.sab file.
- In a terminal, change into the General/LibertyStreamsProxy directory of the git repository you downloaded.
- Login to bluemix using sso, you will need to generate the temporary password using the URL presented. After logging in set your target.
> bluemix login -o email@example.com --sso > bluemix target -o firstname.lastname@example.org -s dev
- Install components and build the proxy server into the target directory.
> mvn -U initialize > mvn clean install
- Push the war file using your ‘App name’, this will start the proxy server as well.
> bluemix¬†cf push LibertyRiver -p target/LibertyStreamsProxy-v1.0.war
- Run the tests, slotting in the ‘App name’ where appropriate.
> mvn clean test -DappUrl=LibertyRiver.mybluemix.net -DskipTests=false
- The following can be accessed from a browser, slotting in the ‘App name’ where appropriate.
Displays the steps in deriving the proxy configuration.
- Invoke the Streams application that upper cases input text.
- Display the Stream application resources
- When done, you can shut down the proxy server from the terminal.
> bluemix cf stop LibertyRiver
Leaving the proxy up may result in excessive charges being incurred by your application if internet ‘trolls’ discovery your proxy’s url. Shutting down the proxy server is advisable.
- The Streams application must be running before the first ProxyServer request is made, the ProxyServer locates the Inet’s operator using a metric.
- Restarting the Streams application may require restarting the ProxyServer when your instance has more than one node.
- This is a rudimentary ProxyServer that can be used as a starting point for creating a fault tolerant, secure and dynamic ProxyServer for your IBM Cloud Streaming Analytics’¬† applications.
Get the Code
- Git Repository with source code for the LibertyStreamsProxy server, Streams application, and downloadable sab file.
- Learn more about using Liberty by visiting WASDev, the Liberty development community.