Streams integrates with other technologies using adapters to popular protocols such as TCP, ODBC, Kafka, JMS, MQTT and HDFS, just to name a few. REST is another established protocol that is gaining popularity because of its use in many cloud-based services. This article describes how to use Streams HTTP adapters to integrate SPL applications to Cloudant and other RESTful, web-based services.
- Setting Up Cloudant
- Using Cloudant’s REST API from Streams
- Using other REST APIs from Streams
- Troubleshooting unsuccessful requests
Download the Streams Integration Samples JazzHub project and extract the CloudantGet and CloudantPost samples.
Setting up Cloudant
Getting connection information
Create a Cloudant account on cloudant.com, or create a Cloudant service on Bluemix. Note the following information.
The host is [username].cloudant.com
If you created the service through Bluemix, the username can be found on the “Service Credentials” tab of the Bluemix service dashboard.
Creating a Cloudant database
Navigate to the Cloudant dashboard by pressing Launch in Bluemix or logging in to cloudant.com.
Click “Add New Database” in the top right corner. Follow the prompts to name and create the database named “test”. An empty database will be created.
To view the contents of the database, click the database name link in the table of the “Databases” tab, and select “All Documents”.
Using Cloudant’s REST API
Putting documents into Cloudant
In Streams Studio, import the CloudantPost sample application into your Streams Studio workspace and launch it using your Cloudant credentials. The url is https://[username].cloudant.com/test .
For more information on Submission Time Values, see this article
This Streams application will create a new Cloudant document containing the string “Cloudant” every two seconds. For each document, the console will print “errorMessage: Created”, along with some other JSON metadata. Let the application run for a few seconds and then cancel the Streams job.
If the request was not successful, a few things might have happened:
- The console printed a JSON error response. You successfully connected to Cloudant, but some error occurred after that point. The “reason” field gives some hints about what went wrong.
- If the console printed a non-JSON error or exception, you have not successfully connected to Cloudant. Double check the url for accuracy.
- See “Troubleshooting unsuccessful requests”
Upon success, new documents have been added to Cloudant. Head back to the “All Documents” view to see them.
To view the data inside a document, select the “pencil” icon on the top-right corner of the dark-colored “document box”.
You should see the document that you just created.
Note that the CloudantPost sample application’s first operator is a Beacon that activates the HTTPPost and gives it the message body data. To change the frequency or number of requests, use the Beacon’s period or iterations parameters. However, the HTTPPost operator can connect with any stream carrying an rstring parameter named “message”.
Listing all documents in Cloudant
Now that we can create documents, we want to access them from the REST API. Launch CloudantGet with the same values, except change the url to [username].cloudant.com/test/_all_docs .
This Streams application will output the contents of the Cloudant database every two seconds.
While CloudantGet is running, you can run CloudantPost at the same time and watch the database grow from CloudantGet’s console output.
For more information on Cloudant’s REST API, see the Cloudant API Reference.
Using other REST APIs from SPL
Making a GET request
Using REST APIs, streams applications can integrate with nearly any web-based RESTful service. You may use the provided samples to work with other services.
To make a GET request to any RESTful service, you need the following information from the service:
- URL to retrieve the resource
- User name to connect to the service
- Password to connect to the service
- Any other service-specific information
In Streams Studio, import the CloudantGet project into your workspace. Launch the application, and specify submission time values based on the information you have collected from the service.
If the request was successful, the result will be printed to the console. If unsuccessful, an exception will display instead (see “Troubleshooting unsuccessful requests”).
Making a POST request
Similarly, to send a post request to a RESTful service, you need to provide url, username, and password of the service.
In Streams Studio, import CloudantPost project into the workspace. Build and launch the application using the appropriate values for the service.
You may need to modify some parameters of the sample’s HTTP operators in order to integrate with different services.
Troubleshooting unsuccessful requests
- Revisit the service’s REST API instructions. The service may expect information that wasn’t included in the request.
- Some REST APIs require fields not supported by Streams operators. In this case, waiting for an appropriate toolkit update may be necessary
This article has shown how Streams applications can integrate with Cloudant and countless other services using REST APIs.