Orchestrate data science workflows using Node-RED

Note: This pattern is part of a composite pattern. These are code patterns that can be stand-alone applications or might be a continuation of another code pattern. This composite pattern consists of:

Summary

Data science workflows that don’t have a user interface are difficult to demonstrate to business users. This developer journey shows you how to build a web interface using Node-RED. The interface will trigger data science workflows that run Jupyter Notebook code on IBM Watson Studio.

Description

Data science’s goal is to extract insight from data. Tools such as IBM Watson Studio are designed to be used by professionals with in-depth knowledge of data science concepts and applied statistics. But developers looking for a way to bridge the gap between data scientists and the business community now have a new way to build that connection.

In this code pattern, you’ll use the Node-RED programming tool to create a web interface, which will trigger an analytics workflow that runs Jupyter Notebook code on IBM Watson Studio. You’ll build an end-to-end app that enables business people to harness the power of data science.

When you complete this code pattern, you will learn how to:

  • Develop on IBM Watson Studio using Python Pandas to derive insights from data
  • Create a web interface using Node-RED
  • Trigger execution of Jupyter Notebooks code within IBM Watson Studio using Node-RED

This pattern is ideal for anyone looking to demonstrate the power of data science directly to business users.

Flow

alt

  1. The object storage system stores the data documents.
  2. The Python code retrieves the document content from the object storage system.
  3. The Jupyter Notebook processes the data and generates insights based on the event received from the UI.
  4. The Jupyter Notebook is powered by Apache Spark.
  5. Node-RED hosts a WebSocket server that enables the Jupyter Notebook on IBM Watson Studio to communicate with the web UI.
  6. An event is triggered from the web UI and is sent to the Python code on the Jupyter notebook using WebSockets. The UI reads the response from the Python code on the WebSocket server and displays the results.

Instructions

  1. Sign up for Watson Studio.
  2. Create IBM Cloud services.
  3. Import the Node-RED flow.
  4. Note the websocket URL.
  5. Update the websocket URL.
  6. Create the notebook.
  7. Add the data.
  8. Update the notebook with service credentials.
  9. Run the notebook.
  10. Analyze the results.
Balaji Kadambi
Shikha Maheshwari
Vishal Chahal
Manjula G Hosurmath