Taxonomy Icon

Artificial Intelligence

Continuous learning with Watson Machine Learning and IBM Db2 Warehouse on Cloud

Get the code View the demo

Summary

Learn how to use IBM Watson Machine Learning, Apache Spark, and Watson Studio to quickly build and prototype models, to monitor deployments, and to learn over time as more data becomes available. In this code pattern, you’ll learn how to use these services to create and deploy self-learning Watson Machine Learning models.

Description

Because model deployment is not a one-time event, you can use IBM Watson Studio to retrain a model with new data. Performance monitoring and continuous learning enable machine learning models to retrain on new data that you or other data sources supply. Then, all of your applications and analysis tools that depend on the model are automatically updated because Watson Studio handles selecting and deploying the best model.

This code pattern uses IBM Watson Machine Learning and Watson Studio to help you put machine learning and deep learning models into your application. After loading source data into IBM Db2 Warehouse on Cloud, the Watson Machine Learning service creates a machine learning model and saves the data back to Warehouse. Feedback data is uploaded to the Watson Machine Learning service to continuously learn and evaluate new data. Then, the model data is exposed through an API.

When you have completed this code pattern, you’ll understand how to:

  • Use Watson Studio to create a project and associate services
  • Use the IBM Machine Learning service to take advantage of machine learning models management (continuous learning system) and deployment (online, batch, streaming)
  • Use the Apache Spark as a Service cluster computing framework optimized for extremely fast and large scale data processing
  • Create and deploy self-learning Watson Machine Learning models

Flow

flow

  1. The initial source data is loaded in the IBM Db2 Warehouse on Cloud database.
  2. The source data is loaded as a data asset in Watson Studio.
  3. The Watson Machine Learning service uses the source data and computes an evaluation using Apache Spark as a Service to create a machine learning model, and saves the evaluation information back to the Db2 Warehouse on Cloud database.
  4. Apache Spark as a Service computes the evaluation.
  5. Feedback data is uploaded to the feedback table in the Db2 Warehouse on Cloud database.
  6. After the evaluation is complete, the Watson Machine Learning service creates a machine learning model.
  7. The model data is exposed through an API.
  8. Applications can use the API to evaluate new data and create a new model based on continuous learning.

Instructions

Get the detailed instructions in the README file. These steps show you how to:

  1. Clone the GitHub repo.
  2. Create a Watson Studio project.
  3. Add and refine a data asset in Watson Studio.
  4. Create Db2 Warehouse on Cloud and add the connection to Watson Studio.
  5. Create Apache Spark as a Service with IBM Cloud.
  6. Create Watson Machine Learning model.
  7. Add a new Watson Machine Learning model to Watson Studio.
  8. Add feedback data and new evaluations to the continuously learning model.
  9. Deploy the model to expose it through an API.
  10. Test the model.