Generate insights from multiple data sources

Get the code Watch the video

Summary

In this code pattern, we’ll generate insights by integrating data from multiple data sources like Db2 On Cloud, CSV File, Db2 Warehouse, using Watson Studio. Telling a story with data usually involves integrating data from multiple sources. Being able to combine data from multiple sources is essential when performing analysis. With this pattern, we’ll work with a few data sources namely Db2 On Cloud, CSV File, and Db2 Warehouse. With the power of Watson Studio, this technique can be applied to other sources like MySQL databases, IBM Db2 Big SQL, Oracle database, PostgreSQL, Microsoft SQL Server, and many more–no matter the dataset size.

Description

In this pattern, we’ll demonstrate the methodology with the following use case: A watch manufacturing company makes five types of watches, in three different branch locations (Manchester, Glasgow and Madrid). The watch company uses three different selling methods (telephone, visiting the store, and online). The sales data for each of these branches is stored in a different data source ( Db2 on Cloud, Db2 Warehouse, and CSV files). We’ll integrate data from all three sources and put it on a single data source (Db2 warehouse). This integrated data will then be used to derive insights–and visualized on an embedded dashboard. This will help us interpret which product is performing the best and which branch is performing the best.

After completing this code pattern, you’ll understand how to:

  • Connect and get data from multiple data sources.
  • Integrate data from multiple data sources.
  • Send integrated data to the Db2 Warehouse.
  • Derive insights and visualize on Watson Embedded Dashboard.

Flow

flow

  1. Extract data from local files (csv file).
  2. Extract data from Db2 on cloud.
  3. Integrate the data in Watson Studio.
  4. Send the data to Db2 Warehouse.
  5. Visualize and derive insights using the embedded dashboard.

Instructions

Get the detailed instructions in the README file. These steps will show you how to:

  1. Clone the repo.
  2. Create Watson services with IBM Cloud.
  3. Create the notebook.
  4. Add the data from local system.
  5. Add the Db2 connection.
  6. Add the Db2 Warehouse connection.
  7. Update the notebook with credentials and Db2 Warehouse table name.
  8. Run the notebook.
  9. Visualize and derive insights using the embedded dashboard analytics.