IBM Cloud Satellite: Run and manage services anywhere Learn more

Performance evaluation of machine learning models

Summary

Machine learning models are numerous and are created to achieve specific tasks. This code pattern shows you a way to compare Watson cognitive service models to help you decide which model performs better for a particular set of data and which might be best for your needs. The code pattern gives you a platform to configure models, provide input data, and run and prepare performance evaluation statistics.

Description

Cognitive services like the Watson services cover many AI scenarios. And in the machine learning world, different models are being created to achieve different tasks. With so many models available, how do you decide which model to use or which model is performing better? The correct question is, which model best fits your needs. This code pattern provides details about Watson cognitive service models for performance evaluation and comparison. The Watson Model Evaluation Workbench application gives you a platform to configure, execute, and test cognitive models, prepare performance evaluation metrics, and calculate performance statistics like confusion matrices and ROC curves. Different models perform differently for a given set of data, and this code pattern helps you determine which model is best for your needs.

Flow

flow

  1. User launches the application.
  2. Cloud authenticates the request and redirects it to the application.
  3. Parses input data provided for evaluating the models.
  4. Invokes the adapter, which calls cognitive services like Natural Language Classifier and Natural Language Understanding.
  5. Parses the cognitive model services configuration.
  6. Connects to cognitive services.
  7. Gets response from cognitive services.
  8. Compares the expected result with the actual result and does performance evaluations.
  9. Performance results are sent back to client devices.
  10. Performance analysis is shown on the UI.

Instructions

Find the detailed steps for this pattern in the README. The steps will show you how to:

  1. Determine the prerequisites.
  2. Create the cognitive models.
  3. Deploy the application to IBM Cloud.
  4. Deploy the application to the local machine.
  5. Run the application.
  6. Analyze the results.