The Model Asset eXchange (MAX) on IBM Developer is a place where you can find and use free open source deep learning models for text, image, audio, and video processing. The curated list includes deployable models that you can run as a microservice locally or in the cloud on Docker or Kubernetes, and trainable models where you can use your own data to train the models.

Model Asset Exchange landing page

Learning Objective

By completing this introductory tutorial, you learn how to deploy a model microservice and consume it in a basic web application.

Prerequisites

Estimated time

It should take you approximately 60 minutes to complete this tutorial. The tutorial modules are:

Tutorial setup

  1. In a terminal window, clone the tutorial repository.

    git clone https://github.com/IBM/max-developer-tutorial.git
    cd max-developer-tutorial
    
  2. If you want to complete module 2 of this tutorial (instead of just reviewing the solution), ensure that Node.js or Python is installed.

You are ready to start the tutorial.

Run a MAX model asset on Docker

Deployable MAX model assets implement microservices that you can run on Docker or Kubernetes.

Components of a MAX asset

In this module, you learn how to:

  • Run a model microservice on Docker by using a pre-built Docker image from Docker Hub
  • Run a model microservice on Docker by using a custom-built Docker image
  • Explore the model microservice

If you only want to explore the functions of a MAX model microservice in a local environment, you can run it by pulling the corresponding Docker image from Docker Hub.

Running the pre-built Docker image

The pre-built Docker images provide an easy way to explore a model microservice without the need to install any other software (aside from Docker) on your machine. However, because the image is pre-built, you cannot customize the microservice or review its implementation.

  1. Open the Object Detector model page.

    Object Detector model landing page

    Note: You can explore the model service API of some models by clicking the Try the API button.

  2. Locate the instructions to deploy the pre-built Docker image from Docker Hub and execute the listed docker run command in a terminal window to start the microservice on your local machine.

    docker run -it -p 5000:5000 codait/max-object-detector
    

    A status message is displayed after the container has started and the service is available, identifying the URL where the service can be accessed.

  3. Open the displayed URL in a web browser, for example, http://0.0.0.0:5000.

  4. Explore the microservice specification. Each running microservice exposes at least the following three endpoints:

    • GET / – Returns the microservice’s Swagger specification.
    • GET /model/metadata – Returns information about the model that the microservice is serving, such as name, description, and license.
    • POST /model/predict – Returns the results of a prediction, given the provided input data. Supported input data types and formats vary by model. Refer to the service’s Swagger endpoint description for details.

    Whenever a prediction is performed, the microservice takes care of:

    • Transforming the input (for example, an image, text, or a sound file) into a format that can be consumed by the deep learning model
    • Invoking the deep learning framework API that the model was implemented in
    • Filtering the model output and transforming it into an application-friendly format
  5. Try the prediction endpoint. You can try the model’s prediction capability by invoking its POST /model/predict endpoint from the Swagger UI or by sending a request using any API client. You can find model-specific example inputs in the model’s GitHub repository.

    A simple curl-based example request for the Object Detector model might look like:

    curl -F "image=@assets/dog-human.jpg" -XPOST http://localhost:5000/model/predict
    

    Prediction results are typically returned as JSON.

     {
      "status": "ok",
      "predictions": [
        {
           "label_id": "1",
           "label": "person",
           "probability": 0.944034993648529,
           "detection_box": [
               0.1242099404335022,
               0.12507188320159912,
               0.8423267006874084,
               0.5974075794219971
           ]
         },
       ...
    
  6. Stop the microservice.

    1. Use the docker ps command to list running containers.

      docker ps
      
    2. Note the CONTAINER ID for the appropriate image.

      CONTAINER ID   IMAGE
      53...          codait/max-object-detector
      
    3. Run the docker stop command to stop the microservice, specifying the CONTAINER ID from the previous step.

      docker stop 53...
      

Running a customizable Docker image

You can review the Docker image content, such as the model serving code implementation, or customize the Docker image by cloning the Model asset’s GitHub repository.

  1. On the Object Detector model page, click Get this model to navigate to the model’s GitHub repository.

  2. Follow the instructions to build the model. The build process typically consists of two steps:

    • Clone the source to your local machine.
    • Build the Docker image.

      To build the Object Detector Docker image, run the following commands in a terminal window:

      git clone https://github.com/IBM/MAX-Object-Detector.git
      cd MAX-Object-Detector
      

      The model serving code is typically implemented in Python by using Flask and a deep learning framework library, such as TensorFlow or PyTorch.

      Note that the consumer of this service does not need to know which deep learning framework was used because the service API is framework agnostic.

      When you are ready to build the image, run the docker build command:

      docker build -t max-object-detector .
      

      During the build process, the required dependencies are downloaded.

      Note that this might take some time depending on the image you are building.

      After the build process is complete, it’s time to deploy.

  3. Run the listed docker run command in a terminal window to start the microservice:

    docker run -it -p 5000:5000 max-object-detector
    

    A status message is displayed after the container has started and the service is available, identifying the URL where the service can be accessed.

  4. Open the displayed URL in a web browser, for example, http://0.0.0.0:5000/.

  5. Do not stop the container and do not terminate the terminal window. You will use the container in the next module.

Module 1 summary

In this module, you:

  • Ran a model microservice on Docker by using a pre-built Docker image from Docker Hub
  • Ran a model microservice on Docker by using a custom-built Docker image
  • Explored the model serving microservice

In the next module, you’ll learn how to consume a model microservice from a Node.js or Python web application.

Consume a MAX model asset

So far, you’ve deployed a model serving microservice on Docker in a local environment and explored the service’s API endpoints. In this module, you learn how to:

  • Invoke the prediction endpoint of a model microservice
  • Process and visualize the results by using a simple web application

The following chart shows the data flow between our sample web application (on the left) and the Object Detector model serving microservice:

architecture

Setup

  1. Ensure that the model microservice is running by executing the following command in a terminal window.

    curl -i http://localhost:5000/model/metadata
    

    If the command returns an error, start the microservice:

    docker run -it -p 5000:5000 codait/max-object-detector
    

In preparation for this tutorial, we have created simple Node.js and Python web application stubs that you will complete by adding the code that is required to invoke the model microservice’s prediction endpoint and process and visualize the results.

Continue with the instructions for your programming language:

Consuming using Node.js

  1. Clone the Node.js sample application repository https://github.com/IBM/max-tutorial-app-nodejs in a terminal window.

    git clone https://github.com/IBM/max-tutorial-app-nodejs.git
    cd max-tutorial-app-nodejs
    

    Note: If you’d like to review the finished sample application (instead of completing the following steps), check out the solution branch by running git checkout solution in the terminal window.

  2. Open the static/index.html, static/js/webapp.js, and app.js files in your favorite editor.

    • static/index.html implements a simple web UI, providing users with the ability to select a PNG/JPG image from their local file system.
    • static/js/webapp.js contains JavaScript code that invokes the model microservice’s prediction endpoint /model/predict, parses the results, and draws annotated bounding boxes around detected objects.
    • app.js implements a simple Express-based web server that serves the static application content (web pages, JavaScript, stylesheets, and so on) and redirects requests to the model microservice.
  3. In the static/js/webapp.js file:

    1. Locate TODO R1 and review the prediction request payload. The model that you are using requires a PNG/JPG-encoded image file as input and accepts a threshold as optional input. The threshold acts as a filter, removing detected objects from the result set that the model is less certain about. In this sample application, we are overriding the default (0.7) for illustrative purposes with a lower value.
    2. Locate TODO T1 and replace the **TODO** placeholder with your microservice’s prediction endpoint /model/predict.
    3. Locate TODO T2 and replace the **TODO** placeholder with your microservice’s prediction request method POST.
    4. Save your changes.

      Note: Remember that you can always review the Swagger specification for the supported model endpoints by opening the microservice URL (for example, http://localhost:5000/) in a web browser.

  4. Install the prerequisites and launch the application.

     npm install
     npm start
    
  5. Open http://localhost:8090/ in your web browser.

    • In Safari: Open the Network tab in the Web Inspector (MacOS:
    • In Google Chrome: Open the Network tab in the Developer Tools (MacOS:
    • In Mozilla Firefox: Open the Network tab in the Web Developer Tools (MacOS:
  6. Choose a sample image from the assets/ directory (or any other PNG/JPG image you might have stored on your machine), and click Detect Objects.

    If you’ve defined the endpoint URL and request method properly, a message appears indicating that no objects were detected. This is expected because you have not yet customized the code that processes the microservice response.

  7. In your browser’s Network tab, inspect the response of your prediction request. It should look similar to the following code:

     {
       "status": "ok",
       "predictions": [
           {
               "label_id": "88",
               "label": "teddy bear",
               "probability": 0.9896332025527954,
               "detection_box": [
                   0.27832502126693726,
                   0.5611844062805176,
                   0.643224835395813,
                   0.8432191610336304
               ]
           },
           {
               "label_id": "1",
               "label": "person",
               "probability": 0.9879012107849121,
               "detection_box": [
                   0.24251864850521088,
                   0.26926863193511963,
                   0.6558930277824402,
                   0.5768759846687317
               ]
           }
       ]
     }
    

The predictions array contains an entry for each detected object, identifying:

  • The object label (for example, person, dog, or cat)
  • The confidence (probability) that the detected object represents the specified label (1 = high confidence, 0 = low confidence)
  • The coordinates (y_1, x_1, y_2, x_2) of a rectangle that encloses the detected object

  • In the static/js/webapp.js file:

    1. Locate TODO T3 and replace the **TODO** placeholder with predictions.
    2. Save your changes.
  • Reload http://localhost:8090/ in your web browser and try to detect objects again.

    The application should display bounding boxes around detected objects along with the label.

    annotated_objects

Consuming using Python

  1. Clone the Python sample application repository https://github.com/IBM/max-tutorial-app-python in a terminal window.

     git clone https://github.com/IBM/max-tutorial-app-python.git
     cd max-tutorial-app-python
    

    If you’d like to review the finished sample application (instead of completing the following steps), check out the solution branch by running git checkout solution in the terminal window.

  2. Open the app.py file in your favorite editor.

    The application implements a simple Flask-based web server that serves the static application content (web pages, stylesheets, and so on) and invokes the model microservice’s prediction endpoint.

  3. In the app.py file:

    1. Locate TODO R1 and review the prediction request payload. The model that you are using requires a PNG/JPG-encoded image file as input and accepts a threshold as optional input. The threshold acts as a filter, removing detected objects from the result set that the model is less certain about. In this sample application, we are overriding the default (0.7) for illustrative purposes with a lower value.
    2. Locate TODO T1 and replace the **TODO** placeholder with your microservice’s prediction endpoint /model/predict.
    3. Save your changes.

      Remember that you can always review the Swagger specification for the supported model endpoints by opening the microservice URL (for example, http://localhost:5000/) in a web browser.

  4. Install the prerequisites and launch the application in the terminal window.

     pip install -r requirements.txt
     python app.py
    
  5. Open http://localhost:8090/ in your web browser.

  6. Choose a sample image from the assets/ directory (or any other PNG/JPG image that you might have stored on your machine) and click Detect Objects.

    If you’ve defined the endpoint URL and request method properly, a message appears indicating that no objects were detected. This is expected because you have not yet customized the code that processes the microservice response.

  7. Inspect the application’s debug output. It should look similar to this:

    {
     "status": "ok",
     "predictions": [
         {
             "label_id": "88",
             "label": "teddy bear",
             "probability": 0.9896332025527954,
             "detection_box": [
                 0.27832502126693726,
                 0.5611844062805176,
                 0.643224835395813,
                 0.8432191610336304
             ]
         },
         {
             "label_id": "1",
             "label": "person",
             "probability": 0.9879012107849121,
             "detection_box": [
                 0.24251864850521088,
                 0.26926863193511963,
                 0.6558930277824402,
                 0.5768759846687317
             ]
         }
     ]
    }
    

The predictions array contains an entry for each detected object, identifying:

  • The object label (for example, person, dog, or cat)
  • The confidence (probability) that the detected object represents the specified label (1 = high confidence, 0 = low confidence)
  • The coordinates (y_1, x_1, y_2, x_2) of a rectangle that encloses the detected object

  • In the app.py file:

    1. Locate TODO T2, uncomment the next line, and replace the **TODO** placeholder with predictions.
    2. Save your changes.
  • Reload http://localhost:8090/ in your web browser and try to detect objects again.

    The application should display bounding boxes around detected objects along with the label.

    annotated_objects

Module 2 summary

In this module, you:

  • Added code to invoke a deep learning model prediction endpoint
  • Processed and visualized the response

Conclusion

In this tutorial, you learned how to deploy a model microservice and consume it in a basic web application. It explained how you set up your environment and run and consume microservices.