Image Segmenter

Overview

This model takes an image file as an input and returns a segmentation map containing a predicted class for each pixel in the input image.

This repository contains 2 models trained on PASCAL VOC 2012. One model is trained using the xception architecture and produces very accurate results but takes a few seconds to run and the other model is trained on MobileNetV2 and is faster but less accurate. You can specify which model you wish to use when you start the Docker image. See below for more details.

The segmentation map returns an integer between 0 and 20 that corresponds to one of the labels below for each pixel in the input image. The first nested array corresponds to the top row of pixels in the image and the first element in that array corresponds to the pixel at the top left hand corner of the image. NOTE: the image will be resized and the segmentation map refers to pixels in the resized image, not the original input image.

Model Metadata

References

• Haozhi Qi, Zheng Zhang, Bin Xiao, Han Hu, Bowen Cheng, Yichen Wei, Jifeng Dai, Deformable Convolutional Networks — COCO Detection and Segmentation Challenge 2017 Entry. ICCV COCO Challenge Workshop, 2017.

• Mark Everingham, S. M. Ali Eslami, Luc Van Gool, Christopher K. I. Williams, John M. Winn, Andrew Zisserman, The Pascal Visual Object Classes Challenge: A Retrospective. IJCV, 2014.

• Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, Piotr Dollár, Microsoft COCO: Common Objects in Context. In the Proc. of ECCV, 2014.

Licenses

Options available for deploying this model

• Deploy from Dockerhub:

  docker run -it -p 5000:5000 codait/max-image-segmenter
  

  Show more

• Deploy on Red Hat OpenShift:

  Follow the instructions for the OpenShift web console or the OpenShift Container Platform CLI in this tutorial and specify codait/max-image-segmenter as the image name.

• Deploy on Kuberneters:

  kubectl apply -f https://raw.githubusercontent.com/IBM/MAX-Image-Segmenter/master/max-image-segmenter.yaml
  

  Show more

  A more elaborate tutorial on how to deploy this MAX model to production on IBM Cloud can be found here.

• Locally: follow the instructions in the model README on GitHub

Example Usage

You can test or use this model

• using cURL
• in a Node-RED flow
• in CodePen
• in a serverless app

Test the model using cURL

Once deployed, you can test the model from the command line. For example if running locally:

curl -F "image=@samples/stc.jpg" -XPOST http://localhost:5000/model/predict

{
  "status": "ok",
  "image_size": [
    256,
    128
  ],
  "seg_map": [
    [
      0,
      0,
      0,
      ...,
      15,
      15,
      15,
      ...,
      0,
      0,
      0
    ],
    ...,
    [
      0,
      0,
      0,
      ...,
      15,
      15,
      15,
      ...,
      0,
      0,
      0
    ]
  ]
}

Test the model in a Node-RED flow

Complete the node-red-contrib-model-asset-exchange module setup instructions and import the image-segmenter getting started flow.

Test the model in CodePen

Learn how to send an image to the model and how to render the results in CodePen.

Test the model in a serverless app

You can utilize this model in a serverless application by following the instructions in the Leverage deep learning in IBM Cloud Functions tutorial.

Options available for training this model

This model can be trained using the following mechanisms:

• Train on IBM Cloud – Watson Machine Learning: follow the instructions in the model training README on GitHub.

Links

• Magicat: command line utility for images inspired by the Unix cat utility (proof of concept)
• Deploy a deep learning-powered ‘Magic cropping tool’: a basic image segmentation and editing tool

Resources and Contributions

If you are interested in contributing to the Model Asset Exchange project or have any queries, please follow the instructions here.