Image Segmenter

Overview

This model takes an image file as an input and returns a segmentation map containing a predicted class for each pixel in the input image.

This repository contains 2 models trained on PASCAL VOC 2012. One model is trained using the xception architecture and produces very accurate results but takes a few seconds to run and the other model is trained on MobileNetV2 and is faster but less accurate. You can specify which model you wish to use when you start the Docker image. See below for more details.

The segmentation map returns an integer between 0 and 20 that corresponds to one of the labels below for each pixel in the input image. The first nested array corresponds to the top row of pixels in the image and the first element in that array corresponds to the pixel at the top left hand corner of the image. NOTE: the image will be resized and the segmentation map refers to pixels in the resized image, not the original input image.

Model Metadata

References

• Haozhi Qi, Zheng Zhang, Bin Xiao, Han Hu, Bowen Cheng, Yichen Wei, Jifeng Dai, Deformable Convolutional Networks — COCO Detection and Segmentation Challenge 2017 Entry. ICCV COCO Challenge Workshop, 2017.

• Mark Everingham, S. M. Ali Eslami, Luc Van Gool, Christopher K. I. Williams, John M. Winn, Andrew Zisserman, The Pascal Visual Object Classes Challenge: A Retrospective. IJCV, 2014.

• Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, Piotr Dollár, Microsoft COCO: Common Objects in Context. In the Proc. of ECCV, 2014.

Licenses

Options available for deploying this model

• Deploy from Dockerhub:

docker run -it -p 5000:5000 codait/max-image-segmenter

• Deploy on Kuberneters:

kubectl apply -f https://raw.githubusercontent.com/IBM/MAX-Image-Segmenter/master/max-image-segmenter.yaml

• Locally: follow the instructions in the model README on GitHub

Example Usage

You can test or use this model

• using cURL
• in a Node-RED flow

Test the model using cURL

Once deployed, you can test the model from the command line. For example if running locally:

curl -F "image=@assets/stc.jpg" -XPOST http://localhost:5000/model/predict

{
  "status": "ok",
  "image_size": [
    256,
    128
  ],
  "seg_map": [
    [
      0,
      0,
      0,
      ...,
      15,
      15,
      15,
      ...,
      0,
      0,
      0
    ],
    ...,
    [
      0,
      0,
      0,
      ...,
      15,
      15,
      15,
      ...,
      0,
      0,
      0
    ]
  ]
}

Test the model in a Node-RED flow

Complete the node-red-contrib-model-asset-exchange module setup instructions and import the image-segmenter getting started flow.

Links

• Magicat: command line utility for images inspired by the Unix cat utility (proof of concept)
• Deploy a deep learning-powered ‘Magic cropping tool’: a basic image segmentation and editing tool