Taxonomy Icon

IoT

Optimize your visual recognition classification

Get the code Watch the video

Summary

Because the data sets that are used in visual recognition classification processes are so large, developers can get bogged down in classifying the images instead of working on other AI and machine learning technologies. By integrating Watson IoT Platform with the Watson Visual Recognition service, you can optimize the processing time to identify objects using the Visual Recognition trained models. This code pattern will show you how to build a Java-based training app to capture images from an IoT-enabled device and subsequently create and train the custom classifier by calling a Java API that is passed a set of images. This code pattern then shows you how to build a prediction app that classifies a new set of images based on the trained model by using Watson IoT Platform, Node-RED, and the Watson Visual Recognition service.

Description

Intelligent image recognition involves two processes:

  1. Building a training model
  2. Classifying images to identify objects based on the training model

This pattern consists of 2 apps: a training app and a prediction app. The prediction app provides an innovative solution for optimizing the classification process. Images are captured and published to Watson IoT Platform, and an IoT subscriber node in Node-RED then takes this published payload and passes it to the Watson Visual Recognition service invocation. The image classification results are stored in a Cloudant DB.

This pattern allows users to create and train a custom classifier using a Visual Recognition service instance. The Visual Recognition service identifies objects presented in an image using a pre-trained default classifier. Therefore, if someone wants VR to detect specific objects in a given image, then you need to create and train your own custom classifier.

In order to display identification results, you need to run the prediction app. IBM Watson IoT Platform gives a developer the flexibility to integrate and perform other services and tasks, such as analytics, storage, and so on. In this pattern, the Watson IoT Platform is used within the prediction app to call the Visual Recognition service to detect objects in an image. The advantage of this implementation is that it reduces the over all execution time for object detection.

Flow

flow

  1. In the training app, the user enters the Classifier name.
  2. In the training app, the user enters the negative images zip path.
  3. In the training app, a camera window gets launched to capture the specified number of positive images.
  4. In the training app, positive images get zipped and pass into java api along with negative images zip. A custom classifier gets created in a Visual Recognition service instance.
  5. In the training app, once training is complete, a message is sent to the Watson Text to Speech service, which announces that training is complete.
  6. In the prediction app, images are captured using camera device and put into a processing queue.
  7. In the prediction app, an image picked from the queue and gets converted into Base64 encoding.
  8. In the prediction app, an encoded payload gets publishes to a topic on Watson IoT Platform.
  9. In Node-RED, the IoT-IN node receives the published payload on the topic.
  10. In Node-RED, Base-64 encoded payload getting passed to VR service node and internally checks 3 things(features of VR): classifyImg, faceDetection, textDetection.
  11. In Node-RED, the result is sent to CloudantDB node to store into the DB.
  12. In Node-RED, the device command is sent to the device through IoT-OUT node to notify the device to send the next image in the queue for processing and to refresh the GUI to display latest results.

Instructions

See the README file for the complete instructions. Here are the main steps that you need to complete to use this code pattern:

  1. Create your IoT app using the Internet of Things Platform Starter boilerplate in IBM Cloud.
  2. Register your IoT device using the Watson IoT Platform dashboard.
  3. Create the Node-RED simulator to receive events from your IoT device and send notifications back to the IoT device.
  4. Build and run the training app and the prediction app.