This tutorial shows how you can use drone aerial images, Watson Studio, and Watson Visual Recognition to survey wildfire-damaged neighborhoods and identify burned and intact homes.

Watson-Studio-screenshot

Learning objectives

After completing this tutorial, you will be able to:

  • Create a Visual Recognition model in Watson Studio running in IBM Cloud
  • Capture images from a drone and zip them into a class
  • Train a model to identify objects in the images
  • Score and count the identified objects

Prerequisites

You can complete this tutorial using an IBM Cloud Lite account.

Estimated time

This tutorial should take approximately 15 minutes to complete.

Step 1: Learn about drones

There are many types of drones available that range from toys to industrial use cases. Many of the drones now include a camera that can store or stream aerial video to the ground. Using the livestream video frames, you can sample frames and send the images to Watson Visual Recognition for classification.

Step 2: Capturing images

One of the fun experiences of flying a drone is capturing video or pictures from a unique aerial perspective. You can use your drone to capture images of interesting objects that you want to train a visual recognition model to autonomously identify.

In this tutorial, I have created three zip files of pictures recorded by drones. I will use these images to identify neighborhoods affected by the devastating 2018 West Coast wildfires. These images will be used as the training set.

Source attribution: USA Today article, various internet sources

Step 3: Set up Watson Studio

In this section, we create a Watson Studio account, create a Project, and create a Watson Visual Recognition model to identify images in several classes.

Create Cloud Object Storage

  1. Create a Cloud Object Storage instance by visiting the IBM Cloud Catalog.
  2. Search for Object in the IBM Cloud Catalog.
  3. Click the Object Storage service tile.

    Cloud Object Storage Catalog screenshot

  4. Click Create.

    Cloud Object Storage Catalog screenshot

Create a Watson Studio service instance

  1. Create a Watson Studio service instance from the IBM Cloud Catalog.
  2. Search for Studio in the IBM Cloud Catalog.

    Watson Studio Catalog screenshot

  3. Click the Watson Studio service tile.

    Watson Studio Service screenshot

  4. Click Create.

  5. After the Watson Studio service is created, click Get Started or visit Watson Studio.

    Watson Studio Launch screenshot

  6. Log in with your IBM Cloud account.

  7. Walk through the introductory tutorial to learn about Watson Studio.

    Watson Studio Welcome screenshot

Watson Studio Projects

Projects are your workspace to organize your resources, such as assets like data, collaborators, and analytic tools like notebooks and models.

Create a new project

  1. Click Create a Project.
  2. Select the Standard tile and press the Create Project button.

    Watson Studio New project screenshot

  3. Name your project Wildfire Burned Homes. The Cloud Object Storage instance created in an earlier step should be prefilled.

  4. Press Create.

    Watson Studio New project screenshot

You are ready to set up your project with Watson Visual Recognition.

Add assets to your Watson Studio project

To add assets, click the Assets tab.

Watson Studio  screenshot

Create a new Visual Recognition model

To create a new Visual Recognition model, click New Visual Recognition model.

Watson Studio  screenshot

Provision a new Watson Visual Recognition service instance

Your project must be associated with a Watson Visual Recognition Service instance. To associate it, click the click here link in the window to provision a new service.

Watson Studio  screenshot

Create a Watson Visual Recognition service

  1. Select the Lite plan and note the features.
  2. Scroll to the bottom and click Create.

Watson Studio  screenshot

Rename the Visual Recognition model

The Default Custom Model name is not descriptive so let’s rename it.

  1. Click the pencil icon to edit the name.

    Watson Studio  screenshot

  2. Rename the model as Count Burned Homes.

    Watson Studio  screenshot

Add custom classes to the Watson Visual Recognition model

  1. Click the + symbol to create a class.

    Watson Studio  screenshot

  2. Name this class Burned Home.

  3. Click Create.

    Watson Studio  screenshot

  4. Add a second custom class by clicking the + symbol again.

    Watson Studio  screenshot

  5. Name this class Intact Home.

  6. Click Create.

    Watson Studio  screenshot

Upload ZIP files to Watson Studio project

Three ZIP files have been prepared that contain aerial drone images. These files are:

  • BurnedHomes.zip
  • AerialHomes.zip
  • NotHomes.zip

  • Click Browse. An operating system native File Dialog opens.

  • Multi-select the three ZIP files BurnedHomes.zip, AerialHomes.zip, and NotHomes.zip.
  • Upload these ZIP files to your Watson Studio project

    Watson Studio  screenshot

Drag the ZIP files to custom classes

  1. Grab the BurnedHomes.zip file from the right navigation and drag it to the Burned Home class.

    Watson Studio  screenshot

    Watson Studio  screenshot

    The images in the ZIP file are added to the Burned Home class.

    Watson Studio  screenshot

  2. Grab the AerialHomes.zip file from the right navigation and drag it to the Intact Home class.

    Watson Studio  screenshot

  3. Grab the NotHomes.zip file from the right navigation and drag it to the Negative class.

    Watson Studio  screenshot

Train your Watson Visual Recognition custom classifier

  1. Click Train Model.
  2. Wait a few minutes for the model to train on the images.

    Watson Studio  screenshot

    Watson Studio  screenshot

  3. After the model has been trained, click the Click here link to view and test your model.

    Watson Studio  screenshot

Step 4: Test your model

  1. Review the Classes and Model details.
  2. Click the Test tab.

    Watson Studio  screenshot

Test Watson Visual Recognition Custom Classifier with sample images

  1. Visit this UK Daily Mail article and download a few of these drone images of devastated California neighborhoods.
  2. Load the images into the Test page by browsing or dragging the images to the Test page.

    Watson Studio  screenshot

  3. Inspect the scores returned by the Watson Visual Recognition Custom Classifier.

    Watson Studio  screenshot

Implement Watson Visual Recognition custom model in your applications

You can incorporate this Watson Visual Recognition Custom Classifier model into your applications using a variety of programming languages.

  1. Click the Implementation tab to review the code snippets.

    Watson Studio  screenshot

  2. Use the following code snippets to classify images against your model. For reference, the full API specification is available.

  3. API endpoint

    https://gateway.watsonplatform.net/visual-recognition/api
    
  4. Authentication

    curl -u "apikey:{apikey}" "https://gateway.watsonplatform.net/visual-recognition/api/{method}"
    
  5. Classify an image (GET)

    curl -u "apikey:{apikey}" "https://gateway.watsonplatform.net/visual-recognition/api/v3/classify?url=https://watson-developer-cloud.github.io/doc-tutorial-downloads/visual-recognition/fruitbowl.jpg&version=2018-03-19&classifier_ids=CountBurnedHomes_1382538940"
    
  6. Classify an image (POST)

    curl -X POST -u "apikey:{apikey}"-F "images_file=@fruitbowl.jpg" -F "threshold=0.6" -F "classifier_ids=CountBurnedHomes_1382538940" "https://gateway.watsonplatform.net/visual-recognition/api/v3/classify?version=2018-03-19"
    

Summary

This tutorial explained how you can use drone aerial images, Watson Studio, and Watson Visual Recognition to survey wildfire-damaged neighborhoods and identify burned homes and intact homes. You should now know how to create a Visual Recognition model in Watson Studio running in IBM Cloud, capture images from a drone and zip them into a class, train a model to identify objects in the images, and score and count the identified objects.