A developer’s journey from attending a Call for Code hackathon to open sourcing drone tech as one of Code and Response’s first projects

On September 20, 2017, Hurricane Maria struck my home, Puerto Rico. After surviving the record-breaking Category 5 storm and being personally affected by its aftermath, I decided I was going to make it my mission to create technology that could help mitigate the impact hurricanes have on our island.

Inspired by Call for Code

Can you imagine trying to plan relief efforts for more than three million people? People in rural areas, including a community in Humacao, Puerto Rico, suffered the most. The people in this community were frustrated that help was promised but never came. So the community came together and painted “water” and “food” on the ground as an SOS, in hope that helicopters and planes would see their message. For me, it was sad and frustrating to see that the reality outside of the metro area was different. Lives were at risk.

Fast-forward to August 2018. Less than a year after the hurricane hit, I attended the Call for Code Puerto Rico Hackathon in Bayamón, Puerto Rico. I was intrigued by this global challenge that asks developers to create sustainable solutions to help communities prepare for, respond to, and recover from natural disasters.

The SOS messages after the Hurricane inspired me to develop DroneAid, a tool that uses visual recognition to detect and count SOS icons on the ground from drone streams overhead, and then automatically plots the emergency needs captured via video on a map for first responders. I thought that drones could be the perfect solution for rapidly assessing damages from the air and they could help with capturing images that could then be processed by AI computer vision systems. At first, I thought of using OCR (optical character recognition) technologies to detect letters. The problem with this approach is that everyone has different handwriting. If we want this to work in other languages, it will be very complex.

After a few hours of coding, I pivoted and decided to simplify the visual recognition to work with a standard set of icons. These icons could be drawn with spray paint, chalk, or even placed on mats. Drones could detect those icons and communicate to first responders on a community’s specific victims needs for food, water, and medicine. I coded the first iteration of DroneAid at that hackathon and won first place. This achievement pushed me to keep going. In fact, I joined IBM as a full-time developer advocate.

DroneAid is so much more than a piece of throwaway code from a hackathon. It’s evolved into an open source project that I am excited to announce today. I’m thrilled that IBM is committed to applying our solution through Code and Response, the company’s unique $25 million program dedicated to the creation and deployment of solutions powered by open source technology to tackle the world’s biggest challenges.

Open sourcing DroneAid through Code and Response

DroneAid leverages a subset of standardized icons released by the United Nations. These symbols can either be provided in a disaster preparedness kit ahead of time or recreated manually with materials someone may have on hand. A drone can survey an area for these icons placed on the ground by individuals, families, or communities to indicate various needs. As DroneAid detects and counts these images, they are plotted on a map in a web dashboard. This information is then used to prioritize the response of local authorities or organizations that can provide help.

From a technical point of view, that means that a visual recognition AI model is trained on the standardized icons so that it knows how to detect them in a variety of conditions (I.e. whether they are distorted, faded, or in low light conditions). IBM’s cloud annotations tool makes it straightforward to train AI using IBM Cloud object storage. This model is applied to a live stream of images coming from the drone as it surveys the area. Each video frame is analyzed to see if any images exist. If they are, their location is captured and they are counted. Finally, this information is plotted on a map indicating the location and number of people in need.

The system can be run locally by following the steps in the source code repository, starting with a simple Tello drone example. Any drone that can capture a video stream can be used since the machine learning model leverages Tensorflow.js in the browser. This way we can capture the stream from any drone and apply inference to that stream. This architecture can then be applied to larger drones, different visual recognition types, and additional alerting systems.

architecture diagram for droneaid

Calling all developers to collaborate in the DroneAid open source community

It’s been quite a journey so far and I feel like we’re just getting started. Let’s unite to help reduce loss of life, get victims what they need in a timely manner and help reduce the overall effects a natural disaster will have on a community.

Our team decided to open source DroneAid because I feel it’s important to make this technology available to as many people as possible. The standardized icon approach can be used around the world in many natural disaster scenarios (i.e., hurricanes, tsunamis, earthquakes, and wildfires) and having developers contribute by training the software on an ongoing basis can help increase our efficiency and expand how the symbols can be used together. We built the foundation for developers to create new applications and envision using this technology to deploy and control a fleet of drones as soon as a natural disaster hits.

Now that you understand how DroneAid can be applied, join us and contribute here: https://github.com/code-and-response/droneaid

Pedro Cruz

Putting AI to work with Watson

Navigating IBM data and AI products

Many enterprises are now looking at artificial intelligence (AI) and how to use their own data. Enterprise needs vary. Depending on what data is being stored and sent, the data might be required to be kept on-premises. IBM is a great fit for enterprises with strict business needs (like HIPPA or GDPR) because IBM keeps these requirements in mind as the software is written. IBM also provides on-ramps with free-to-use offerings, learnings paths, tutorials, and videos.

In this blog, I’ll look at these sets of data and AI products from a developer’s point of view:

Product families

Before looking at each set of products, let’s take a quick detour and see how they can be used.

Deployment options

Most of the IBM data and AI products can be accessed through:

  1. IBM Cloud: The IBM public cloud offering that offers lite and pay-as-you-go plans
  2. IBM Cloud Pak for Data: A bundle of data and AI add-ons that can be installed on Red Hat OpenShift

The following image shows a representation of each stack for consuming the data and AI offerings on both IBM Cloud Pak for Data and IBM Cloud.

IBM Cloud IBM Cloud Pak for Data
alt alt

NOTE: The Watson Machine Learning Accelerator product suite requires Power hardware.

Watson APIs

These services are the most easily understood, and knowing AI concepts is not required to use them. They each have a REST API that can be called, and are meant to be called at the application level by SDKs. These services use pre-built models and provide a user-friendly way to create user custom models.

alt

Watson APIs:

  • Include services such as Assistant, Discovery, Tone Analyzer, and Visual Recognition
  • Have well-maintained APIs
  • Have SDKs for popular languages (such as the Java language, Node, Swift, and Go)
  • Have generous free tier plans that let you get your hands dirty without excessive costs
  • Are available on the IBM Cloud and as add-ons to IBM Cloud Pak for Data
  • Are truly a great place for developers to start their AI journey

Watson Assistant

The main “chatbot” offering and so much more. Watson Assistant comes pre-trained with industry-relevant content. It has an intuitive editor; provides middleware for integrating with Slack, Facebook, and many other platforms; and can even be integrated with Watson Voice Gateway so that you can talk to your assistant over a phone.

Watson Discovery

The “AI search” offering. Simply put, use the Discovery tool or API to load a bunch of documents (such as .pdf or .doc files) and have Discovery build a model for you. You can query this model with natural language, and Discovery pulls out relevant parts of the documents. You can make Discovery smarter by using “Smart Document Understanding” to ensure documents are read properly or by using “Watson Knowledge Studio” to build a domain or industry-specific model.

Watson Visual Recognition

Explaining this service is pretty simple. Step one is to upload an image and step two is to read the output from the service after it tried to classify the image. That’s it! By default, Watson Visual Recognition comes with two pre-built classifiers (general and food). But where it excels is letting you create your own classifiers. You can use the Watson Visual Recognition tools to upload your own images to serve as “training data,” and Watson Visual Recognition creates the model for you. This model is callable through a REST API or can be exported as a Core ML model for iOS devices.

And more

  • Watson Text to Speech: Text goes in, audio comes out
  • Watson Speech to Text: User’s voice goes in, text comes out
  • Watson Translator: Text goes in, select your source and destination languages, more text comes out
  • Watson Natural Language Understanding: A short string of text goes in, and concepts, entities, keywords, emotions, and sentiment comes out
  • Watson Natural Language Classifier: Build text classifiers based on training data, and then classify text against the models that are built

Tabular representation

Offering On Cloud On Prem Free tier (on Cloud) SDK support
Assistant
Discovery
Visual Recognition
Text to Speech
Speech to Text
Translator
Natural Language Understanding
Natural Language Classifier

Now onto the hard stuff, no more pre-made models. So, let’s move on to Watson Studio.

Watson Studio

Watson Studio gives you an environment and tools to collaboratively work with data. Data can be imported (through connectors), viewed, refined, and analyzed. Models can then be created with the Watson Studio Deep Learning and Machine Learning tools.

Watson Studio image

Watson Studio:

  • Is based on various open source technology such as Jupyter Notebooks and R Studio
  • Provides tools based on popular open source frameworks such as Tensorflow, Keras, and Pytorch
  • Includes 2 GB of free storage on IBM Cloud Object Storage
  • Is available on a public cloud and as add-ons to IBM Cloud Pak for Data
  • Is an excellent platform to begin your data science journey

Jupyter Notebooks

Watson Studio provides support for running Jupyter Notebooks. There are several environments to choose from (Python 3.x, R3.4, and Scala 2.11), and each has the option to add a Spark kernel as well. You can share your Notebook collaboratively, add data from your Object Store as a data frame, publish your Notebook, and use many other of the features you’ve come to expect.

Watson Machine Learning

The Watson Machine Learning service is a major part of Watson Studio. It provides a set of APIs that can be called to interact with a machine learning model. The model can be created using a Jupyter Notebook, SPSS Modeler, or AutoAI, and then deployed to an instance of the Watson Machine Learning service. After it’s deployed, you can score data by using a REST call against an API that the service provides.

OpenScale

Watson OpenScale allows monitoring and management of machine learning models that are built on various platforms: Watson Studio, Amazon Sagemaker, Azure Machine Learning, and other popular open source frameworks.

And more

  • AutoAI: A graphical tool in Watson Studio that automatically analyzes your data and generates candidate model pipelines that are customized for your predictive modeling problem.
  • SPSS Modeler: Watson Studio offers a variety of modeling methods taken from machine learning, artificial intelligence, and statistics.
  • Data visualization: Watson Studio provides the ability to visualize data with Data Refinery.
  • Cognos Dashboards: A dashboard for viewing and analyzing data, instead of using pandas or pixiedust.
  • Connections: Watson Studio provides connections to import data from IBM Services (Db2, Cognos, Cloudant, Cloud Object Storage, and many more) and for third-party services (Amazon S3, Hortonworks HDFS, Microsoft SQL Server, Salesforce, MySQL, Oracle, Google BigQuery, and many more).

Watson Machine Learning Accelerator

IBM Watson Machine Learning Accelerator is geared toward enterprises. It is a software bundle that is optimized to run on Power hardware. It bundles IBM PowerAI, IBM Spectrum Conductor, IBM Spectrum Conductor Deep Learning Impact, and support from IBM for the whole stack including the open source deep learning frameworks. It provides an end-to-end, deep learning platform for data scientists.

Watson Machine Learning Accelerator

PowerAI Vision

IBM PowerAI Vision provides a video and image analysis platform that offers built-in deep learning models that learn to analyze images and video streams for classification and object detection. PowerAI Vision is built on open source frameworks and provides sophisticated methods to create models with an easy to understand user interface.

And more

  • Spectrum Conductor: Deploys modern computing frameworks and services for an enterprise environment, both on-premises and in the cloud.
  • Snap ML: A library developed by IBM Research for training generalized linear models with the intent on removing training time as a bottleneck. Snap ML scales gracefully to data sets with billions, and offers distributed training and GPU acceleration.
Steve Martinelli

Call for Code 2019 Finalist: AsTeR prioritizes the severity of emergency calls

There are few guarantees when it comes to natural disaster, but one near certainty is that victims will bombard emergency call centers with requests for help. This can become overwhelming very quickly, and a solution is needed to handle these massive volume spikes. That’s because when these call centers become saturated, lives are put at risk. Technology can play a major role in helping these call centers assess and prioritize calls based on severity rather than simply addressing them based on which victim calls first.

That’s where AsTeR comes in. A team of European graduate students from the University of California, Berkeley — Meryll Dindin, Florian Fesch, Thomas Galeon, Pierre-Louis Missler, and Oskar Radermecker — built this application to filter and assess calls when a human is unable to immediately respond.

Team AsTeR Pictured: (From left to right) Pierre-Louis Missler, Meryll Dindin, Oskar Radermacker, and Florian Fesch

“When we look at how emergency calls are dealt with nowadays, it’s very old technology,” Fesch said. “AsTeR seeks to improve that by bringing state-of-the-art technology to solve this big issue, especially when large-scale disasters happen. We do this by helping call centers figure out which calls are most important and maximizing first responders’ ability to help people.”

The team kicked off the AsTeR idea during a UC Berkeley hackathon in April, which IBM hosted along with Clinton Global Initiative University. It was there that the team first tapped into Galeon’s experience as a first responder in France to design the API. Galeon served as a firefighter in France and trained in a call center.

“That was very interesting for me,” Galeon said, “because I realized that you’d often spend three, four, five minutes with a victim on the line, and at the end of the day you wouldn’t get that much information. Most of it would be just reassuring the person. But sometimes, you could actually trim that down to something like five seconds. That made me realize when there are a lot of calls coming in, typically during a natural disaster, that are just clogging up the line and preventing people from actually getting through to operators and getting the help they’re needing, so that’s really the premise of AsTeR.”

How it works

At its core, AsTeR is an API that call centers would be able to integrate into their existing technology. The core feature that the team developed first, was the natural language processing operators, using IBM Watson Studio. Once you have the audio signal coming in, AsTeR transforms it into a transcript using IBM Watson’s speech-to-text technology.

“So first part is feature extraction, sentiments analysis and keywords extractions,” Dindin said. “And on top of that, we also use our own open source data sentiments. So we just extract words from that in a smart way, transformed with this first match rates, which is fed into a classifier. And we have our own optimization on top of that, so we’ve got the best model. And from that it gives you a class whether the information is relevant.”

AsTeR solution diagram

Calls are then classified based on their severity — whether there are potential fatalities, injuries, disease, flood, fire — and the call is given a priority score. The team uses IBM Cloud Foundry for the SQL database and another IBM service for the speech-to-text endpoint.

The team decided to build AsTeR as an API so call centers could easily tap into it.

“Your emergency call centers already have software,” Dindin said. “Just plug in the endpoint that will be receiving audio signal and just train it for a time. Then you can access a database that is our own, being processed online on a scalable infrastructure — that’s the core product. That’s why an API is so great in getting them back the priority score, the geo-localizations, the hotspots.”

What’s next

Ultimately, the AsTeR team sees this as potentially more than an API and an information collection platform that tackles analyzing emergency calls. A live map feature, which is integrated in the solution, could also display credible data sources such as government, drones or satellites. AsTeR could become a dashboard that aggregates all the information first responders need to organize rescue operations.

The team hopes to bring AsTeR to an open source model and encourage other developers to help them build out the technology. From there, the team would like to roll out the beta version for call centers to test the app.

Mostly, the team is just excited to bring their solution to emergency responders to help when the next disaster strikes.

“Natural disasters can occur everywhere and to anyone; we’re never ready enough,” Radermecker said. “And just by looking at what is being done, I feel like something more is possible, and I think addressing it after the natural disaster is too late. It’s going to take initiative, it’s going to lay the groundwork and doing that early enough is something we can control and it’s crucial.”

Kevin Allen

Call for Code 2019 Finalist: Sparrow Platform provides disaster info and medical help through any app, with or without internet

When the team members of Project Sparrow were brainstorming ideas for this year’s Call for Code Global Challenge, they reflected on their personal experiences with tech and stories they have heard around natural disasters. The team comes from a geographically diverse background with members Jay Lohokare, Ajit Rajurker, Sheng Chen, Mihir Mistry, and Ameya Apte hailing from different areas of the US, China, and India.

Team Sparrow Pictured: Jay Lohokare (left) and Ajit Rajurkar (right)

They also began assessing existing disaster-related applications and found a gap they could fill by creating the Sparrow Platform. Their application, an open-source, AI-enabled platform that serves as a one-stop enabler of medical and psychological well-being during and after disasters, has been named a top-five finalist in the Call for Code 2019 Global Challenge.

“The problem Sparrow seeks to solve is that there are many applications out there which are already doing bits and pieces of work related to disaster recovery,” Rajurker said. “But there is no single platform where people can reach and access all of them. Sparrow is trying to cover those gaps and connect all of them together with the people.”

How it works

The Sparrow Platform is a two-part solution. The first part aims to connect people to all other applications out there so they can get information during a disaster. The team describes the second part as the “Uber for medical help,” connecting victims with live experts who can address disaster-related health issues.

For users, Sparrow is an easy-to-use conversational AI interface, accessible from many existing chat apps. Ask questions, and Sparrow ensures you get what you need. Sparrow achieves this by being a user’s single point of connection with all applications, communication platforms, doctors/experts, etc.

“Think of Sparrow as Alexa, but it is something that’s ubiquitously accessible through many platforms, chat applications, many devices, with or without internet,” Lohokare said. “So you could talk to Sparrow, through any apps you have and ask the question, ‘Hey, are there any updates about the hurricane that’s coming to Florida?'”

In disaster situations, there are two scenarios — internet is available or internet is not available. The Sparrow Platform is always accessible, because it is designed to be available offline. Sparrow converts a smartphone into a mobile tower, and other people’s smartphones that are close to it are turned into mobile towers, allowing for communication with the people around you. Eventually the messages in these disconnected areas reach smartphones that are within range of internet and then the messages reach the rest of the world.

Powered by the IBM Cloud and various IBM Services, this solution is composed of a mesh network hardware and uses IBM Cloud to connect all chat applications with Twilio. The goal is to have open communications even when systems and electricity are down in the event of a natural disaster. The team’s research found that the ratio of victims to medical or rescue aid workers was large, which meant that there aren’t enough rescue aid workers to help or answer every victim’s question.

In addition to the open communications during and after a disaster, the Sparrow Platform acts as a liaison between helpers and victims. The Watson Visual Recognition component of the Sparrow Platform recognizes health symptoms in a victim and recommends steps to treat the symptoms, such as medication or next medical steps. Then a medical worker or expert who is present or nearby can directly comment on the steps and recommendations and provide any followup if needed.

“Sparrow will directly connect that user or that victim with a doctor anywhere in the world and the doctor need not be on the same platform, in the same location, know the same language,” Lohokare said. “Sparrow is essentially like a middleman that is acting like a communications agent that helps both the victims and the doctors.”

Sparrow Platform solution diagram

The team implemented the IBM Cloudant database and IBM Cloud Functions into their solution. They based their framework around the AI services Watson Visual Recognition, Tone Analyzer, and Watson Assistant.

What’s next for Project Sparrow

Since submitting its solution, the Project Sparrow team has run the beta test of the Sparrow platform with a very limited set of users. They already have about 50 doctors on board, with about 80-odd users tapping into the platform. The key thing that the team wanted to accomplish with the beta test was to actually solve user needs and determine ways to improve the user experience.

The team is also in the middle of creating an adoption document or quick start guide that allows any doctor anywhere in the world to quickly onboard Sparrow or create a group of doctors within a city to gather and adopt this solution. They are aiming to bring on more doctors across China, India, and the United States to make sure the platform operates effectively. As they scale up and increase the amount of users, the team will continue to rely on IBM Cloud and incorporate a road platform to reach organizations, such as the United Nations and American Red Cross.

“The vision we have is, down the line, all countries, all systems, essentially will be connected to Sparrow,” Lohokare said. “There will be no 9-1-1, no 100. There will be Sparrow for all countries, all people, all disasters.”

Jina Kim
Liz Klipp

Build a driver distraction application using the Watson Visual Recognition service

Each day in the United States, approximately 9 people are killed and more than 1,000 are injured in crashes caused by distracted driving. This is an alarming number of accidents for any country, and a prevention method must be used.

Artificial intelligence can be used to help with this problem. The government can install IoT cameras around the city to monitor these drivers and promote safer driving by applying strict rules. This blog explains how easy is to implement this project by using IBM Watson Studio on IBM Cloud.

What is Watson Studio

IBM Watson™ Studio is a full suite of data science tools that can be used by any data scientist to implement an AI project with minimal coding. The tool includes tools such as Data Refinery for data preparation, tools like SPSS® Modeler to create a machine learning model without any code. In this blog, I use Watson Visual Recognition to create an image classifier within 30 minutes without any code.

How to create the application

We start off by creating the service on Watson Studio and building the model on it. We then integrate the model into our end application to see it in action. Let’s get started.

Visual recognition using Watson Studio

  1. Create a Watson Studio Visual Recognition project.

    Creating a Studio project

  2. Get the image data set from Kaggle.

  3. Drag the ZIP folder to train the model. You will see all 10 classes of distraction, including one negative class (c0- safe driver).

    Distraction classes

  4. Send the data for training and wait for the results

Dashboard to check class imbalance (optional)

One main problem in data science is class imbalance. Class imbalance occurs when the data in each model class does not have an equal number of examples for training. This can lower the model accuracy and return inaccurate results. A simple dashboard is created to see how varied the image dataset is. This dashboard was created using Cognos Dashboards.

Dashboard

Model integration – Android application

To use the visual recognition model created, we need a front end. This is a sample application created to demonstrate the model integration capabilities of Watson Studio visual recognition models.

Steps

  1. Download the Android project.

  2. Edit the visual recognition credentials in the values/strings.xml file.

    Editing credentials

  3. Edit the model ID in the MainActivity.java file.

    Model ID

Demo of the application in action

Demo

Full demo video

Watch the video to understand how Watson Studio on IBM Cloud is used to build this project.

Anchal Bhalla

Build your own visual recognition app powered by AI

My name is Bryan Escatel, and I’m a senior at Menlo Atherton High School and an intern working with the Cognitive Applications team at IBM. Ever since I started at IBM, I’ve wanted to learn how to make and develop my own app. In the process of developing this app, I’ve had many ups and downs–and I struggled immensely at first. However, thanks to Upkar Lidder’s help, I created a visual recognition app that uses IBM Watson Visual Recognition service to analyze images, objects and other content.

Prerequisites

Install the following:

Steps

The IBM Cloud Mobile services SDK uses CocoaPods to manage and configure dependencies.

Step 1. Open terminal on your MAC OS computer.

Search “Terminal” on the search bar on you computer.

Search Terminal image

Step 2. Clone the Repo

git clone the repo and cd into it by running the following command:

git clone github.com/IBM/watson-visual-recognition-ios.git && cd watson-visual-recognition-ios

Step 3. Install dependencies with Carthage.

Run the following command to build the dependencies and frameworks:

carthage update --platform iOS

Note: Carthage can be installed with Homebrew.

Step 4. Create an IBM cloud service

Create the following services: Watson Visual Recognition. Copy the API Key from the credentials and add it to Credentials.plist

<key>apiKey</key> <string>YOUR_API_KEY</string>

Step 5. Run the app with Xcode.

Launch Xcode using the terminal: open “Watson Vision.xcodeproj”

Step 6. Test app in simulator.

To run the simulator, select an iOS device from the dropdown and click the ► button

Test app image

Now you’re able to click and drag photos into the photo gallery and select those photos from the app.

Step 7. Run the app on an iOS.

Since the simulator does not have access to a camera, and the app relies on the camera to test the classifier, you should run it on a real device. To do this, you’ll need to sign in the application and authenticate with your Apple ID:

  • Switch to the General tab in the project editor (The blue icon on the top left).

  • Under the Signing section, click Add Account.

Sign in image

After signing in with your Apple ID and password, you’ll need to create a certificate to sign your app (in the General tab) and follow the next few steps:

  • In the General tab of the project editor, change the bundle identifier to: com.<YOUR_LAST_NAME>.Core-ML-Vision.

Bundle identifier image

  1. Select the personal team that was just created from the team dropdown.
  2. Plug in your iOS device.
  3. Select your device from the device menu to the right of the build and run icon.
  4. Click build and run.
  5. On your device, you should see the app appear as an installed appear.
  6. When you try to run the app the first time, it will prompt you to approve the developer.
  7. In your iOS settings navigate to General > Device Management.
  8. Tap your email, and tap trust.

You’re now ready to use the app.

Term details:

  • General: Watson Visual Recognition’s default classification. It will return the confidence of an image from thousands of classes.
  • Explicit: Returns percent confidence of whether an image is inappropriate for general use.
  • Food: A classifier intended for images of food items.
  • Custom classifier(s): Gives the user the ability to create their own classifier.

Demo

Example 1: The object scanned was a can. As you can see, IBM Watson detected a 67 percent probability of a can.

Can image

Example 2: The object scanned was a shoe.

Shoe image

As you can see, IBM Watson detected an 81 percent probability of a shoe.

Result image

Updating your app is fairly simple. All you need to do is teach and train your model. The app and model will update all at once when you push the train button so it’s all ready to go.

Bryan Escatel
Max Katz
Peter Moskovits

A Watson Visual Recognition recommendation example

The e-commerce boom makes the online environment more competitive. Internet retailers seek competitive advantages and a personalized experience for their clients. To achieve a truly personalized experience, you must know an individual user’s choices or preferences in detail. How cool would it be if you have a recommendation system that recommends products based on a person’s features (like their age or gender)?

A good recommendation system can typically boost your sales. When it comes to apparel or jewelry, using a good recommendation engine with images is very important. Usually, people show greater interest in the products when they get a personalized recommendation.

For example, if a person wants to buy a piece of jewelry or eye glasses, they would select an item that they like or that looks good on them. So, providing a personalized recommendation of products based on their features helps them in getting a product that they would like.

To help with this, we can use the Watson™ Visual Recognition service. This service helps you with recognizing a person’s features such as their age or gender by using the image of the person as input. We developed a hybrid mobile application that uses MobileFirst Foundation integrated with a recommendation system that takes an image of the user as input and detects their features with the help of a Watson Visual Recognition model. Based on these features, the recommendation engine returns a personalized recommendation of jewelry products. This can be extended to any other product.

This recommendation engine is implemented using an IBM Cloud account, Python 3, and Java 1.8.0.

We have created a sample application for you that you can use as a foundation and build on top of it. Check out Recommendation system based on visual recognition. It takes approximately 15 minutes to implement it.

This blog is just an introduction to how we can use Watson Visual Recognition for personalized recommendations. To explore more examples, check out Integrate a virtual mirror with e-commerce products and Build a ‘try-and-buy’ mobile application with augmented reality capabilities for a furniture store.

Have fun!

Rahul Reddy Ravipally

Live coding event: Using Node-RED and AI to analyze social media

Natural disasters and social media

With social media so prevalent today, people all over the world can connect instantly online through various social media platforms. And though there are negative aspects to social media, it has proved to be helpful after a natural disaster. People can mark themselves as safe, connect with others to let them know where they are, and also raise awareness and funds for charitable donations.

When building your Call for Code submission, it is obviously useful for you to know important weather data. But consider also how technology today can be improved upon when it comes to social networks and other forms of communication through multimedia.

Mark your calendar for a live coding tutorial on May 14

With less than two months until Call for Code submissions close on July 29, we have an exciting live coding event coming up next week around social media. In this live tutorial, John Walicki (STSM and CTO of Global Developer Advocacy) will demonstrate how Node-RED flows paired with AI can analyze social media during and after a natural disaster. Walicki will walk you through the steps to create a sample application that uses Node-RED, Watson, and Twitter to look at how survivors of a natural disaster use social media to post their sentiments, pictures of locations and damages, and also how they can get connected to first responders.

After a disaster, these methods could be useful to further understand how natural disasters affect mental health and whether social media helped first responders locate survivors. Images shared on social media by individuals could also prove useful in determining specific types of terrain or infrastructure impacted by natural disasters.

The livestream is over now, but you can still catch the replay here! And make sure to bookmark IBM Developer’s Twitch channel to catch future streams!

Resources

IBM Developer Staff