Build a dynamic avatar-based mobile chatbot powered with AI and AR capabilities

Summary

This code pattern demonstrates how to build a dynamic, real-time avatar-based chatbot, which is an Android mobile application with AI and AR capabilities. This mobile app dynamically fetches the information using APIs in Watson Assistant by using webhooks (in IBM Cloud Functions).

Description

Customers are quickly becoming used to communicating with chatbots while at a store or while on the company’s website. Adding AI and AR capabilities to chatbots create a range of possibilities that have not been used before. By taking advantage of the mobile, AI, and AR capabilities that are readily available in the IBM Cloud catalog — IBM Cloud Functions, IBM Mobile Foundation, Watson Assistant, and Watson Text to Speech and Speech to Text services — you can quickly build an avatar-based chatbot that runs on your Android mobile device.

Flow

Architecture diagram for building a dynamic avatar-based mobile chatbot

  1. User asks a query to the AR Avatar (mobile app).
  2. Mobile application converts the speech to text using the Watson Speech to Text service and sends the query to IBM Mobile Foundation.
  3. IBM Mobile Foundation securely passes the query to Watson Assistant.
  4. Watson Assistant triggers appropriate IBM Cloud Functions, based on the query received.
  5. Based on the query, IBM Cloud Functions triggers the appropriate service or services, in the back-end server application.
  6. The back-end server application returns the response for the query.
  7. IBM Cloud Functions sends the response to Watson Assistant.
  8. Watson Assistant frames a response to reply to the user and sends it to IBM Mobile Foundation.
  9. IBM Mobile Foundation securely sends the response to the mobile application.
  10. Mobile application converts the text into speech using the Watson Text to Speech service.
  11. Mobile application replies to the user using the AR Avatar.

Instructions

Ready to get started? Check out the README for detailed instructions how to:

  1. Create Watson services with IBM Cloud.
  2. Update the details in the back-end application.
  3. Deploy the back-end application.
  4. Set up IBM Cloud Functions.
  5. Set up Watson Assistant.
  6. Set up IBM Mobile Foundation Server and CLI.
  7. Set up Google Cloud Anchors.
  8. Configure the Android mobile app.
  9. Build and run the Android mobile app.