Having worked in the JEE (Java Platform, Enterprise Edition) space for 20 years, I was excited to apply Watson APIs to building cognitive applications with the Java community. As luck would have it, the chance did arrive when Devoxx Founder, Stephan Janssen expressed an interest in building a cognitive search engine that would allow users to glean answers from the wealth of information contained in Devoxx and Voxxed resources.

I set to work on designing the application and cognitive search engine. We first used a Speech to Text service to convert the videos to text and stored the text documents in Concept Insights on Bluemix. I worked with Stephan to add metadata such as author of the video or article, the publication date and author sentiment using the Combined Call service from AlchemyAPI. We then added the tone of the narrative using Bluemix Tone Analyzer and a thumbnail tag using the Visual Recognition service, phew!

The fun part is that Stephan and I open sourced this application at Devoxx UK in June 2016 and two Java Champions, Jim Weaver and Daniel Deluca joined us on this project! It was a lot of fun to outline next steps in our presentation session, check out the video.

 

Using the ever-popular NAO robot, here’s how the process flow went for our Devoxx use case:

Ask a question to NAO Robot -> Convert Speech to Text -> Send question to Conversation service -> Receive a response using the Conversation flow, else by calling Retrieve and Rank, if question is out of scope for Conversation service -> Convert Text to Speech -> Hear the answer from NAO Robot


500563-ibm-watson-nao-robot

 

We could not continue to use Concept Insights API, since it was sunsetted so now it was time to play with Retrieve and Rank! Stephan used the Web UI in Retrieve and Rank to upload Voxxed articles and a set of questions. He trained the data set by ranking the uploaded questions. I in turn created conversation flows to ask, How to, What and Where questions. A user can now converse with the cognitive application, just as if you and I were talking to each other. Some conversations that go back and forth were set up to maintain a context, similar to the human brain.

Now came the time to setup a REST endpoint (/inquiry) for sending inquiries to Conversation. Jim wrote the Spring Boot app that provides a stateless REST client, taking in questions and retrieving answers from Conversation flows which in turn may call Retrieve and Rank for questions that need to be looked up in the Devoxx corpus. The app is running in Bluemix: askdevoxxwatson.mybluemix.net This application leverages a wide variety of technologies including IBM Watson, Cloud Foundry, Java, Spring Boot, Microservices, Node.js, Amazon Alexa, AWS Lambda, and SoftBank Robots.

Here’s an example conversation that a user can have with AskDevoxx, either through a user interface that resembles text messaging or through speaking out loud via a voice interface such as a NAO Robot or an Amazon Echo.


exampleconversation-askdevoxx

 

Daniel then integrated the NAO by calling /inquiry endpoint and now you can ask questions to the NAO.  To see all the action and ask your questions to the NAO enhanced with cognitive services, come on out to IBM’s keynote at JavaOne on Thursday, 22 September!

We have open sourced the GitHub repo, https://github.com/devoxx/AskDevoxx for developers to play with and hope you will extend the code to achieve the, “Art of the Possible!”

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

Save

1 comment on"AI AI, Captain – A Watson DevoxxEd use case from Sandhya Kapoor"

  1. this is nice one. thanks sandhya

Join The Discussion

Your email address will not be published. Required fields are marked *