Overview

Skill Level: Beginner

Beginner

This tutorial will help you to connect your Sphero-BB8 robot to Watson IoT Platform and control it through voice commands. For example, change the color of the BB8 to Red, Blue or Green. Also, instruct the BB8 to run around or Spin for a specified time.

Ingredients

Hardware Requirements

Software Requirements

** Service fee may apply. Estimated Monthly Costs: < $40

Step-by-step

  1. Introduction

    Playing with BB-8 droid is ultimate fun. In the previous recipe, we showed how one can control the BB-8 with a Node-RED application over Internet. And in this recipe, we want to take it one step further and control the robot by voice commands through Watson IoT Platform. To demonstrate the same, we replace the Bluemix Node-RED application in the previous recipe with a Java application that will stream the voice and sent it across to Watson IoT Platform. The remaining components of the previous recipe stay as is.

    Following are the list of components involved in this recipe,

    archi-voice

    As shown, the user with a mic sends one of the following commands to BB-8 droid through Watson IoT Platform.

    • Command to change the color of the BB8 to Red, Green or Blue
    • Command to Spin
    • Command to Roll

    A Java application developed using the Watson Developer Cloud Java SDK, streams the voice from the user and invokes the Watson Speech to Text(STT) service to convert the voice to text. The returned text then parsed and mapped to one of the above commands. In this sample, we showcase a simple mapping wherein if a sentence contains a word “Red”, then it maps to the color changing command – Red. The command is then sent to the Watson IoT Platform through MQTT. The Java application uses the Watson IoT java client library for sending the commands to the Watson IoT Platform.

    On the device side, Node-RED application will receive the command from the Watson IoT Platform and forward it to BB-8 droid using the Cylon APIs. Upon receiving the command, the BB-8 droid performs the respective action.

  2. Connect BB-8 to Watson IoT Platform

    The previous recipe made use of the Create Toolchain button to help deploy the necessary services on to Bluemix. Click on the Create Toolchain button provided below, provide a Custom Name to your application and choose to click on Create button, to quickly deploy the Watson IoT Platform – Node-RED Bluemix starter app. Post successful deployment, you shall have both the above mentioned services, up & running on Bluemix along with a Cloudant NoSQL DB at your disposal.

    Toolchain-8

    Note: If you are a User using the United Kingdom Region in your Bluemix environment, then please make use of the steps mentioned in the IoT Recipe Deploy Internet of Things Platform Starter service on Bluemix to deploy the setup. Alternatively, you can also try using the Deploy to Bluemix button, to deploy the setup under your United Kingdom Region, provided your Jazzhub account is validated. Users of US South Region can ignore this step.

    deploy

    In case, you have not followed the previous recipe, perform the following steps to connect BB-8 to Watson IoT Platform and subscribe for the commands:

    1. Carry out the steps mentioned in this section Install Node-RED and Cylon driver from the part-1 of this recipe to install necessary modules to your laptop that connects to BB-8.
    2. Carry out the steps mentioned in this section Connect BB-8 from your laptop to connect BB-8 to your laptop.
    3. In order to send BB-8 events and receive commands from Watson IoT Platform, we need to register the BB-8 device first, in the IBM Watson IoT Platform. Carry out the steps mentioned in this section Register your BB-8 Device in Watson IoT Platform from the previous recipe to do the same.
    4. Carry out the steps mentioned in this section Connect BB-8 to Watson IoT Platform to connect BB-8 to Watson IoT Platform and send collision events.
    5. Carry out the steps mentioned in this section Create a Node-RED flow to receive commands from Watson IoT Platform to configure the Node-RED application to subscribe to commands from the application.

     

    In this step, we showed how to connect BB-8 to Watson IoT Platform and make it ready for the commands.

  3. Create Speech-to-Text service in Bluemix

    In this step, we will create the Speech-to-Text service that is required to convert user voice command to text.

    1. Open your browser and go to Bluemix. If you are an existing Bluemix user, log in as usual. If you are new to Bluemix you can sign up for a free 30-day trial.
    2. Click this link to create the Speech-to-Text service.
    3. Specify a name and click “create” to create the service.
    4. Click on Service Credentials as shown below and note down the credentialscredentials

    At this step, we have successfully created the Speech-to-Text service.

  4. Setup the application environment

    In this step, we will start the application that streams the voice, converts to text and sends to Watson IoT Platform.

    You can use either the same Laptop that we used to interact with BB-8, or use any other device to initiate the voice command.

    1. Download and install Maven and Git if not installed already.
    2. Obtain the Voice Transmission sample by cloning the iot-cognitive-samples project using git clone as follows:
      git clone https://github.com/ibm-watson-iot/iot-cognitive-samples.git
    3. Navigate to the command transmission project as shown below:

      cd iot-cognitive-samples/voice-transmission/command-transmission
    4. Build the sample using the Maven command as follows:
      mvn clean package
    5. Once the build is successful, update the application.properties file present under target/classes directory, with the Watson IoT Platform API key details, the authentication credentials of Speech To Text service and BB-8 device details.
      ## Mandatory fieldsid = <random-id>Organization-ID = <Your Watson IoT Platform Organization id>Authentication-Method = apikeyAPI-Key = <Token that you generated in Watson IoT Platform>Authentication-Token = <Token that you generated in Watson IoT Platform>## Device on behalf the application needs to subscribe to events or publish eventsDevice-Type = <Device type of BB-8 that you noted in the registration step>Device-ID = <Device Id of BB-8 that you noted in the registration step>## Speech To Text Credentialsstt-username = <Username of the STT service>stt-password = <Password of the STT service>
    6. Navigate to the source directory, where you had performed the Maven build command, i.e:
      cd iot-cognitive-samples/voice-transmission/command-transmission
    7. Execute the following maven execution command, to execute the Command Transmission sample:
      mvn exec:java -Dexec.mainClass="com.ibm.watsoniot.BB8VoiceController"
    8. As the execution begins, you should see a series of messages, that mentions successful connection to the Watson IoT Platform and waits for Audio Input from the user:
      Jul 27, 2016 5:38:10 PM com.ibm.iotf.client.AbstractClient createClientINFO: main: Org ID    = xxxxxx         Client ID    = a:xxxxxx:rj5jh322Jul 27, 2016 5:38:11 PM com.ibm.iotf.client.AbstractClient connectINFO: main: Connecting client a:rj5jh3:rj5jh322 to ssl://xxxxxx.messaging.internetofthings.ibmcloud.com:8883 (attempt #1)...Jul 27, 2016 5:38:13 PM com.ibm.iotf.client.AbstractClient connectINFO: main: Successfully connected to the IBM Watson IoT Platform

    In this step, we have successfully started the application and connected to Watson IoT Platform. In the next step, we will show how to send the voice commands.

  5. Initiate the commands

    Now, hold the Mic at an appropriate distance from the mouth and speak into it, to initiate the commands. This Audio Streaming shall be open for the next 1 hour, as per the current sample, which can be always tweaked to increase or decrease the timeline.

    Change the color of BB-8 through Voice

    Speak any sentence involving the word Red, for example, “Change the color to Red?“, and observe that the color of the BB-8 changes to Red. You can also observe the transcript and the command sent to the device in the application console.

    Got tanscript: Change the color to Red
    Publishing command: {"action":"color","values":{"red":255,"blue":0,"green":0}}

    Similarly, use the word Green and Blue to change the color of the BB-8 to Green and Blue respectively. Note that the sample supports only Red, Green and Blue colors. Statements involving any other colors are just ignored.

    Instruct the BB-8 to Spin

    Speak any sentence involving the word Spin, for example, “Can you Spin?“, and observe that the BB-8 starts spinning for a configured amount of time. Node-RED flow on the device side can be tweaked to increase/decrease the spin time. You can also, observe the transcript and the command sent to the device in the application console,

    Got tanscript: can you spin
    Publishing command: {"action":"spin"}

    Instruct the BB-8 to Roll

    Speak any sentence involving the word Roll, for example, “Roll around“, and observe that the BB-8 starts running for a configured amount of time. Node-RED flow on the device side can be tweaked to increase/decrease the roll time. You can also, observe the transcript and the command sent to the device in the application console,

    Got tanscript: roll around
    Publishing command: {"action":"roll"}

    In this step, we have successfully sent the voice command to control BB-8 droid through Watson IoT Platform.

  6. Conclusion

    In this recipe, we showcased how one can control the BB-8 droid using voice. As the process, we showed the possibilities of live streaming of the audio and converting it to text, on the fly, without having to write it on the disk, thus saving time and enhancing the performance. This scenario can be easily extended to interact/control remote devices like smart ovens through voice.  Developers can look at the code made available in the Github repository to understand what is happening under the hood. Developers can consider this recipe as a template for sending voice commands over IBM Watson IoT Platform.

    As a next step go through list of cognitive recipes built around Watson IoT Platform and Bluemix.

    Also, refer to the WIoTP Documentation to further understand the IBM Watson IoT Platform capabilities, to come up with your own Application, Device & Gateway samples in the world of Internet of Things.

1 comment on"Control Sphero-BB8 by Voice through IBM Watson IoT Platform"

  1. […] learn how to bring The Force to life yourself using IBM Watson IoT, check out the recipe, “Control Sphero-BB8 by Voice through IBM Watson IoT Platform,” on […]

Join The Discussion