Create a virtual reality speech sandbox  

Add Watson-powered natural language voice interaction from the Unity environment

Last updated | By Scott D’Angelo


This developer journey will show you how to build advanced interactive speech systems for virtual reality with just two Watson services: Watson Speech-to-Text for transcription and Watson Conversation for parsing the meaning of the words. Discover how to leverage the Watson Unity SDK to implement the services right from the Unity development environment.


Virtual reality (VR) enables users to feel like they truly inhabit a different space. In a VR environment, speech is a more natural interface than other methods for certain interactions. You don’t want to pause an experience to stare at a control or, heaven forbid, type a command; you want to be in the moment. The ability to simply speak instructions keeps you in that moment and helps provide an entirely new dimension of immersion for users.

By learning how to add speech controls to VR environments, you can build more richly interactive, immersive experiences — and position your own skills for the next big technology revolution. When you complete this developer journey, you will understand how to add IBM Watson Speech-to-Text and Conversation services to a virtual reality environment built in Unity, the popular 3D development platform.

There are several popular VR head-mounted devices that offer users powerful immersive experiences. Their popularity and versatility make them ideal candidates for speech interaction. This developer journey shows you how to implement speech controls for Google Cardboard and HTC Vive, two of the most popular head-mounted VR devices.


  1. The user launches the Speech sandbox using Unity and a VR hardware system.
  2. While speaking into the VR unit, the user issues commands that are translated using Watson’s Speech-to-Text service.
  3. The Watson Conversation service provides voice communication with the user.

Related Blogs

Newsletters: The Curious Developer’s Best Friend

The great thing about software development is that there is always something new to learn! The terrible thing about software development is that there is always something new to learn! Luckily, there are tons of wonderful people sharing their knowledge every week in helpful and entertaining newsletters ... and unfortunately, it can be really easy...

Continue reading Newsletters: The Curious Developer’s Best Friend

Related Links