Create a Kubernetes speech assistant

Get the code View the demo

Summary

Imagine being away from your computer when you realize you need to deploy a back end on a Kubernetes cluster that you have not yet provisioned. Knowing that provisioning a cluster could take several minutes after selecting the right configuration for your task, you visualize sitting in front of your computer staring idly at the screen, waiting for the cluster to be initialized and wishing that you could start work immediately.

Not in my world!

This code pattern shows how to build a speech assistant for Kubernetes in the cloud.

Get examples of building applications with Watson™ Assistant and the Kubernetes Service on IBM Cloud. Learn how to build an Android application using the IBM Identity and Access Management, OpenID, and a Node.js back-end server connected to Watson Assistant.

Description

This developer code pattern demonstrates a Kubernetes speech assistant application for an Android mobile device. You can simply talk on your phone in natural language to provision, view, and manage your Kubernetes clusters without having to do it manually on the cloud interface.

The pattern showcases an Android app that mobile device users interact with and a Node.js back-end server that holds the application logic and talks to the IBM Cloud Kubernetes Service. The pattern demonstrates use of the IBM Identity and Access Management using the OpenID Connect specifications for a native application. It also showcases Watson Assistant to understand the natural language spoken by the mobile users, holding the context of the speech and converting the speakers’ intent into executable Kubernetes commands that are run on IBM Cloud.

In this code pattern, you learn the following skills:

  • Create an Android application connected with the OpenID Connect specifications for IBM Identity and Access Management.
  • Develop a Node.js server using Express.js, which interfaces with the Watson Assistant and the IBM Cloud Kubernetes Service.
  • Set up Watson Assistant to create intents, entities, and a dialog flow.
  • Convert speech to text and text to speech, natively for an Android application.
  • Manage OpenID Connect authorization tokens on an Android application.
  • Deploy a Node.js back-end server on the IBM Cloud Kubernetes Service.

Flow

Kubernetes speech assistant architecture flow diagram

  1. A user logs into an IBM Cloud account on the Android app.
  2. The user is redirected to the IBM Cloud login page on their phone browsers, using the OpenID Connect protocol from IBM Identity and Access Management.
  3. After successful authentication, a request is initiated for a generic IBM Identity and Access Management token.
  4. The request sends the authorization code retrieved from Step 2 to the OpenID Connect Protocol to get the generic token.
  5. A request is initiated to get the list of accounts associated with the IBM Cloud user ID.
  6. The request sends the generic IBM Identity and Access Management access token to the IBM Account Management API to retrieve the list of accounts associated with the user ID. The user selects an account from the list.
  7. A request is initiated to get an IBM Identity and Access Management token for the selected account.
  8. The request sends the generic IBM Identity and Access Management refresh token and the selected account ID to the OpenID Connect protocol to retrieve a token linked to the selected cloud account.
  9. The token object persists on the user device for future use, until expiration.
  10. A request is initiated to get the list of resource groups for the selected account.
  11. The request sends the account-specific access token to the IBM Resource Controller API to retrieve the list of resource groups associated with the account. The user selects a resource group from the list.
  12. A request is initiated to get a Watson Assistant session.
  13. The request talks to the Watson Assistant API SDK through the Node.js server to get a new Watson Assitant session.
  14. The Android application configuration is completed, and the view changes so that user can start talking with the application.
  15. The user taps the mic button and talks to send a speech command to the application.
  16. The speech is converted to text using the native Android Speech to Text converter.
  17. A fresh token request is issued. (It uses the previous refresh token if the access token has expired.)
  18. That token and the user text input are both sent to the Node.js back-end server hosted on a Kubernetes cluster on IBM Cloud.
  19. The user text input is sent to Watson Assistant to extract knowledge about the text and entities present.
  20. If the context for the user input is complete and all knowledge required to run the Kubernetes command is obtained, the following info is sent to the Kubernetes service API on IBM Cloud: a request for the user action, the user access token, other parameters for running the command.
  21. The result of the executed command (or a follow-up question to get further details about the user request) is sent to the Android application.
  22. The text received by the application is converted to speech by using the native Android Speech to Text converter.
  23. The speech is relayed to the user to continue the conversation.

Instructions

Ready to get started? Find detailed technical steps for this code pattern in the README.md file in the GitHub repository.

  1. Clone the repo.
  2. Set up IBM Identity and Access Management.
  3. Create IBM Cloud services.
  4. Configure Watson Assistant.
  5. Deploy NodeJS server to Kubernetes.
  6. Configure the Android application.
  7. Run the application.