Convert text to speech using Watson Libraries

The IBM Watson Text to Speech Service is a cloud service that lets you convert written text into natural-sounding audio in various languages and voices within an existing application or within watsonx Assistant. Watson Text to Speech enables fast and accurate speech transcription in multiple languages for various use cases, including customer self-service apps that:

• Build a mobile to-do app that captures voice memos to save as written to-do items
• Use as a simple ready-to-use model to capture English voices without any customization
• Use as a lightweight feature that can be invoked from the constrained mobile device

In this tutorial, walk through the steps of starting a Watson Text to Speech Service, connecting to it through Docker in your local system, processing the data set, and using the Watson Text to Speech Service. The tutorial also shows you how to modify speech synthesis characteristics using different parameters.

Prerequisites

To follow this tutorial, you must have:

• Your entitlement key to access the IBM Entitled Registry
• A Watson Text to Speech trial license
• Docker installed

Note: Podman provides a Docker-compatible command-line front end. Unless otherwise noted, all of the Docker commands in this tutorial should work for Podman if you simply alias the Docker CLI with the alias docker=podman shell command.

Step 1. Set up the environment

Step 1.1. Log in to the IBM Entitled Registry

The IBM Entitled Registry contains various container images for the Watson Text to Speech Service. After you obtain the entitlement key from the container software library, you can log in to the registry with the key and pull the container images to your local machine. Use the following command to log in to the registry.

echo $IBM_ENTITLEMENT_KEY | docker login -u cp --password-stdin cp.icr.io

Step 1.2. Clone the sample code repository

1. Clone the sample code repository.

   git clone git@github.com:ibm-build-lab/Watson-Speech.git
   

2. Go to the directory that contains the sample code for this tutorial.

   cd Watson-Speech/single-container-tts
   

Step 1.3. Build the container image

Build a container image with the provided Dockerfile with two pretrained models (en-us-michaelv3voice and fr-ca-louisev3voice), which include support for two different languages: English (en_US) and Canadian French (fr_CA). You can add more models to support other languages by updating the provided Dockerfile as well as the env_config.json and sessionPools.yaml files in the config directory.

docker build . -t tts-standalone

Step 1.4. Run the container to start the service

Run the container on Docker using the container image that was created in the previous step to start the Watson Text to Speech Service.

docker run --rm -it --env ACCEPT_LICENSE=true --publish 1080:1080 tts-standalone

The service runs in the foreground. Now, you can access this service in your notebook or local machine.

Step 2. Watson Text to Speech analysis

Step 2.1. Data loading and preprocessing on text

Watson Text to Speech offers parameters for various text-to-speech syntheses for an entire request: rate_percentage, pitch_percentage, and spell_out_mode. By using these parameters, you can modify the audio output.

1. Import and initialize some helper libraries that are used throughout the tutorial.

   from matplotlib import pyplot as plt
   import IPython.display as ipd
   import librosa
   import pandas as pd
   %matplotlib inline
   import soundfile as sf
   

2. Create a custom function to plot the amplitude frequency.

   def print_plot_play(fileName, text=''):
       x, Fs = librosa.load(fileName, sr=None)
       print('%s Fs = %d, x.shape = %s, x.dtype = %s' % (text, Fs, x.shape, x.dtype))
       plt.figure(figsize=(10, 5))
       plt.plot(x, color='blue')
       plt.xlim([0, x.shape[0]])
       plt.xlabel('Time (samples)')
       plt.ylabel('Amplitude')
       plt.tight_layout()
       plt.show()
       ipd.display(ipd.Audio(data=x, rate=Fs))
   

3. Load the text data.

   consumer_df = pd.read_csv('consumer_data.csv')
   

4. Preprocess the text. Speech synthesis services accept the data in a JSON format. There are many escape characters that appear in the text that are not valid for a JSON string. Therefore, you must replace those characters.

   def clean(doc):
       stop_free = " ".join([word.replace('X','').replace('/','').replace("''",'').replace(',','').replace(':','').replace('{','').replace('}','').replace('"','') for word in doc.split()])
       return stop_free
   

Step 2.2. Setting up the service

1. Set up the parameters for using the Watson Text to Speech Service.

   headers = {"Content-Type": "application/json", "Accept":"audio/wav"}
   params ={'output': 'output_text.wav'}
   text_to_speech_url ='http://localhost:1080/text-to-speech/api/v1/synthesize'
   

2. Use the following example text in the subsequent steps to try out the Watson Text to Speech Service.

   data ='{"text": "Text to Speech service provides APIs that use IBM\'s speech-synthesis capabilities to synthesize text into natural-sounding speech in a variety of languages, dialects, and voices"}'
   

3. Create a custom function to get speech from the text using the Watson Text to Speech Service.

   def getSpeechFromText(headers,params,data,file_name):
       request =requests.post(text_to_speech_url,headers=headers,params =params,data=data)
       print(request.status_code)
       with open(file_name, mode='bx') as f:
           f.write(request.content)
   

4. Send the example text to the Watson Text to Speech Service and use the custom function to get the speech output.

   file_name = 'text_to_speech_sample1.wav'
   result = getSpeechFromText(headers, params, data, file_name)
   print_plot_play(file_name)
   

   

Step 3. Modify speech synthesis characteristics

The Watson Text to Speech Service includes query parameters that you can use to globally modify the characteristics of speech synthesis for an entire request.

• rate_percentage
• pitch_percentage

Use the rate_percentage parameter to adjust the rate percentage and get the output from the Watson Text to Speech Service.

params = {'voice': 'en-US_AllisonV3Voice', 'rate_percentage': -5}
file_name = "text_to_speeh_rate.wav"

Use the pitch_percentage parameter to adjust the pitch percentage and get the output from the Watson Text to Speech Service.

params = {'voice': 'en-US_AllisonV3Voice', 'pitch_percentage': -10}
file_name = "text_to_speeh_pitch.wav"

Step 4. Case study: Playback customer calls

By using text-to-speech services, you can create a box model to capture English voices without any customization. The following example shows passing multiple texts into a speech synthesis service to save multiple files.

customer_call_dir = 'customer_call/'
params = {'voice': 'en-US_AllisonV3Voice'}
i = 0
for data in consumer_clean_list:
    if i < 5:
        file_name = customer_call_dir + "consumer_call" + str(i) + ".wav"
        data1 = '{"text": "' + data + '"}'
        getSpeechFromText(headers, params, data1, file_name)
        i = i + 1
        print("Saving consumer calls --- " + file_name)

Conclusion

This tutorial walked you through the steps of starting a Watson Text to Speech Service, connecting to it through Docker in your local system, processing the data set, and using the Watson Text to Speech Service to convert text to speech. This tutorial also shows you how to modify speech synthesis characteristics using different parameters. To try out the service, work through the Watson Text to Speech Analysis notebook.