Integrate watsonx Assistant with watsonx.ai foundation models

Watsonx is IBM's enterprise-ready Data and AI platform for training, testing, and deploying AI technologies, including generative AI using Large Language Models (LLMs) using the watsonx.ai component of watsonx. The extensibility of IBM watsonx Assistant provides a simple way to perform an integration between a dialog flow and generative AI inferencing services in watsonx.ai. This integration brings generative AI features directly into watsonx Assistant.

This hands-on tutorial explains how to create the integration and use it in a simple use case. It is outside the scope of this guide to provide an introduction to all watsonx Assistant features. For that, check out the documentation

Prerequisites

To follow this tutorial, you need:

• A watsonx Assistant instance on IBM Cloud. This can be any service plan, including lite.
• Access to a watsonx.ai project.

Estimated time

It should take you approximately 20 minutes to complete this tutorial.

Steps

Step 1. Obtain the OpenAPI definition of the watsonx.ai generation endpoint

Before you can add the integration for watsonx.ai to a watsonx Assistant virtual assistant, you must download the OpenAPI definition for the watsonx.ai foundation model inferencing service. An example definition file is available in the assistant toolkit GitHub repository. For this tutorial, you use a version that is modified with the addition of support for the decode_method parameter of the API. Download this version of the watsonx-openapi.json file to your workstation.

Step 2. Add an empty virtual assistant to the watsonx Assistant instance

For initial experimentation with the integration, use a new virtual assistant. If you have a newly created watsonx Assistant service instance, it will not have any assistants defined. Create one using the Create Assistant wizard that uses the name Watsonx integration, and then select web as the deployment location.

The values that you provide in the other parts of the wizard do not matter. If you have other assistants defined already in your instance, add one following the documentation.

Step 3. Create a watsonx.ai API key

If you have previously created an API key for calling watsonx.ai from a notebook or just the REST endpoint with curl, you can skip these steps and reuse that API key. Otherwise:

1. Log in to watsonx.ai, and make sure that the same account as the Watson Machine Learning instance for your watsonx.ai project is selected.

2. To generate an API key from your IBM Cloud user account, go to Manage access and users - API Keys, and click Create. In the pop-up window, provide a name like watsonx-apikey and a short description, then click Create.

   

3. Immediately make a copy of the API key and store it in a secure location as there is no way to retrieve it again.

Step 4. Getting your watsonx.ai project ID

1. In the watsonx.ai page, select the left 4-bar pull-down menu, expand Projects, and then click View all projects.

2. From the list, select the project that you are using for the integration.

3. Click the Manage tab, and ensure that General is selected. Copy the Project ID to a location where you can access it later. This is not sensitive like the API key, so it can be in a notepad or something equivalent.

   

Step 5. Add the custom extension for watsonx.ai to the Integrations catalog

1. Return to your browser tab with the assistant that you created. In the assistant left navigation panel, go down to the lower section and click Integrations.

2. Scroll down to the Extensions section, and click Build custom extension.

   

3. Review the getting started guide, and click Next.

4. On the basic information tab, name the extension watsonx, and provide an optional description. Click Next.

5. Either drag the file or use the file browser to select the watsonx-openapi.json file, and then click Next.

   

6. Review the information displayed, and click Finish. The integration catalog shows the integration.

   

7. Click Add in the lower-right corner of the new integration tile, and confirm Add when prompted.

8. Review the Get started information, and click Next.

9. In the Authentication panel, select Oauth 2.0, enter the API key that you created earlier, keep the remaining default values, then click Next.

   

10. On the last page, review the POST request parameters and response values. The assistant can set parameters in the request before calling the extension and will parse the response to show the user the output. Click Finish, then click Close if you are not returned to the Extensions section of the Integrations panel.

Congratulations, you now have a way to make a functional integration from an Action in an assistant with a watsonx generative AI endpoint.

Step 6. Create your action

Now, it's time to create your action and have it use the integration to watsonx. To get started, you create an action that uses generative ai to draft a marketing message from your prompt.

1. In the browser tab with watsonx Assistant, click Actions in the upper left.

2. In the main panel, click Create action.

3. Select Start from Scratch.

4. For what the customer says, enter Marketing Message, and click Save.

   

5. Click the pencil icon to set a step title. Use Prompt for a prompt as the title.

6. For what the assistant says, you can either make it a short message or be more prescriptive. Humans, like generative AI models, can often do better when requests are specific. For example:

   "Please provide a prompt for the model to use to create the marketing message. Type as much as you like, and include lists of details for me to use with the model."

   

   Expand Define customer response, and specify that the assistant will expect a response from the user as Free Text input. When you click Free Text, the UI shows User enters free text below the Assistant says panel. When this is shown, click New Step to configure the next part of the Action.

7. Name the next step Call watsonx.ai endpoint by clicking the pencil (edit) icon. In the And then section, select Use an extension.

   

8. Select watsonx for the extension to use. Then, select the Generation operation to show the parameters that can be sent to watsonx.ai on every invocation. For version, choose Enter text, and then provide 2023-05-29. For input, choose Action Step Variables, and then choose 1. Prompt for a prompt, continuing in the same way. Set the model_id to google/flan-t5-xxl and the project_id to your watsonx.ai project ID saved previously. Don't apply yet.

   

9. Expand the optional parameters. For this text generation scenario:

   • Set parameters.temperature to 0.8
   • Set parameters.max_net tokens to 200
   • Set parameters.min_new_tokens to 50
   • Set parameters.repetition penalty to 2

   • Set parameters.decoding_method to sample

     Click Apply.

   

10. Click New Step.

11. In this step, select with conditions. In the condition, click the first item, and select watsonx (step 2).

    

12. Select Ran successfully.

13. Click Set variable values, and click Set new value. Then, click New session variable.

    

14. Call the variable result, select free text, and click Apply.

    

15. For the variable assignment To, select Expression, and type $. After you paste this, a pull-down menu appears. Select watsonx (step 2).

    

    Select body.results. The box fills in with the assignment for the result variable. Now append [0].generated_text to the expression.

    

    Click Apply.

16. The full expression for the variable should be displayed, as shown in the following image.

    

17. In the Assistant says panel, select function, and then enter ${result}. Or, you can select Session variables, and then result.

    

The assistant should be ready. It's time to test it out.

Step 7. Testing the action

1. Click Preview.

2. After the greeting, type I need to create a marketing message, and send to the chatbot.

3. The chatbot replies with the prompt for the prompt.

4. Enter the following text into the chatbot.

    Generate a 5 sentence marketing message for a company with the given characteristics.
   
    Details
    Characteristics:
   
    Company - Golden Bank
   
    Offer includes - no fees, 2% interest rate, no minimum balance
   
    Tone - informative
   
    Response requested - click the link
   
    End date - July 15
   
    Email
   

5. Send to the chatbot. If everything worked, you receive a response from the model with the message.

   

   You most likely will see something different because you are using a sample (versus greedy) decoding approach. If you don't like the response, rerun the assistant to get another message. Note that with these parameters, the assistant might create incorrect information like the year for the date (because only July 15 is specified) or phone numbers. In a real use case of this type, a human would use this generated message as a starting point, making edits as necessary and then approving the final version.

Next Steps

Note that the provided watsonx-openapi.json file does not include definitions for all parameters and is meant to be a starting point for further expansion. One good task is to edit the file to add an additional optional parameter for the random_seed, and then try the assistant with the seed 111 to get a result that matches the Prompt Lab marketing message example.

Experiment in prompt lab for other scenarios using different prompts and parameters, then create new actions and adjust the parameters for the new use case. Because all of the prompt is required for input in the current design, zero or one-shot (like the one in this tutorial) work the best. However, with a bit more effort, it should be possible to define an assistant variable with more examples and then use an expression to build the prompt for the integration. Try it out!

Try out watsonx Assistant and watsonx.ai to learn more features and functions.