IBM Developer

Tutorial

Build a RAG application with watsonx.ai flows engine

Infuse your apps with generative AI without diving too deep into the complexities of machine learning models

By Roy Derks

On this page

Are you a developer looking to enhance your JavaScript applications with some generative AI? Building AI applications shouldn't be that different from building any other application, as you'll learn in this tutorial where we explore how to build a question-answer application using JavaScript and IBM’s watsonx.ai flows engine.

The question-answer application uses Retrieval-Augmented Generation (RAG) to answer questions about a set of documents, which will be stored in a vector database. This tutorial is perfect for anyone who is eager to use AI in their application without diving too deep into the complexities of machine learning models. Let’s get started with learning how to infuse your applications with generative AI!

What is watsonx.ai flows engine?

There are many ways to build AI applications. Most of them aren't that easy to get started with or require prior knowledge about machine learning. With the new watsonx.ai flows engine, developers of all skills levels are able to create generative AI applications. It simplifies how you integrate with Large Language Models (LLMs) and connect them to your (enterprise) data. Think of watsonx.ai flows engine as an end-to-end developer toolkit for building AI applications (see the following figure).

Overview of watsonx.ai flows engine

Developing AI applications with watsonx.ai flows engine doesn't require you to know about machine learning or be an expert in writing data-heavy applications. Instead, you can use a CLI and SDK to generate AI-driven applications with pre-built flows for tasks like Retrieval-Augmented Generation (RAG). Using an interactive flow language, you can make customizations to flows and make connections to LLMs and vector databases by using a configuration file.

watsonx.ai flows engine is currently in public preview, access to models from watsonx.ai and vector databases in watsonx.data are subject to a fair use policy.

Prerequisites

Before you begin, you should create an account for watsonx.ai flows engine for this tutorial:

You can experiment with the product completely for free, and signing up gives you access to models from watsonx.ai and a vector database running in watsonx.data. You don't need to sign up for these products separately, as access will be provided out-of-the-box.

Steps to get started

If you'd like to watch me demo these steps, watch this video:

Step 1: Download a data set

Before setting up the question-answer RAG application using the watsonx.ai flows engine CLI, you need to have a data set that can be uploaded to the vector database.

Download a sample data set called "watsonxdocs", which contains all of the documentation for the watsonx platform.

Create a new project directory on you local machine and move into this directory:

mkdir question-answer
cd question-answer

In this directory, run the following command to download the data set:

curl -O https://watzen.ibm.stepzen.com/downloads/watsonxdocs.zip

The sample data set doesn't include information about IBM Decision Optimization and SPSS Modeller flows.

This command downloads a zip file called watsonxdocs.zip that you need to extract, so you'll get a new directory containing a set of html files.

Step 2: Download the watsonx.ai flows engine CLI

The watsonx.ai flows engine CLI can be used to initialize new projects. For this tutorial, we'll use it to initialize a new RAG project.

Download the CLI using the link on the top of the installation page. You need to be logged in to download the CLI.

watsonx.ai flows engine installation page

After downloading the file, you can place it inside your project directory and install the CLI using pip:

pip install <DOWNLOAD_FILENAME> --force-reinstall

Be sure to replace <DOWNLOAD_FILENAME> with the name of the file you downloaded. For example, if the file is called wxflows_cli-1.0.0rc40-py3-none-any.whl you need to run the command pip install wxflows_cli-1.0.0rc40-py3-none-any.whl --force-reinstall.

Step 3: Set up a new project using the CLI

The following command initializes a new watsonx.ai flows engine project for you, by asking you for a couple pieces of information that is needed to set up the project:

wxflows init --interactive

In the interactive mode, provide the following information:

- [?] Do you wish to use retrieval-augmented generation (RAG)?: yes
- [?] Choose the document collection for context retrieval: create from local data
- [?] Path to the data: ./watsonxdocs
- [?] File types to include: Markdown or HTML(ignore other files)
- [?] Chunk size (in tokens): 250
- [?] Chunk overlap (in tokens): 25
- [?] Collection name: watsonxdocs
- [?] Endpoint name: wxflows-genai/watsonxdocs

This step creates a couple of files:

  • wxflows.toml with your project configuration
  • .env.sample with the possible environment variables
  • watsonx.docs.tsv that includes your chunked dataset

You can rechunk your data by running wxflows data build --data-directory ./watsonxdocs --chunk-overlap 25 --chunk-size 250 --data-type md --force.

The .env.sample file includes all the possible environment variables to connect to various data sources. For the purposes of this tutorial, the only variable you need to set is for connecting to the out-of-the-box connection to models from watsonx.ai:

Create a new file called .env and add the following:

STEPZEN_WATSONX_HOST=shared

This will let you connect to models from watsonx.ai, which we need for creating embeddings for our data set and the actual questioning and answering later on. Please note there's a fair use policy and you cannot use this connection in a production environment.

Step 4: Upload your data set to the vector database

To upload your chunked data set to the vector database, you need to run the following command:

wxflows collection deploy

Depending on the size of your data set, this command might take several minutes to complete. If you've already uploaded your data to the vector data store, you can run wxflows flows deploy to only deploy changes to the wxflows.toml file instead.

Step 5: Explore a flow

Now, let's look at what the flow for questioning and answering looks like. When building AI applications with watsonx.ai flows engine, you use a flow language to create and customize flows. There are a couple of prebuilt flows available, which are added to the wxflows.toml file that was created in step 3.

The wxflows.toml file includes a list of flows that you can use to interact with the LLMs and the vector database. Several example flows will be available and myRag is the one we'll be using in this tutorial.

To enable the flow myRag you need to uncomment it in the wxflows.toml file:

   [stepzen.cli]
   endpoint = "wxflows-genai/watsonxdocs"

   [wxflows.deployment]
   # Example flows.
   flows="""
      // myPrompt = ragAnswerInput | topNDocs | promptFromTopN | ragInfo
      myRag = ragAnswerInput | topNDocs | promptFromTopN | completion(parameters:myRag.parameters) | ragInfo
      // myRagWithGuardrails = ragAnswerInput | topNDocs | promptFromTopN | completion(parameters:myRagWithGuardrails.parameters) | ragScoreInfo | hallucinationScore | ragScoreMessage | ragInfo
   """
   ai_engine = "WATSONX"

The flow myRag consists of 5 steps:

  • ragAnswerInput is a convenience step that gathers the arguments used by the subsequent steps
  • topNDocs retrieves the N documents that most closely match the provided question
  • promptFromTopN computes a prompt from the documents matched in topNDocs
  • completion uses the generation API of watsonX to respond to the previously computed prompt, tuned with the specified parameters. Since the parameters are an object, we map an input argument from myRag to the parameters argument of the completion step.
  • ragInfo pulls results from all of the steps together into an output JSON object.

We'll try out some of the other flows later on, but for now let's deploy this single flow.

Step 6: Deploy flows using the CLI

To deploy your flows or deploy any changes to the flows in your wxflows.toml file, run this command:

wxflows flows deploy

The CLI will deploy your flows, and print the endpoint it's deployed to in your terminal:

Found flow definition in the configuration

Provisioning the watsonx.ai flows engine environment

- myRag = ragAnswerInput | topNDocs | promptFromTopN | completion(parameters:myRag.parameters) | ragInfo

Published flow myRag to https://ACCOUNT_NAME.DOMAIN_NAME/wxflows-genai/watsonxdocs/__graphql in 3.545s

Step 7: Set up a new application

To try out this flow, you can use the endpoint directly using HTTPS or the SDK.

Before consuming a flow using the SDK, you need to set up a new JavaScript application. For this, we'll be using an example application from this public repository.

You can either clone the repository and run the application in the app directory locally, or you can use a free service called StackBlitz to run the example in the browser.

You need to have Node.js (version 18 or higher) installed to run the application locally.

To run the application in the browser, you can click the following button and open the page in the new tab:

Open in StackBlitz

On this page, you need to open the file .env and add your own values for the environment variables:

Add environment variables for wxflows

You can generate the values needed by running the following command in the terminal on your local machine:

   echo VITE_WXFLOWS_ENDPOINT=https://$(wxflows whoami --account).$(wxflows whoami --domain)/wxflows-genai/watsonxdocs/graphql
   echo VITE_WXFLOWS_APIKEY=$(wxflows whoami --apikey)
   echo VITE_WXFLOWS_COLLECTION="watsonxdocs"

Then, paste these lines into the .env file in the StackBlitz page.

Step 8: Ask a question

Everything is set up now, and you can ask your first question. Because the watsonxdocs data set contains the product documentation for the watsonx platform, you can ask all sorts of questions about the product.

Asking questions in the stackblitz user interface

For example, the question "What is WML?" will return something like "Watson Machine Learning (WML) is a tool provided by IBM watsonx. It offers a full range of tools for building, training, and deploying machine learning models.".

That's it! You can now continue building AI applications using watsonx.ai flows engine as there's much more to explore. Try any of the other pre-built flows such as myRagWithGuardrails, or why not build your own flows!

Summary and next steps

The tutorial explained how to build a question-answer tool using JavaScript and IBM watsonx ai flows engine. The question-answer tool you built uses Retrieval-Augmented Generation (RAG), to answer questions based on a set of documents that are stored in a vector database.

With the new knowledge you should have a good foundation to start integrating generative AI into your own applications, by leveraging watsonx.ai flows engine using the CLI and SDK.

We're curious to find out what you're building. Join our Discord community and let us know!