Build a multiturn AI agent with RAG and OpenSearch

Enterprise systems often require users to follow complex workflows. These workflows include creating tasks, updating procedures, and running maintenance plans. Each workflow involves multiple steps, several screens, and supporting data. Users must gather information and coordinate each step.

Field technicians and operators often find these workflows slow and error-prone. Developers need better ways to simplify user interaction while keeping a clear task structure.

Multiturn AI agents address this need. These agents keep conversation context across multiple interactions. They also retrieve workflow data from backend systems. Simple chatbots answer single questions. Multiturn agents track conversation history, follow workflow progress, and call backend APIs to run structured tasks.

This tutorial shows how to build a production-grade multiturn AI agent. The agent guides users through complex workflows using natural language. You will generate synthetic workflow data and store it in OpenSearch with vector embeddings. You will build a standard retrieval augmented generation (RAG) pipeline and a workflow-aware retrieval pipeline. You will also create an IBM Carbon Design System UI that shows responses alongside workflow progress.

Architecture of a multiturn AI agent

The application uses a three-tier architecture and runs in a single container:

Data layer: OpenSearch stores about 1,000 workflow documents. These documents include procedures, task plans, and step-by-step instructions. Each document uses 1024-dimensional vectors from the IBM watsonx.ai intfloat/multilingual-e5-large embedding model. Each document also includes metadata such as step order, dependencies, required inputs, and completion criteria.

Backend layer: A FastAPI application provides REST endpoints for standard RAG and the Workflow Agent. The standard RAG endpoint runs a single vector search in OpenSearch. The Workflow Agent endpoint keeps conversation state, retrieves workflow steps, checks dependencies, and converts user input into backend API calls.

Frontend layer: A React application uses Carbon Design System v11. The UI uses a dark theme (g100). It shows standard RAG results and Workflow Agent results side by side. It also shows workflow progress, step status, and live updates during execution.

Deployment: The application runs in a single Docker container based on Red Hat Universal Base Image 9 with Python 3.12. Supervisord runs nginx and uvicorn. Nginx serves the React app on port 9000 and routes API requests to the backend. Uvicorn runs the FastAPI app on port 8000.

Data flow: When a user sends a natural language request, the front end sends requests to /rag and /workflow-agent endpoints. The standard RAG endpoint converts the query into a vector, retrieves matching documents, and calls IBM Granite 3 8-B Instruct to generate a response. The Workflow Agent endpoint keeps conversation context, retrieves workflow steps, checks step order, converts input into API actions, and returns both the response and the execution status.

Prerequisites

Before you start, make sure that you have the following tools and accounts:

Tools and software

• Python 3.12 or later.
• Node.js 20 or later.
• Docker desktop 24 or later.
• OpenSearch 2.11 or later
• IBM watsonx.ai account.
  • Sign up for free trial at IBM watsonx.ai.
  • Create a project and record the Project ID and API Key.

OpenSearch setup

Choose one of the following options:

• Amazon OpenSearch Service: Use a managed cloud service with a free tier.
• Self-hosted OpenSearch: Install OpenSearch on your local machine or on-premises system.
• OpenSearch Serverless: Use a serverless deployment.

Set up the endpoint URL and credentials for your chosen option:

• Amazon OpenSearch Service credentials.
• Self-hosted OpenSearch security configuration.

IBM watsonx.ai setup

• Sign up at IBM Cloud.
• Create a project.
• Save your Project ID and API key.

Required skills

• Basic Python programming.
• Basic React development.
• Understanding of REST APIs.
• Basic Docker usage.

You do not need prior knowledge of conversational AI or workflow systems.

Steps

Step 1. Clone the repository and set credentials

1. Clone the project repository and open the project folder.

    git clone https://github.com/IBM/multiturn-context-aware-workflow
    cd multiturn-context-aware-workflow/code
   

2. Create the environment file.

    cp .env.example .env
   

3. Open the .env file and add your values.

    # OpenSearch settings
    OPENSEARCH_HOST=https://your-opensearch-endpoint.com
    OPENSEARCH_PORT=443
    OPENSEARCH_USERNAME=admin
    OPENSEARCH_PASSWORD=your-password
    OPENSEARCH_INDEX=workflow_documents
    OPENSEARCH_USE_SSL=true
   
    # IBM watsonx.ai settings
    WATSONX_API_KEY=your-watsonx-api-key
    WATSONX_PROJECT_ID=your-project-id
    WATSONX_URL=https://us-south.ml.cloud.ibm.com
   
    # Model settings
    EMBEDDING_MODEL_ID=intfloat/multilingual-e5-large
    LLM_MODEL_ID=ibm/granite-3-8b-instruct
   

   Security Note: Do not commit the .env file. Keep all credentials out of version control.

Step 2. Review the workflow data model

Review the workflow document in OpenSearch. Each document stores workflow content, a vector embedding, and metadata.

{
  "_id": "workflow-uuid-12345",
  "page_content": "Workflow: High CPU Troubleshooting\nStep 1: Check CPU usage metrics...",
  "embedding": [0.023, -0.041, ...1024 floats...],
  "metadata": {
    "type": "operational_procedure",
    "title": "High CPU Troubleshooting",
    "category": "performance",
    "steps": [
      {
        "step_number": 1,
        "description": "Check CPU usage metrics in monitoring dashboard",
        "required_inputs": ["server_name"],
        "dependencies": [],
        "completion_criteria": "CPU metrics retrieved"
      },
      {
        "step_number": 2,
        "description": "Identify top processes consuming CPU",
        "required_inputs": [],
        "dependencies": [1],
        "completion_criteria": "Process list obtained"
      }
    ],
    "estimated_duration": "15 minutes",
    "required_permissions": ["read_metrics", "view_processes"]
  }
}

• Use step numbers to define step order. The system runs steps in this order.
• Use dependencies to control execution. The system checks that the required steps are complete before it runs the next step.
• Use required inputs to validate user data. The system checks for inputs such as server name before execution.
• Use completion criteria to track step status. The system decides when a step is complete.
• Use metadata and conversation state to track user progress. The agent tracks the current step during the workflow.

Step 3. Generate workflow data

The data_generator.py script creates a dataset with multiple workflow types:

Generate the data:

cd backend
python data_generator.py

The script creates the /data/workflow_dataset.json file. This file stores all workflow documents. The script uses Python dataclasses and includes step order, dependencies, required inputs, and completion rules.

Note: The script generates data in memory and saves it to JSON. In the Docker setup, the UI generates the same data when you click the Populate Data button.

Step 4. Index workflow data in OpenSearch

The ingestion script converts workflow documents into vectors and stores them in OpenSearch. This step enables semantic search on workflow data.

Run the ingestion script:

cd backend
python ingest.py

The process takes about 3–5 minutes for 1,000 documents. The script processes documents in batches of 20 with short delays to avoid API limits.

The ingest.py script performs the following actions:

• Creates an OpenSearch index with 1024-dimensional vectors.
• Stores workflow metadata for retrieval.
• Generates embeddings using IBM watsonx.ai

For more details, see the backend/ingest.py file in the sample code.

Step 5. Implement the standard RAG pipeline

The Standard RAG pipeline provides a baseline for comparison with the Workflow Agent. It runs a single vector search and does not store conversation state.

The implementation is in the backend/rag_pipelines.py file. The StandardRAG class extends BaseRAG and includes two main methods:

• retrieve(): Runs a KNN search in OpenSearch using cosine similarity.
• generate_answer(): Prepares the context and calls IBM Granite 3 8-B Instruct.

This pipeline makes one network call. It works well for factual queries but does not track conversation history or manage multi-step workflows.

Step 6. Implement the workflow agent

The workflow agent extends the standard RAG pipeline. It adds conversation state, workflow control, and backend execution.

The implementation is in the backend/rag_pipelines.py file within the WorkflowAgent class. Key methods include:

• process_instruction(): Controls conversation flow and detects user intent
• _execute_next_step(): Checks dependencies and runs workflow steps
• _call_backend_api(): Converts steps into backend API operations

For more details, review the backend/rag_pipelines.py file that has inline comments.

Step 7. Build the FastAPI backend

The FastAPI backend provides REST endpoints for the standard RAG pipeline and the Workflow Agent. It handles session state and processes user requests.

The implementation is in the backend/main.py file.

Run the backend locally:

cd backend
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 8000 --reload

Visit http://localhost:8000/docs for interactive API documentation.

Step 8. Build the React front end

The React front end provides a user interface for the standard RAG pipeline and the Workflow Agent. It shows responses side-by-side, workflow progress, step status, and performance data.

1. Build and run the front end.

    cd frontend
    npm install
    npm start
   

2. Open the application.

    http://localhost:3000
   

   The front end sends API requests to the backend at http://localhost:8000.

   For component details such as layout, workflow progress, and conversation features, review the code in the frontend/src directory.

Step 9. Run the application with Docker

Docker runs the full application stack in one container. This setup includes the React front end, FastAPI backend, nginx, and supervisord.

1. Build and run the application:

    cd Code
    docker compose up --build
   

2. Open the application:

    http://localhost:9000
   

   For Docker configuration details, see the Dockerfile and docker-compose.yml files in the repository.

Step 10. Test the application with sample workflows

Test the application to compare the standard RAG pipeline and the Workflow Agent. These tests show how each system handles simple queries, multi-step workflows, and missing inputs.

Make sure the backend and front end are running. Then, run these tests in the UI.

Scenario 1: Simple query

How do I troubleshoot high CPU usage?

Standard RAG: Retrieves the troubleshooting guide and returns all steps as a numbered list. The response includes the full procedure with step details. It uses multiple sources and returns the result in about 6 seconds.

Workflow Agent: Retrieves the same guide and presents steps with clear markers such as Step 1 and Step 2. It highlights required inputs and asks the user if they want to continue with execution. It returns the result in about 5 seconds and sets the status to query_answered.

Key differences:

• Format: Workflow Agent uses clear step labels. Standard RAG uses a simple list.
• User interaction: Workflow Agent asks for the next action. Standard RAG only shows information.
• Response time: Workflow Agent is slightly faster.
• State tracking: Workflow Agent tracks conversation state. Standard RAG does not.

Result: Both approaches work for simple queries. Standard RAG returns direct answers. Workflow Agent adds structure and supports step-by-step execution.

Scenario 2: Multi-step workflow execution

This scenario shows how the Workflow Agent runs a multi-step troubleshooting workflow and keeps context across user inputs.

Turn 1 (2.65s): Start workflow

User input: Start the high CPU troubleshooting workflow for server api-prod-01

• The agent starts the High CPU Troubleshooting workflow
• Step 1 becomes active: Check CPU usage metrics in monitoring dashboard
• The agent stores the server name api-prod-01
• Status: workflow_started

Turn 2 (0.00s): Ask a follow-up question

User input: What did you find?

• The agent keeps context for api-prod-01
• Completes Step 1: CPU metrics retrieved
• Starts Step 2: Identify top processes consuming CPU
• Status: step_completed

Turn 3 (0.00s): Provide a short confirmation

User input: Yes, analyze it

• The agent maps it to CPU metrics from the previous step
• Completes Step 2: Process list obtained
• Starts Step 3: Analyze thread dumps for blocking operations
• Status: step_completed

Turn 4 (0.00s): Confirm action

User input: Yes, apply the fix

• The agent completes Step 3: Thread analysis complete
• Starts Step 4: Apply recommended fix or escalate
• Marks all 4 steps as complete
• Status: step_completed

Key observations:

• Context tracking: The agent keeps the server name across all turns.
• Language handling: The agent understands questions and pronouns such as it.
• Step execution: The agent runs steps in order from Step 1 to Step 4 and checks dependencies.
• Response time: The first step takes longer. Later steps return results quickly.
• Progress tracking: The UI shows step status and completion.

Standard RAG: Does not track context. Each query runs as a separate request. The system does not know the current workflow step.

Workflow Agent: Tracks conversation state across multiple turns. Checks step dependencies. Extracts inputs such as server name. Calls backend APIs to run each step.

Result: Use the Workflow Agent for multi-step workflows.

Scenario 3: Workflow with missing inputs

Conversation Flow:

Turn 1 (3.27s): Start workflow

User input: Create a maintenance work order

• The agent selects the Create Maintenance Work Order workflow
• The agent lists the required inputs: asset_id, maintenance_type, priority, scheduled_date
• The agent shows the estimated duration: 5 minutes
• The agent asks the user to confirm the execution
• Status: query_answered

Turn 2 (7.61s): Provide partial inputs

User input: Asset is PUMP-001, it is preventive maintenance

• The agent reads asset_id and maintenance_type
• The agent retrieves a related workflow: New Asset Onboarding, Plan 10
• The agent lists steps: asset registration, monitoring setup, software installation, network configuration, validation, documentation
• The agent asks the user to confirm execution for asset PUMP-001
• Status: query_answered

Turn 3 (7.39s): Provide remaining inputs

User input: Priority is high, schedule for next Monday

• The agent reads priority and scheduled_date
• The agent retrieves a related workflow: Incident Response - Plan 7
• The agent lists steps from incident acknowledgment to closure
• The agent suggests scheduling for next Monday
• Status: query_answered

Key observations:

• Context tracking: The agent does not keep workflow context across turns.
• Query handling: Each user input triggers a new vector search.
• State management: The system treats each input as a separate query.
• Workflow control: The system does not continue the same workflow.
• Design insight: The example shows the need for state tracking in workflow systems.

Standard RAG: Provides general information about work orders. It does not track which inputs are complete or missing.

Workflow Agent: Tracks required inputs across turns. Checks that all inputs exist before execution. Keeps context for the active workflow.

Result: Use the Workflow Agent when inputs are incomplete.

Step 11. Deploy to IBM Cloud Code Engine

Deploy the application to IBM Cloud Code Engine. This platform runs containerized applications and manages scaling, infrastructure, and security.

This step shows how to deploy the container and manage credentials for OpenSearch and IBM watsonx.ai.

For production deployment on IBM Cloud:

1. Push your repository to GitHub. The .env file is ignored, so credentials are not stored in the repository.

2. Create a Code Engine project:

   ibmcloud ce project create --name workflow-agent
   

3. Create a build from your Git repository.

    ibmcloud ce build create --name workflow-build \
      --source https://github.com/your-org/multiturn-workflow-agent \
      --context-dir Code \
      --dockerfile Dockerfile
   

4. Create secrets for OpenSearch credentials.

    ibmcloud ce secret create --name opensearch-creds \
      --from-literal OPENSEARCH_HOST=https://your-endpoint.com \
      --from-literal OPENSEARCH_USERNAME=admin \
      --from-literal OPENSEARCH_PASSWORD=your-password
   
    ibmcloud ce secret create --name watsonx-creds \
      --from-literal WATSONX_API_KEY=your-key \
      --from-literal WATSONX_PROJECT_ID=your-project-id
   

5. Deploy the application.

   ibmcloud ce app create --name workflow-agent \
     --build-source workflow-build \
     --port 9000 \
     --env-from-secret opensearch-creds \
     --env-from-secret watsonx-creds \
     --min-scale 1 --max-scale 3
   

6. Get the application URL.

   ibmcloud ce app get --name workflow-agent --output url
   

Open the application URL from the previous command. For example, https://workflow-agent.xxx.us-south.codeengine.appdomain.cloud.

In the UI, click Populate Data. Wait for the ingestion process to finish (about 3–5 minutes). After ingestion completes, enter queries to test the workflow agent.

Summary

In this tutorial, you built a multiturn AI agent for workflow tasks. You used structured workflow data, vector search, and a stateful agent design.

You completed the following tasks:

• Structured workflow data with step order, dependencies, and required inputs.
• Added conversation state to track user progress across steps.
• Checked dependencies and inputs before running each step.
• Converted user input into backend API calls.
• Compared a Standard RAG pipeline and a Workflow Agent.
• Deployed the full application with a FastAPI backend and a React front end using Docker.

Key benefits of multiturn workflow agents

• Conversation context: Keeps context across multiple user inputs. Users can provide information step by step.
• Dependency checks: Runs steps in the correct order based on defined dependencies.
• Backend execution: Converts user input into API calls that run system tasks.
• Error handling: Handles missing inputs and invalid steps. It provides clear feedback.
• Progress tracking: Shows workflow status and completed steps.

When to use each approach:

Next steps

Improve the workflow agent with these features:

• Workflow branching: Add decision rules to support different paths based on user input or system state.
• Step rollback: Add undo support for steps that change system data.
• Approval flow: Require user approval before high-risk actions.
• Multi-user support: Allow multiple users to work on the same workflow with role-based access.
• Production deployment: Deploy the application on Red Hat OpenShift or IBM Cloud Code Engine using the UBI Python 3.12 base image.