Build an agentic RAG pipeline with multi-agent AI workflow

Traditional RAG systems work in a linear pipeline: query encoding, vector similarity search, top-k document retrieval, and response generation. This approach works reasonably well for simple information retrieval. However, it fails on complex technical queries that require multi-faceted reasoning, domain knowledge, and coordinated retrieval strategies.

In enterprise technical domains, engineers search for components with exact specifications, such as a 400VAC input and 28VDC output at 9kW. Traditional RAG fails here because it cannot distinguish quantitative specifications from qualitative requirements. It also lacks domain-specific reasoning and cannot coordinate hybrid retrieval strategies for complex queries.

Agentic RAG solves this problem by using specialized, collaborating agents coordinated through intelligent orchestration. In contrast to monolithic pipelines, domain-specific agents adapt retrieval strategies based on query features and intermediate findings. This architecture combines structured SQL precision for exact specification matching with vector semantic search for contextual relationships, matching expert human reasoning at automated scale.

Case study: Enterprise power supply discovery

This implementation achieved 95% accuracy in technical product discovery. The system handles queries from procurement teams, engineers, and technical specialists searching an industrial power supply catalog. In this domain, specification mismatches can cause costly procurement mistakes and project delays.

When a user queries "I need a power supply with 400VAC input and 28VDC output at 9kW," the agentic system decomposes the query and executes multi-strategy retrieval.

Agentic RAG system architecture: Multi-agent orchestration for technical search

Before diving into the technical implementation, consider this scenario: A procurement engineer needs to find a power supply for a new industrial control system and queries: "I need a power supply with 400VAC input and 28VDC output at 9kW."

The agentic RAG system, which was built with CrewAI, processes this query through a sophisticated multi-agent pipeline designed specifically for technical product discovery. The architecture employs CrewAI's crew-based approach, where four specialized agents collaborate to solve complex queries.

class AgenticRagCrewPipeline(Crew):
    def _create_agents(self) -> List:
        return [
            create_answer_generator_agent(),
            create_product_answer_generator_agent(),
            create_reqs_retriever(),
            create_products_retriever()
        ]

This modular approach provides several advantages:

• Maintainability: Individual agents can be updated without affecting the entire system.
• Scalability: New agents can be added for additional functionality.
• Testability: Each agent can be tested in isolation.
• Performance: Agents can be optimized for their specific tasks.

Intent classification layer

The system begins with intelligent intent classification that determines query routing. This process is crucial for performance optimization and ensures that each query is processed by the most appropriate agent:

1. Product-specific queries: "What are the specs for Model-X123?" Routes to the products retriever agent using direct product lookup with specification extraction.

2. Requirement-based queries: "I need a 9kW AC-DC converter with isolation." Routes to the requirements retriever agent using multi-criteria matching with fallback mechanisms.

3. Pricing queries: "What is the cost of Model-Y456?" Routes to human support consultation by redirecting to the appropriate department.

4. Irrelevant queries: General questions outside domain scope. Routes to a polite decline with guidance, enforcing system boundaries.

The intent classification agent uses a combination of keyword matching, semantic similarity, and learned patterns to achieve high accuracy in routing decisions.

Hybrid data architecture

The agentic RAG system employs a sophisticated dual-storage approach that combines the precision of structured data (a SQL database) with the flexibility of semantic search (a vector database).

Structured data layer

The structured layer leverages Presto in watsonx.data to create a SQL database. This database stores quantitative specifications extracted from product datasheets using large language models. It enables precise filtering for voltage ratings, power specifications, and technical standards.

CREATE TABLE products (
    id VARCHAR(50) PRIMARY KEY,
    name VARCHAR(200),
    input_voltage_min DECIMAL(10,2),
    input_voltage_max DECIMAL(10,2),
    output_voltage DECIMAL(10,2),
    power_rating DECIMAL(10,2),
    efficiency DECIMAL(5,2),
    temperature_min DECIMAL(5,2),
    temperature_max DECIMAL(5,2),
    isolation_rating VARCHAR(50),
    standards_compliance TEXT[]
);

This structured approach enables:

• Precise filtering: Exact matches on numerical specifications.
• Range queries: Finding products within specific parameter ranges.
• Complex joins: Combining multiple specification criteria.
• Performance optimization: Indexed queries for fast retrieval.

Semantic data layer (vector database)

The vector database stores semantic embeddings that capture contextual information and nuanced relationships between concepts. This maintains full context while enabling fuzzy matching and handling variations in terminology.

# Example embedding structure
{
    "document_id": "product_123_datasheet",
    "chunk_id": "section_2_specifications",
    "embedding": [0.123, -0.456, 0.789, ...],  # 768-dimensional vector
    "metadata": {
        "product_category": "power_supplies",
        "technical_domain": "electrical_engineering",
        "content_type": "specifications"
    }
}

The semantic layer provides:

• Contextual understanding: Captures nuanced relationships between concepts.
• Fuzzy matching: Handles variations in terminology and phrasing.
• Conceptual similarity: Finds related products even without exact matches.
• Multilingual support: Can handle queries in different languages.

Technical deep dive: Agent specialization

The following sections describe each of the agents in the implementation.

Products retriever agent

The products retriever agent handles product-related queries with smart query cleaning and retrieval.

def create_products_retriever():
    return Agent(
        role="Intelligent Document Retrieval Orchestrator",
        goal="Retrieve the most relevant and complete document chunks",
        backstory="""You are a powerful reasoning agent with access to the products_retriever_tool.
        Extract search terms from the query e.g., 'M123', '400V AC input', '28V DC output',
        '9kW power supply', 'AC to DC', 'Isolated'

        Then pass the user query and these search terms to products_retriever_tool to get
        the most contextually relevant and complete set of chunks.""",
        verbose=True,
        allow_delegation=False,
        llm=get_llm()
    )

The products retriever agent employs advanced query processing techniques. It removes greetings and signatures, leaving only pure search terms. For example, "Hello, I'm looking for information on M12345-101, thanks!" is simplified to "M12345-101" (a sample model name).

The agent employs multiple retrieval strategies in conjunction:

• Exact match retrieval: Direct search of product database.
• Semantic search: Vector similarity search for semantic matches.
• Specification filtering: SQL queries for technical specifications.
• Fuzzy matching: Typos and name variations in product names.

Requirements retriever agent

The requirements retriever agent specializes in complex requirement-based queries with advanced term categorization.

def create_reqs_retriever():
    return Agent(
        role="Intelligent Document Retrieval Orchestrator",
        goal="Retrieve relevant chunks for requirement-based queries",
        backstory="""You are a powerful reasoning agent with access to the reqs_retriever_tool.
        Extract two types of search terms from the query:
        - Quantitative (e.g., '400V AC input', '28V DC output', '9kW power supply')
        - Qualitative (e.g., 'AC to DC', 'Isolated', 'Bidirectional')

        Then pass the user query and these search terms to reqs_retriever_tool.""",
        verbose=True,
        allow_delegation=False,
        llm=get_llm()
    )

The requirements retriever agent extracts and categorizes search terms into quantitative specifications (such as "400VAC," "28VDC," and "9kW") and qualitative specifications (such as "AC to DC converter" and "isolated power supply").

The requirements retriever agent uses an advanced multi-stage retrieval process with a smart fallback strategy:

• Primary strategy: Attempts to retrieve matches from structured SQL database based on quantitative terms.
• Fallback strategy: Falls back to semantic search based on quantitative and qualitative terms if SQL returns no results.
• Hybrid approach: If SQL returns results, performs semantic search for additional context and identifies overlapping documents for priority ranking.

This is particularly robust because it can find the product when users do not use exact jargon. If a user asks for a "switching power supply" but the data sheet calls it a "switched-mode power converter," the semantic understanding resolves that difference.

Answer generator agents

The system employs two specialized agents to handle different forms of answers, and each one is designed to respond to a particular query type.

def create_answer_generator_agent():
    return Agent(
        role="Final Answer Synthesizer for Product Queries",
        goal="Generate complete, accurate, and well-structured responses",
        backstory="""You are an AI assistant specialized in technical product information.
        Your job is to format retrieved information into clear, professional responses.

        Critical Rules:
        1. When users mention "output", interpret as output voltage unless power is explicitly mentioned
        2. Use tables for specifications only when explicitly requested
        3. Keep responses concise unless detailed comparisons are requested
        4. Always include source citations
        5. Never hallucinate information not present in source documents
        """,
        verbose=True,
        allow_delegation=False,
        llm=get_llm()
    )

The answer generator agent excels at identifying requirement matches, suggesting alternatives when there are no exact matches, and making straightforward, concise recommendations with technical rationale.

Technical deep dive: Retrieval pipeline

The system selects retrieval strategies dynamically based on query attributes and intermediate results. The hybrid retrieval pipeline combines complementary techniques to ensure complete coverage.

Product-specific query processing

When a user asks for a specific model like "What are the specs for M2786-101?", the system:

• Query cleaning: Removes greetings and extracts clean search keywords.
• Hybrid retrieval: Queries SQL database for exact matches and performs semantic vector search in Milvus.
• Result prioritization: Results that appear in both result sets are prioritized because they are most likely to be exactly what the user is looking for.
• Re-ranking: When results are too many, an intelligent re-ranking mechanism selects the most appropriate document pieces.
• Answer generation: Provides concise responses with proper source attribution, never hallucinating information.

Requirement-based query processing

For requirement-based queries like "I need a power supply that can handle 400VAC input and deliver 28VDC output at 9kW," the system demonstrates advanced intelligence:

• Term extraction: Identifies quantitative specs ("400VAC," "28VDC," "9kW") and qualitative adjectives ("AC to DC converter").
• Smart fallback strategy: First attempts structured SQL database search, then falls back to semantic search if needed.
• Engineering logic: The precision agent knows that power requirements typically mean "at least as much power," rather than exactly matching the specified value.
• Contextual understanding: Knows domain-specific vocabulary and applies engineering domain expertise.

Summary: Key takeaways and applications

The agentic RAG architecture represents a significant advancement over traditional RAG systems for complex technical domains. Using specialized agents, hybrid data structures, and intelligent orchestration, the system achieved 95% accuracy while improving user experience and operational efficiency.

Key success factors include:

• Domain specialization: Domain-specific agents and retrieval strategies.
• Hybrid data architecture: Combination of structured and semantic search for comprehensive coverage.
• Intelligent orchestration: Context-aware task routing based on query attributes.
• Quality control: Strong validation to prevent hallucination and ensure correctness.

With careful architectural design, AI systems can match human expert performance in complex technical tasks while retaining the scalability of automation.

The architectural principles demonstrated in this power supply discovery system transfer across diverse scientific and technical domains, including:

• Chemical compound discovery, where agents match molecular structures and reaction conditions
• Pharmaceutical applications connecting compounds with therapeutic targets and clinical trial eligibility
• Aerospace component specification navigating complex MIL-STD requirements
• Semiconductor IC selection based on electrical characteristics and package constraints
• Mechanical engineering applications matching bearings, fluid systems, and equipment to precise load ratings

This universal applicability stems from combining structured quantitative data (voltage ratings, pressure limits) with semantic understanding of qualitative requirements (aerospace-grade, chemical compatibility). Any technical domain with complex specification matching can achieve similar accuracy gains by adapting this multi-agent orchestration framework.

To implement agentic RAG successfully, start with a solid foundation in domain-specific requirements and invest in quality data preparation. The agentic approach provides a powerful framework for building reliable AI systems that handle real-world technical complexity.

Learn more about agentic RAG:

• Optimizing your RAG solutions with IBM watsonx
• Build a RAG agent to answer complex questions
• Build smart AI agents by integrating CrewAI and watsonx