Introduction to branched RAG

Branched retrieval-augmented generation (branched RAG) is an advanced iterative approach to information retrieval and response generation. It works by breaking down complex queries into multiple sub-questions, enabling more targeted retrieval of information. Each sub-question is used to gather specific insights, refining the overall context. The model then synthesizes the final response by integrating insights from these branched retrievals, leading to a more accurate and comprehensive answer.

Traditional RAG pipelines often face challenges when handling complex queries that involve multiple dimensions or layers of information. For example, a query such as, "What is the impact of machine learning on healthcare and finance?," spans two distinct sectors (healthcare and finance), each requiring specialized information. To address such complex queries effectively, branched RAG breaks them down into simpler sub-questions, allowing for more precise and relevant responses.

In this article, we’ll describe an Investment Advisor Tool that uses branched RAG to deliver personalized financial advice by deconstructing user queries into sub-questions and retrieving relevant information from financial reports, market analyses, and investment guidelines.

The following graphic represents our branched-RAG implementation flow:

The data collection framework leverages multiple open-source financial APIs:

• Yfinance API: Provides unrestricted access to Yahoo Finance data, offering granular stock and ETF information at intervals as low as 1-minute. Its open-source nature and easy setup make it ideal for high-frequency data collection.
• Newsapi: The News API enables searching and retrieving live articles from the web based on criteria like keywords, publication date, source, and language. Results can be sorted by date, relevancy, or popularity. An API key is required for access, and the free development tier allows up to 100 requests per day.
• Alpha Vantage: A popular service for financial data, including stock prices, forex, and cryptocurrency information. The API is simple to use, with a free tier permitting up to 5 requests per minute and 500 requests per day.
• Finnhub: Finnhub API provides real-time stock market, cryptocurrency, and financial news data, making it a powerful tool for developers working in financial analysis and trading systems. The free tier of Finnhub's API has a limit of 60 API calls per minute, with an additional limit of 30 API calls per second.
• Polygon.io Stocks API: Provides REST endpoints to access the latest market data from all U.S. stock exchanges. It also offers insights into company financials, stock market holidays, corporate actions, and more. The free plan allows up to 5 requests per minute.

In the following code snippet, the function, recommendation_and_financials_data, is designed to gather stock recommendations and financial data for a list of symbols provided in data["stocks"]. Leveraging the yfinance library, it retrieves relevant information for each symbol, structures it in JSON format, and saves it to designated files in the specified directory. Additionally, the function checks for existing data and preserves it, ensuring that newly retrieved data is seamlessly integrated with previously stored information.

Implementing branched RAG with watsonx

To create our Investment Advisor Tool, we used these components:

• LLMs available in watsonx.ai
• A Milvus vector store
• Llama Index
• IBM Code Engine (for deployment).

This approach uses IBM's Granite-13b-chat-v2, which is available in watsonx.ai, as the foundational language model, combined with BAAI's bge-small-en-v1.5 embedding model to facilitate efficient semantic search. The system's core strength lies in its SubQuestionQueryEngine architecture, which intelligently decomposes complex queries into manageable sub-questions for enhanced retrieval accuracy. Document processing is handled through VectorStoreIndex, enabling efficient similarity-based search across market research data.

The generation parameters are conservatively set (temperature=0.0, top_k=50) to prioritize factual and consistent outputs, making the implementation particularly well-suited for financial analysis and market research, where accuracy is paramount.

In the following code snippet, the get_generated_text function processes financial queries using a generative AI model with LlamaIndex. It initializes a client, configures model settings, loads data, and sets up a SubQuestionQueryEngine to break down queries into sub-questions for detailed insights.

Summary

A branched RAG system improves the accuracy for complex queries because by breaking down queries, the engine can handle nuanced questions more effectively. Also, a branched RAG system handles ambiguity because decomposing sub-questions helps to disambiguate multi-topic queries and ensures each aspect is addressed individually. Finally, branched RAG systems provide more precise retrieval with more relevant results and reduced noise from irrelevant information.

By leveraging branched RAG in an Investment Advisor Tool, we significantly enhanced the precision and relevance of financial guidance. By breaking down complex investment queries into targeted sub-questions, the tool retrieved detailed and contextually pertinent data on various investment opportunities, risks, and market conditions. This iterative approach allows for a deeper exploration of financial topics, leading to more informed and tailored investment recommendations. As a result, users receive comprehensive, nuanced advice that addresses their specific needs and helps them make well-informed investment decisions.

In addition to working with in the financial industry use case, branched RAG also works well in academic research by decomposing research questions into sub-queries, retrieving relevant academic papers and data sets, and synthesizing in-depth insights.

Acknowledgements

This article was developed through the Financial Services Market (FSM) AI Engineering Community in IBM Client Engineering. The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of IBM as a whole