💻 computer science

Beyond Sequential Hybrid Retrieval: A Parallel Framework for Accurate and Scalable RAG

This paper introduces PH-RAG, a parallel hybrid retrieval framework that simultaneously executes sparse and dense retrieval with fusion and reranking to achieve state-of-the-art accuracy and improved latency on open-domain question answering, outperforming complex agentic baselines without requiring knowledge graphs or iterative critics.

Original authors: Dilawaiz Hameed, Mariam Shaiq, Ibrar ul Hassan Akhtar

Published 2026-08-26✓ Author reviewed ⓘ

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Dilawaiz Hameed, Mariam Shaiq, Ibrar ul Hassan Akhtar

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Modern computers have become remarkably good at writing and talking, mimicking human conversation with startling fluency. Yet, these digital minds suffer from a fundamental flaw: they are trapped in the past. Their knowledge is frozen at the moment they were trained, meaning they cannot know about events that happened yesterday, nor can they easily access the vast, specific details of a company's internal documents or a library's entire collection. When asked a question they cannot answer from memory, they often invent facts, creating confident-sounding but completely false stories. To fix this, engineers developed a method called retrieval-augmented generation. Instead of relying solely on its internal memory, the computer first searches a database of real documents, finds the most relevant pages, and then uses those pages as a reference to construct its answer. This keeps the machine honest and up-to-date.

However, searching for the right information is harder than it sounds. There are two main ways computers look for answers. One method, often called sparse retrieval, works like a traditional library card catalog, matching the exact words in a question to the words on a page. It is excellent at finding specific names, dates, or technical terms but fails if the user asks a question using different words than those in the document. The second method, known as dense retrieval, uses a more intuitive approach. It understands the meaning behind the words, allowing it to find a document that discusses the same concept even if it never uses the exact same vocabulary. For years, researchers have tried to combine these two methods to get the best of both worlds, but they usually did so by running one search after the other. This sequential approach creates a bottleneck, slowing down the system as the amount of data grows.

A team of researchers from Pakistan has proposed a different way to handle this challenge. They built a system that runs both search methods at the exact same time, rather than one after the other. Imagine a librarian who sends two assistants to find a book: one assistant checks the card catalog for exact titles, while the other uses their understanding of the story's theme to scan the shelves. In a traditional setup, the librarian waits for the first assistant to return before sending out the second. In this new system, both assistants are dispatched simultaneously, and the librarian waits only for whichever one takes the longest to finish. This parallel approach, which the researchers call PH-RAG, allows the computer to gather information much faster without sacrificing accuracy.

The researchers tested their system using a collection of over five thousand Wikipedia articles and a set of one thousand trivia questions. They found that by running the two search methods in parallel and then carefully merging the results, their system could find the correct answer more often than previous, more complex systems. Specifically, their method successfully placed the right answer at the very top of the list 65.6 percent of the time. This was a slight improvement over a leading system that relied on a complex network of relationships between facts, known as a knowledge graph. The new system achieved this higher accuracy while being much simpler to build and operate, proving that you do not need a massive, intricate structure to get good results if you use the right tools efficiently.

A key part of their success was how they combined the lists of results from the two different search methods. They did not simply pick the top answer from one list or the other. Instead, they used a strategy that gave more weight to the method that understands meaning, while still keeping a significant role for the method that finds exact words. This blend allowed the system to catch answers that were missed by either method working alone. After merging the lists, the system performed one final, careful review of the top ten candidates. It re-evaluated each potential answer against the original question to ensure the very best match was placed at the top. This final step did not change which documents were found, but it ensured that the most relevant one was presented first, which is crucial when the computer has limited space to read before it starts writing its answer.

The researchers also looked closely at how fast their system worked as the size of the library grew. They found that for small collections of documents, the speed difference between running searches one after another and running them together was negligible. However, as the collection grew to between five thousand and twenty thousand documents, the parallel system became significantly faster, cutting the waiting time by up to 64 percent. This is because the method that searches for exact words takes longer as the library gets bigger, while the method that understands meaning stays relatively fast. By running them together, the system avoids waiting for the slower method to finish before starting the faster one. The study suggests that for most real-world applications involving mid-sized collections of text, running searches in parallel is a highly efficient way to improve both speed and accuracy without needing to build a more complicated machine.

The findings challenge the idea that more complex systems are always better. The researchers compared their approach to a system that uses a knowledge graph and an artificial intelligence agent that repeatedly checks its own work. While that complex system is powerful, the new parallel method matched or exceeded its performance on standard questions using a much simpler design. This suggests that for many everyday tasks, such as answering questions about general knowledge or company policies, a well-engineered, straightforward system can outperform elaborate, multi-step processes. The study does not claim to have solved every problem in computer science, particularly those involving questions that require connecting multiple facts across different documents. However, it demonstrates that by carefully coordinating existing tools and running them in parallel, we can build systems that are both faster and more reliable, offering a practical path forward for making artificial intelligence more useful in the real world.

Technical Summary: Beyond Sequential Hybrid Retrieval: A Parallel Framework for Accurate and Scalable RAG

Problem Statement

Large Language Models (LLMs) face two critical limitations in knowledge-intensive applications: static parametric knowledge (frozen at pre-training) and a propensity for hallucination. Retrieval-Augmented Generation (RAG) mitigates these by grounding generation in external documents. However, existing hybrid RAG systems, which combine multiple retrieval strategies (e.g., sparse lexical and dense semantic), typically execute these retrievers sequentially. This sequential execution creates latency bottlenecks, as total retrieval time becomes the sum of individual retrieval times. As corpus sizes grow, the cost of sparse retrieval (e.g., BM25) scales linearly, often dominating the critical path and limiting the scalability of hybrid systems. Furthermore, complex agentic pipelines (e.g., those with iterative critics or knowledge graphs) often introduce significant engineering overhead and latency without necessarily addressing the fundamental efficiency of the retrieval stage.

Methodology: The PH-RAG Framework

The authors propose PH-RAG (Parallel Hybrid RAG), a modular, four-stage pipeline designed to execute retrieval strategies concurrently rather than sequentially.

Parallel Retrieval:
- Two retrievers are dispatched simultaneously via a persistent thread pool to eliminate per-query thread-spawning overhead.
- Sparse Retriever: Uses BM25 (rank-bm25 library) for exact keyword and entity matching.
- Dense Retriever: Uses a BGE-large-en-v1.5 bi-encoder (1,024-dimensional, instruction-tuned) with embeddings stored in a FAISS IndexFlatIP (exact inner-product search). Query instructions are prepended to improve retrieval quality.
- Concurrency: The system utilizes a ThreadPoolExecutor with two workers. While Python's Global Interpreter Lock (GIL) prevents true CPU-level parallelism for these specific operations in CPython, the authors model the speedup based on the theoretical maximum of max(t_BM25, t_Dense) + overhead, which is realizable in production environments using native threads or process-based parallelism.
Result Fusion:
- The ranked lists from both retrievers are merged using one of two strategies:
  - Reciprocal Rank Fusion (RRF): A rank-based method robust to score scale mismatches.
  - Weighted Score Fusion: Raw scores are min-max normalized to [0, 1] and combined using weights ( $w_{BM25}=0.4$ , $w_{Dense}=0.6$ ), reflecting the observed superior standalone performance of the dense encoder while retaining the complementary lexical signal of BM25.
Cross-Encoder Reranking:
- The top 10 fused candidates are rescored using a lightweight cross-encoder (ms-marco-MiniLM-L-6-v2).
- Unlike bi-encoders, the cross-encoder jointly encodes the query and passage, capturing fine-grained token-level interactions to refine the ordering of the top passages. This stage does not alter the set of retrieved documents but improves the precision of the top-ranked item.
Answer Generation:
- For evaluation purposes, the system uses a simple extractive generation step (selecting the sentence with highest token overlap) to isolate retrieval performance from generative capabilities. The framework is designed to be compatible with any generative LLM.

Key Contributions

Parallel Hybrid Architecture: A framework that executes BM25 and dense retrieval concurrently, reducing latency compared to sequential baselines.
Strong Encoder Integration: The use of the instruction-tuned BGE-large-en-v1.5 encoder, which significantly raises the performance ceiling of the dense component before fusion.
Modular Ablation: A four-stage pipeline where fusion strategy, encoder strength, and reranking can be independently ablated to quantify their specific contributions.
Theoretical Latency Model: A validated model characterizing parallel speedups across seven corpus sizes (1,000 to 100,000 documents), demonstrating that parallel execution offers the greatest benefit when sparse and dense retrieval costs are comparable (typically 5,000–20,000 documents).
Empirical Evaluation: A comprehensive evaluation on TriviaQA showing that a simpler parallel architecture can match or exceed more complex agentic systems.

Results

Evaluated on 1,000 validation questions from TriviaQA with a corpus of 5,470 Wikipedia passages, the full PH-RAG configuration (Weighted Fusion + Reranker) achieved:

Hit@1: 0.656
Hit@5: 0.797
Recall@20: 0.833
MRR: 0.718

Comparisons:

PH-RAG surpassed the HybGRAG baseline (reported on the STARK-MAG benchmark) on all four metrics, despite HybGRAG utilizing a knowledge graph and an iterative critic module.
PH-RAG variants significantly outperformed agentic baselines (ReAct, Reflexion, AVATAR) and single-retriever baselines (VSS, Dense-only).
Ablation Findings:
- Fusion: Weighted fusion (0.656 Hit@1) outperformed RRF (0.638 Hit@1), suggesting that calibrated score blending is superior to rank-only aggregation when retriever reliability is known.
- Reranking: While reranking did not change set-membership metrics (Hit@k, Recall) because it reorders a fixed candidate set, it improved the top-rank precision, which is critical for downstream generators with limited context windows.
- Hybrid vs. Single: The hybrid system outperformed both BM25-only and Dense-only configurations, confirming that sparse and dense signals are complementary.

Latency Analysis:

Theoretical parallel speedups ranged from 1.27× to 1.64× for corpora between 5,000 and 20,000 documents.
The advantage peaks when BM25 and dense retrieval latencies are comparable. At very small corpus sizes, thread coordination overhead negates the benefit; at very large sizes, BM25 dominates the critical path so completely that the relative gain diminishes.

Significance and Claims

The paper argues that architectural simplicity combined with careful engineering can match or exceed the performance of significantly more complex agentic RAG systems. Specifically:

Efficiency: Parallel execution provides a deployment-time efficiency benefit that is orthogonal to accuracy, offering a scalable solution as corpus sizes grow.
Accuracy: A well-engineered hybrid retriever (strong dense encoder + complementary sparse signal + reranking) can achieve high retrieval quality without the need for knowledge graphs or iterative self-reflection loops, which are often computationally expensive.
Target Setting: The framework is explicitly designed for flat, unstructured passage corpora (common in open-domain QA), where the additional machinery of graph-based RAG may be unnecessary.

The authors maintain modesty regarding their claims, noting that the comparison with HybGRAG is cross-dataset (PH-RAG on TriviaQA vs. HybGRAG on STARK-MAG) and that the parallel speedup figures are theoretical models validated against per-retriever measurements rather than wall-clock gains within the specific CPython environment used for testing. They identify future work as implementing process-based parallelism to realize measured speedups and evaluating end-to-end answer faithfulness using generative LLMs.