Agentic Hybrid RAG for Evidence-Grounded Muon Collider Analysis

This paper introduces "Agentic Hybrid RAG," a framework that combines hybrid retrieval with agentic reasoning to enhance evidence-grounded scientific question answering for muon collider research, validated by a new domain-specific benchmark and superior performance over existing baselines.

The Gist

Imagine you are a detective trying to solve a very complex case about a futuristic particle accelerator called a Muon Collider. This machine is so advanced that the information about how it works is scattered across thousands of different scientific papers, written in a language full of confusing jargon, acronyms, and math.

If you try to find the answer by just reading one paper or asking a smart AI a simple question, you might get the wrong answer or miss the crucial clue entirely. That's where this paper comes in. The authors built a special "super-detective" system to help scientists find the truth in this mountain of documents.

Here is how their system works, explained simply:

1. The Problem: The "Library of Confusion"

The Muon Collider field is like a massive library where books are written in different dialects.

• The "Exact Match" Problem: Sometimes you need to find a specific technical term (like a specific code name for a machine part). If you use a smart search that looks for "meaning," it might miss the exact code name.
• The "Meaning" Problem: Sometimes you ask a question using different words than the author used (e.g., "background noise from decaying particles" vs. "beam-induced backgrounds"). A strict keyword search might miss this, even though it's the right answer.

2. The Solution: The "Hybrid Search Engine"

The authors created a system that uses two search strategies at the same time, like a detective using both a fingerprint scanner and a human intuition check.

• The Keyword Scanner (Sparse): This is like a strict librarian who only finds books if you give them the exact title or author name. It's great for finding specific acronyms and technical terms.
• The Meaning Reader (Dense): This is like a smart assistant who understands the concept behind your question. It can find a book about "noise from decaying particles" even if you asked about "backgrounds from muon decays."

They combine these two results into one perfect list, ensuring they don't miss anything whether you ask for the exact term or the general idea.

3. The "Agent": The Smart Investigator

Sometimes, a single question is too big to answer in one go. Imagine asking, "How do we stop the machine from overheating?" The answer might be in three different chapters of three different books.

The system includes an AI Agent (a smart assistant) that acts like a detective breaking a big case into smaller clues:

• Step 1: Break it down. The agent looks at your big question and asks itself, "What are the smaller parts of this?" It might split the question into: "What causes the heat?", "What materials stop the heat?", and "How do we measure the heat?"
• Step 2: Hunt for clues. It runs a search for each of these smaller questions.
• Step 3: Gather the evidence. It collects all the relevant pages from the different books and puts them in one folder.

4. The "Grounded" Answer: No Guessing Allowed

This is the most important rule of the system: The AI is not allowed to make things up.

Once the agent has gathered all the evidence (the specific pages from the scientific papers), it writes the final answer.

• The Rule: It must cite exactly which page it got the information from.
• The Safety Net: If the papers don't have enough information to answer the question, the system is programmed to say, "I don't know," rather than making a wild guess. This prevents "hallucinations" (lying confidently).

5. The Result: A New Benchmark

The authors didn't just build the system; they built a test to prove it works.

• They created a collection of 215 real Muon Collider papers.
• They wrote 58 specific questions (some with answers in the books, some without).
• They tested their "Hybrid Agent" against other standard search methods.

The Verdict: Their system was better at finding the right pages and writing better, more accurate answers than the other methods. It found more relevant evidence and was less likely to get confused by the complex language of particle physics.

Summary Analogy

Think of this system as a team of researchers working on a case:

1. The Librarian finds the exact books with the right keywords.
2. The Translator finds books that talk about the same ideas but use different words.
3. The Detective breaks the big mystery into small clues and checks every angle.
4. The Judge writes the final report, but only uses facts found in the books and refuses to guess if the evidence is missing.

This paper shows that by combining these roles, scientists can navigate the complex world of Muon Collider research much faster and more accurately than before.

Technical Summary

Technical Summary: Agentic Hybrid RAG for Evidence-Grounded Muon Collider Analysis

Problem Statement
Muon collider research encompasses a rapidly expanding and heterogeneous body of literature spanning accelerator physics, detector instrumentation, and high-energy phenomenology. Scientific questions in this domain rarely rely on isolated facts; instead, they require synthesizing evidence distributed across multiple papers, subfields, and technical reports (e.g., connecting beam-induced background studies with detector shielding designs). While Large Language Models (LLMs) offer potential for analysis workflows, they struggle to produce faithful, evidence-grounded outputs without explicit external grounding. Standard Retrieval-Augmented Generation (RAG) systems face a dual challenge: they must retrieve precise technical terminology (acronyms, symbols) often missed by dense semantic retrieval, while also capturing paraphrased scientific concepts that sparse lexical methods overlook. Furthermore, integrating agentic reasoning into RAG risks "retrieval drift," where excessive exploration compromises the precision required for scientific verification.

Methodology
The authors propose an Agentic Hybrid RAG framework designed to balance high-precision retrieval with controlled, evidence-grounded reasoning. The system operates through three tightly coupled stages:

1. Hybrid Retrieval Backbone:

   • Sparse Retrieval: Utilizes BM25 to ensure robust matching of exact technical terms, acronyms (e.g., BIB, MDI, VBS), and named entities prevalent in High-Energy Physics (HEP).
   • Dense Retrieval: Employs sentence-transformers (specifically all-MiniLM-L6-v2) to embed queries and document chunks into a shared vector space, capturing semantic similarity for paraphrased descriptions and exploratory queries.
   • Fusion: The rankings from both retrievers are merged using Weighted Reciprocal Rank Fusion (RRF). The system uses a default weighting of $w_d=0.9$ (dense) and $w_s=0.1$ (sparse) with a smoothing constant $K=60$. This hybrid approach ensures that exact terminology is not lost to semantic generalization while maintaining recall for conceptually similar but lexically distinct content.

2. Agentic Query Decomposition:

   • For complex queries, a lightweight agent (using GPT-OSS-120B) decomposes the original query into a set of targeted subqueries ($N \le 5$).
   • The process involves three steps: (a) Domain Tagging to identify relevant physics subfields; (b) Query Classification to determine the retrieval strategy (precise fact, broad synthesis, or reasoning); and (c) Subquery Generation to create complementary queries probing specific dimensions (e.g., mechanism, motivation, limitations).
   • Crucially, the agent is constrained to avoid inventing paper titles or numerical values, ensuring subqueries remain grounded in the original question.
   • Subqueries are processed through the same hybrid retrieval pipeline, and results are aggregated into a unified evidence pool via deduplication.

3. Evidence-Grounded Answer Generation:

   • The generator conditions on the consolidated evidence set (top-$M$ chunks).
   • The model is explicitly instructed to cite supporting evidence and to abstain from answering if the retrieved material is insufficient, preventing hallucination of unsupported scientific claims.

Key Contributions

• Benchmark Construction: The authors constructed the first dedicated benchmark for retrieval-augmented scientific question answering in the muon collider domain. It includes a curated corpus of 215 publications (5,813 chunks) and a benchmark of 58 questions (45 retrievable, 13 unanswerable) with expert-curated relevance annotations and reference answers.
• Framework Design: The proposal of an agentic hybrid RAG architecture that integrates hybrid retrieval with controlled query decomposition, specifically designed to maintain traceability between generated claims and literature evidence.
• Comprehensive Evaluation: A systematic evaluation demonstrating that the proposed framework outperforms representative baselines in retrieval effectiveness, answer quality, evidence coverage, and factual grounding.

Experimental Results
Extensive evaluation on the constructed benchmark yielded the following findings:

• Retrieval Performance: The hybrid retriever provided the strongest retrieval backbone, significantly outperforming standalone dense or sparse retrievers. It effectively balanced the need for exact keyword matching (critical for HEP acronyms) with semantic generalization.
• Agentic Impact: Agentic reasoning was found to be most effective for controlled evidence expansion and answer synthesis. It successfully recovered evidence missed by initial retrieval without introducing significant noise.
• Overall Performance: The Agentic Hybrid RAG system consistently outperformed baseline retrieval and RAG models across all metrics, including Precision@1, Recall@5, Mean Reciprocal Rank (MRR), and graded Normalized Discounted Cumulative Gain (gNDCG). It also demonstrated superior ability to abstain on unanswerable questions, a critical capability for scientific integrity.

Significance and Claims
The paper claims that the combination of hybrid retrieval and controlled agentic reasoning is the primary driver of performance gains, validating an "evidence-aware" design. The work establishes an end-to-end workflow—from corpus construction to answer generation—that serves as a foundation for future evidence-grounded scientific question answering and HEP analysis agents. The authors posit that this framework addresses the specific challenges of the muon collider domain, where information is fragmented across accelerator, detector, and phenomenology communities, and where the cost of hallucination in scientific analysis is high. The code and data are set to be released upon publication to facilitate reproducibility and further research.