NightFeats @ MMU-RAGent NeurIPS 2025: A Context-Optimized Multi-Agent RAG System for the Text-to-Text Track

Technical Summary: NightFeats – A Context-Optimized Multi-Agent RAG System

Problem Statement
The paper addresses three structural limitations inherent in conventional Retrieval-Augmented Generation (RAG) pipelines:

1. Retrieval Myopia and Multi-Hop Brittleness: Existing systems often fail to chain causally relevant information for complex reasoning, leading to shallow domain exposure and hallucinated conclusions.
2. Provenance Drift and State Collapse: In iterative reasoning, verified facts are frequently overwritten, conflicting evidence is silently merged, and source attribution is lost, eroding traceability.
3. Scalability and Domain-Adaptation Bottlenecks: Monolithic pipelines struggle with heterogeneous sources, temporal drift, and domain shifts, degrading reliability without principled fallback mechanisms.

The core challenge is to construct a multi-agent workflow that retrieves verifiable evidence, extracts factual assertions, resolves contradictions, and composes a coherent, fully cited answer while preserving temporal relevance and factual rigor.

Methodology
NightFeats is a structured, multi-agent system implemented using the OpenAI Agents SDK and GPT-5. It decomposes knowledge synthesis into three coordinated phases governed by explicit intermediate representations and handoff contracts:

1. ResearchAgent (Retrieval and Evidence Construction):

   • Adaptive Routing: Dynamically selects between Recent mode (Brave Search API with freshness filtering), Historical mode (FineWeb corpus for foundational subjects), or Direct-answer mode (bypassing retrieval for unambiguous queries).
   • Seven-Step Pipeline: Includes query expansion (1–3 subqueries), retrieval via Jina Reader, hierarchical semantic chunking (700 tokens, 100 overlap), and embedding with text-embedding-3-small.
   • Temporal-Semantic Reranking: Uses a fusion score $s_i\_tilde = (1 - w) * s_i + w * 1/(1 + d_i/365)$ with $w=0.3$, balancing semantic relevance ($s_i$) against document age ($d_i$).
   • Adaptive Filtering: Applies a dynamic cutoff threshold ($\tau = \max(\tau_{min}, \mu - \alpha * \sigma)$) and deduplication (cosine similarity > 0.85) to ensure evidence diversity.

2. GeneratorAgent (Curation and Reconciliation):

   • Extracts atomic, cited facts and clusters them to detect overlaps and contradictions.
   • Contradiction Handling: Classifies contradictions into three severity levels. Minor issues are resolved via fact weighting; important and critical contradictions trigger targeted sub-retrievals by dedicated Fact and Reconciliation sub-agents.
   • Bounded Iteration: The refinement loop is capped at two iterations to prevent recursive drift, outputting a validated fact set.

3. WriterAgent (Evidence-Grounded Composition):

   • Synthesizes curated facts into prose where every sentence includes explicit, normalized inline citations.
   • Generates a reference list preserving one-to-one provenance, ensuring the output is a verifiable narrative aligned strictly with retrieved evidence.

Key Contributions
The paper proposes three primary architectural contributions:

1. Structured Intermediate Representations: Each pipeline stage outputs a standardized data model containing provenance metadata, contradiction annotations, and normalized citations, rendering the reasoning chain inspectable and auditable.
2. Hybrid Temporal-Semantic Retrieval: The system fuses semantic relevance with recency-aware decay scoring and adaptive cutoff filtering to maintain topical accuracy and temporal freshness.
3. Bounded Iterative Curation: A controlled refinement loop enforces targeted reconciliation for high-severity contradictions (up to 2 iterations), ensuring factual coherence without incurring the latency or drift of unconstrained iteration.

Results
NightFeats was evaluated in the MMU-RAGent NeurIPS 2025 competition against proprietary baselines (Claude-SonnetV2 and Nova-Pro) using automatic metrics, LLM-as-a-Judge assessments, human Likert ratings, and live pairwise preferences on the RAG-Arena platform.

• Automatic Metrics: NightFeats scored lower on ROUGE-L (0.159) compared to baselines (0.181–0.200). The authors attribute this to the structural penalty of including explicit numeric citations, which reduces lexical overlap with reference answers that lack such annotations. BERTScore remained comparable (0.840 vs. 0.850–0.851).
• Human and Dynamic Evaluation: NightFeats was awarded Best Dynamic Evaluation in the text-to-text track. In the live RAG-Arena platform, where real users selected preferred responses, NightFeats outperformed baselines. This indicates that its focus on factual traceability, temporal currency, and bounded reconciliation aligns more closely with human user preferences than systems optimized for lexical similarity.

Significance and Claims
The paper positions NightFeats not as a state-of-the-art claim on a single metric, but as a reproducible, modular architecture for multi-agent RAG reasoning. Its significance lies in demonstrating that:

• Architectural Transparency: Systems built around verifiable evidence grounding and citation traceability are better aligned with real-world user preferences than those optimizing narrowly for automatic similarity metrics.
• Principled Decomposition: Breaking knowledge synthesis into retrieval, curation, and composition phases with explicit handoff contracts addresses the structural limitations of monolithic RAG pipelines.
• Evaluation Shift: The results validate the need for evaluation frameworks that explicitly reward citation quality, source reliability, and factual grounding, rather than penalizing the structural artifacts (citations) required to achieve them.

The authors acknowledge limitations, specifically noting a production latency of 10–15 seconds per query due to iterative curation and external API calls, compared to 6–8 seconds in development. Future work is proposed to address latency through parallelization and to develop metrics that reward citation quality rather than penalizing it.