💬 NLP

MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QA

MAGE-RAG is a multigranular adaptive graph framework for long-document multimodal QA that constructs query-time evidence subgraphs by iteratively activating and pruning page and element nodes, thereby balancing evidence coverage with noise reduction to achieve state-of-the-art performance on LongDocURL and MMLongBench-Doc.

Original authors: Yilong Zuo, Xunkai Li, Jing Yuan, Qiangqiang Dai, Hongchao Qin, Ronghua Li

Published 2026-06-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yilong Zuo, Xunkai Li, Jing Yuan, Qiangqiang Dai, Hongchao Qin, Ronghua Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of a few clues on a desk, you are handed a massive, 500-page encyclopedia filled with text, charts, photos, and complex diagrams. Your goal is to find the specific answer to a question hidden somewhere inside.

This is the challenge of Long-Document Multimodal Question Answering. The paper introduces a new system called MAGE-RAG to solve this problem. Here is how it works, explained simply:

The Problem: The "One-Size-Fits-All" Mistake

Current systems try to solve this mystery in two flawed ways:

The "Text-Only" Approach: They read the words but throw away the pictures and the layout. It's like reading a transcript of a movie but never seeing the actors or the scenery. You might miss a crucial clue hidden in a chart or a diagram.
The "Whole Page" Approach: They grab entire pages of the book and show them to the AI. But if the answer is just a small sentence in the middle of a page, the AI gets overwhelmed by all the irrelevant junk (like ads, footers, or unrelated images) surrounding it. It's like trying to find a needle in a haystack by handing the whole haystack to a robot.

Both methods are stuck in a "fixed" mode: they grab a set number of pages or chunks no matter how hard or easy the question is.

The Solution: MAGE-RAG (The Smart Detective)

MAGE-RAG acts like a smart, adaptive detective who doesn't just grab pages; they build a dynamic map of the document.

1. The Map (The Multigranular Graph)

Before the detective even starts looking, MAGE-RAG builds a special "map" of the entire document.

Page Nodes: These are the big landmarks (the whole pages).
Element Nodes: These are the tiny details (a specific paragraph, a table, a chart, or a list).
Connections: The map draws lines between these items, showing how they relate. For example, it knows that a chart is inside a specific section, or that a table is next to a paragraph.

This map allows the system to zoom in on a tiny detail or zoom out to see the whole page, depending on what is needed.

2. The Investigation (Query-Time Control)

When you ask a question, MAGE-RAG doesn't just dump data on the AI. It plays a game of "Hot and Cold" with a strict budget (a limit on how much time and memory it can use).

Step 1: The Entry Point: It starts by grabbing a few likely pages (like the detective walking into the right room).
Step 2: The Iterative Hunt: The system has a "controller" that acts like a project manager. It asks:
- "Do we have enough info?"
- "Should we zoom in and open that specific chart?" (Open Node)
- "Should we look at the page before or after this one?" (Search/Expand)
- "Is this paragraph irrelevant? Let's ignore it." (Prune)
Step 3: The Budget: The detective has a rule: "I can only look at 5 pages and open 10 details." If the answer is simple, they stop early. If the answer is complex and scattered across the book, they use their full budget to dig deeper.

3. The Final Report (Structured Rendering)

Once the detective has gathered the right clues, they don't just hand over a messy pile of papers. They organize the evidence into a clean, structured report. They show the AI the specific text, the exact crop of the chart, and the relevant page layout, stripping away all the noise.

Why It's Better (The Results)

The paper tested this on two difficult datasets (LongDocURL and MMLongBench-Doc) filled with long, complex documents.

Accuracy: MAGE-RAG achieved about 52.75% accuracy on one test and 53.26% on the other, beating previous methods that relied on fixed page counts or text-only searches.
Efficiency: It didn't just win by reading more; it won by reading smarter. It successfully found answers that were scattered across different pages or hidden inside complex layouts, which other systems missed.
Transparency: Because the system keeps a "trace" (a log of every step it took), we can see exactly why it succeeded or failed. Did it miss the right page? Did it prune a clue too early? This makes the system easier to debug.

The Bottom Line

MAGE-RAG changes the game from "grab a few pages and hope for the best" to "build a map, follow the clues, and only show the AI exactly what it needs to solve the puzzle." It balances the need to see the big picture (the page) with the need to see the fine details (the specific chart or text), all while staying within a strict budget of time and resources.

Technical Summary: MAGE-RAG

Problem Definition

Long-document multimodal question answering (QA) requires systems to locate sparse evidence within lengthy PDFs and integrate heterogeneous clues from text, tables, images, charts, and complex layouts. Existing Retrieval-Augmented Generation (RAG) approaches face three primary limitations:

Fixed Top-k and Static Budgets: Current methods map every query to a fixed number of pages or chunks. This creates a trade-off where small $k$ misses dispersed cross-page evidence, while large $k$ introduces excessive noise and irrelevant regions, particularly for unanswerable or single-page questions.
Single-Granularity Evidence: Text-based retrieval compresses context but loses visual and layout information. Conversely, page-level visual retrieval preserves the original layout but forces the reader to process large irrelevant regions, increasing inference costs and noise.
Lack of Query-Time Control: Standard pipelines pass retrieved candidates directly to the reader without an explicit mechanism to track evidence states, manage gaps, or dynamically expand/prune evidence based on the specific needs of a query.

The paper formulates long-document multimodal QA as a query-time evidence subgraph construction problem under budget constraints, aiming to balance evidence coverage, noise control, and inference cost.

Methodology: MAGE-RAG

MAGE-RAG (Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG) addresses these limitations through a framework comprising four interconnected components:

1. Offline Multigranular Evidence Graph Construction

Before query time, the system parses the PDF into a heterogeneous evidence graph $G_D = (V, E)$ :

Nodes: The graph contains Page Nodes ( $V_p$ ) representing full-page views (preserving layout and visual context) and Element Nodes ( $V_e$ ) representing fine-grained components (headings, paragraphs, tables, charts, images, formulas).
Edges: Relations encode document structure, including containment (page-to-element), reading order, layout adjacency, section hierarchy, and semantic-neighbor relations.
Attributes: Each node stores multimodal descriptions, including summaries, text content, bounding boxes, and image references.

2. Page-Level Initial Grounding

At query time, the system uses page-level visual retrieval (e.g., ColPali) to identify the Top- $k$ most relevant pages. This serves as a robust entry point, initializing the working memory with page nodes rather than attempting to search the entire element space from scratch.

3. Online Evidence Controller

An iterative controller manages the evidence state ( $S_t$ ) through a sequence of actions within a defined budget ( $B$ ). The state of each node can be Inactive, Active (selected for working memory), Opened (detailed content exposed), or Pruned (discarded).

Action Space: The controller performs ACTIVATEPAGE, ACTIVATENODE, OPENNODE, SEARCHEVIDENCE (focused re-retrieval), PRUNENODE, and STOP.
Decision Process: An evaluator estimates the marginal utility of candidate actions based on the current question, evidence state, and recent trace. It decides whether to expand the graph, open specific elements, search for new pages, or prune low-value nodes.
Budget Constraints: The process is bounded by limits on initial pages, activation actions per iteration, total iterations, and final opened nodes, preventing unbounded context growth.

4. Structured Multimodal Reader Rendering

The final evidence subgraph is rendered into a structured input for the Large Vision-Language Model (LVLM) reader. This input aligns page images, local node crops, and bounding boxes with XML-structured text containing node summaries and types. This allows the reader to consume compact, relevant, and visually grounded evidence rather than raw, unstructured text or full pages.

Key Contributions

Problem Formulation: The authors diagnose the limitations of fixed-granularity retrieval and propose formulating long-document QA as budgeted evidence subgraph construction, emphasizing the need for query-time state management.
MAGE-RAG Framework: The paper introduces a system that unifies page-level visual entry points, a multigranular document graph, an online evidence controller, and structured rendering. This allows for the dynamic selection, expansion, and pruning of evidence within a shared state space.
Unified Protocol and Analysis: The authors establish a unified comparison protocol across LongDocURL and MMLongBench-Doc, analyzing performance not just by accuracy but by evidence budget, trace behavior, and granular breakdowns (single-page vs. cross-page, evidence types).

Experimental Results

The system was evaluated on LongDocURL and MMLongBench-Doc, comparing against Direct MLLM, Text RAG (BM25, ColBERTv2), Page-level Visual RAG (M3DocRAG, VisRAG), and Graph/Agentic RAG (MLDocRAG, G2-Reader).

Performance: MAGE-RAG achieved 52.75% overall accuracy on LongDocURL and 53.26% accuracy (51.19 F1) on MMLongBench-Doc.
Comparisons: It outperformed Direct MLLM, Text RAG, Page-level Visual RAG, and the Graph/Agentic baseline G2-Reader. Notably, it showed significant improvements in cross-page reasoning tasks compared to visual RAG baselines (e.g., 34.70% cross-page accuracy vs. 24.87% for M3DocRAG on MMLongBench-Doc).
Granularity: The system demonstrated that combining page-level anchors with element-level local evidence yields more stable performance across heterogeneous evidence types (text, tables, charts) than single-granularity approaches.
Trace Analysis: Detailed trace analysis revealed that correct samples do not necessarily require longer traces or more iterations than incorrect ones; rather, they benefit from more effective evidence selection (opening relevant nodes and pruning irrelevant ones).

Significance and Claims

The paper claims that MAGE-RAG demonstrates that query-time evidence subgraph construction can effectively balance dispersed evidence coverage with context-noise control. By treating retrieval not as a one-shot decision but as an iterative, stateful process, the framework adapts to the specific evidence requirements of each question.

The authors modestly note that the framework's upper bound depends on the quality of the offline evidence graph and that the controller introduces additional computational costs (evaluator calls). They suggest the approach is particularly well-suited for long-document scenarios with sparse evidence, cross-page dependencies, or complex visual layouts, while acknowledging it may not be cost-effective for all short-document or single-hop cases. The work provides a traceable, auditable mechanism for diagnosing failures in multimodal document understanding, moving beyond simple accuracy metrics to analyze the evidence construction process itself.