💬 NLP

MCite-RL: Towards Reliable Multimodal RAG via Citation-enhanced Agentic Reinforcement Learning

The paper proposes MCite-RL, a citation-enhanced agentic reinforcement learning framework that improves the reliability of multimodal RAG by introducing an iterative Agentic Refinement module for dynamic visual citation and a dual-level reward mechanism to jointly optimize answer accuracy and source traceability.

Original authors: Suifeng Zhao, Zida Liu, Xinyu Lei, Lei Sun, Jun Gao, Sujian Li

Published 2026-08-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Suifeng Zhao, Zida Liu, Xinyu Lei, Lei Sun, Jun Gao, Sujian Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern digital age, artificial intelligence has learned to read text and look at pictures with remarkable skill. When these systems are asked complex questions, they often turn to a library of documents to find the right answer, a process known as retrieval-augmented generation. Imagine asking a librarian for a specific fact; the librarian finds the book, reads the page, and tells you the answer. For years, this system worked well with text alone. However, as these AI models have grown more sophisticated, they have begun to handle documents filled with charts, graphs, and photographs. The challenge now is not just finding the right page, but pointing exactly to the specific spot on that page—the tiny section of a graph or a small box in a photo—that proves the answer is true. Without this ability to point to the evidence, the system is like a student who gives the right answer on a test but cannot show their work, leaving the teacher unable to verify if the student truly understood the material or just guessed.

A team of researchers from Peking University and Panasonic Connect has developed a new method to solve this problem of "showing their work" in the visual world. They call their system MCite-RL. The core idea is to teach the AI not just to find an answer, but to treat the act of finding the evidence as a careful, step-by-step investigation. Instead of guessing the location of the proof in one quick glance, the system is trained to act like a detective who zooms in on a document, cuts away the irrelevant parts, and narrows down the search area until only the crucial piece of information remains. This process is guided by a new type of learning where the AI receives feedback not just on whether the final answer is correct, but on how well it located the visual proof along the way.

The researchers found that previous methods often failed in two specific ways. Sometimes, the AI would point to a picture that was too big, covering a whole chart instead of the single number needed, making the citation useless for verification. Other times, the AI would give a correct answer but point to a completely different part of the image, as if the answer and the evidence were disconnected. To fix this, the team created a training process that combines a "cold start" phase, where the AI learns the basic steps of searching and cropping images, with a reinforcement learning phase. In this second phase, the AI plays out many different scenarios, trying to find the answer and the proof. If it successfully narrows down the image to the right spot and gets the answer right, it receives a reward. If it gets lost or points to the wrong area, it learns from that mistake.

The results of this approach were tested on three different sets of challenging documents, including financial reports and long Wikipedia pages. The new system significantly outperformed existing methods. On one major test, it improved the precision of pointing to the correct visual evidence by more than 26 percent compared to standard methods. Perhaps more surprisingly, by forcing the AI to be precise about where it found the information, the system also became better at getting the actual answer right, with accuracy rising by nearly 6 percent on average. The study suggests that when an AI is trained to be rigorous about its sources, it becomes more reliable overall. The researchers note that while the system is a significant step forward, it still requires careful human oversight, especially in high-stakes situations, and it currently works best with single images rather than complex, multi-page documents. Ultimately, this work demonstrates that teaching an AI to point to its evidence is not just a way to verify its work, but a powerful tool to help it think more clearly.

Technical Summary: MCite-RL

Problem Statement
Multimodal Retrieval-Augmented Generation (RAG) systems aim to enhance the traceability and verifiability of Multimodal Large Language Models (MLLMs) by linking generated answers to visual evidence. However, current approaches, particularly those relying on Supervised Fine-Tuning (SFT) or static RAG pipelines, face two critical limitations:

Imprecise Visual Citation: Models often fail to localize evidence at the correct granularity, producing bounding boxes that are either too coarse (covering irrelevant regions) or too fine (missing key details), leading to spatial drift.
Answer-Citation Decoupling: There is frequently a misalignment between the generated answer and the cited visual region. Models may generate a correct answer based on internal reasoning but cite a visual region from a completely different context, undermining the system's trustworthiness.

Existing methods treat visual grounding as a static, single-step process or an implicit intermediate signal, lacking the iterative refinement and explicit optimization necessary for reliable, fine-grained citation in complex multimodal documents.

Methodology: MCite-RL
The authors propose MCite-RL, a citation-enhanced agentic reinforcement learning framework designed to jointly optimize answer accuracy and source traceability. The framework consists of two core components:

Agentic Refinement Workflow:
- Instead of a single-step generation, the model operates as an agent performing a sequential decision-making trajectory ( $\tau$ ).
- The process involves iterative retrieval and progressive visual cropping. The agent alternates between retrieving relevant images and pruning irrelevant visual regions to narrow the search space.
- This dynamic process transforms citation from a static step into an evidence-driven reasoning loop, allowing the model to progressively converge on the optimal visual region ( $\hat{v}_{cT}$ ) before generating the final answer and citation jointly.
Citation-Enhanced Reward Mechanism:
- The framework employs a composite reward function $R(\tau)$ that integrates process-level and outcome-level feedback within a Reinforcement Learning (RL) paradigm (specifically using Group Relative Policy Optimization, GRPO).
- Process-Level Reward ( $R^{proc}_{cit}$ ): Uses Intersection-over-Ground-truth (IoGT) to evaluate intermediate crops. This incentivizes the agent to preserve core evidence throughout the pruning process, preventing information loss during the search.
- Outcome-Level Reward ( $R^{out}_{cit}$ ): Uses Intersection-over-Union (IoU) to evaluate the final predicted citation against the ground truth, ensuring precise localization.
- Additional Rewards: The function also includes retrieval efficiency (nDCG), answer accuracy (Exact Match), and format constraints to ensure valid agentic trajectories.

Training Pipeline
The training follows a two-stage process:

SFT Cold-Start: The model is first fine-tuned on synthesized data containing step-wise agentic trajectories (retrieval, cropping, answering) generated by GPT-5. This establishes a baseline policy and familiarizes the model with the required structural sequence.
Citation-Enhanced RL: The SFT-tuned model undergoes RL training using GRPO. The agent generates multiple rollouts, and the policy is updated based on the citation-enhanced rewards, directly optimizing for both citation precision and answer correctness.

Key Contributions

Framework: Introduction of MCite-RL, which integrates agentic workflows with citation-enhanced RL to address the reliability gap in multimodal RAG.
Workflow Design: A novel agentic refinement mechanism for visual citation that uses iterative evidence refinement to align answers and citations within a shared reasoning process.
Reward Function: Development of a multi-level reward function that explicitly optimizes citation precision (both process and outcome) alongside answer accuracy, treating citation as a primary objective rather than a by-product.
Empirical Validation: Extensive evaluation demonstrating that explicit citation supervision drives improvements in both citation precision and answer quality.

Experimental Results
The authors evaluated MCite-RL on three benchmarks: Wiki-VISA, FinRAGBench-V, and MMLongBench-Doc, using Qwen2.5-VL (3B and 7B) backbones.

Citation Precision: MCite-RL achieved significant improvements over baselines. On the Qwen2.5-VL-7B backbone, it improved the Intersection-over-Union (IoU) metric by an average of 26.16% compared to vanilla RAG. Specifically, it reached 36.05% citation precision on Wiki-VISA and 37.22% on FinRAGBench-V, outperforming strong baselines like VRAG-RL and proprietary models (e.g., GPT-4o, Gemini-2.0).
Answer Accuracy: The integration of citation rewards led to a concurrent increase in answer accuracy. For the 7B model, MCite-RL achieved 60.00% accuracy on Wiki-VISA, a 4–6% improvement over previous strong baselines.
Ablation Studies: Removing the RL stage or citation rewards resulted in significant performance degradation, confirming that strategic reward optimization is the primary driver of success. The study also noted that outcome-level rewards are critical for preventing overly expansive crops, while process-level rewards ensure evidence preservation.

Significance and Claims
The paper claims that MCite-RL provides a systematic solution to the challenge of reliable multimodal RAG by demonstrating that explicitly optimizing citation precision at the objective level is critical. The authors argue that citation-enhanced reinforcement learning not only improves the verifiability of outputs but also enhances answer quality by forcing the model to ground its reasoning in precise, shared visual evidence. The work highlights that reliable multimodal RAG requires more than just accurate answers; it necessitates precise evidence localization and consistent answer-citation alignment, which can be achieved through iterative agentic refinement and multi-level reward supervision.

Limitations
The authors acknowledge that the current training process introduces computational overhead and that the formulation is limited to bounding-box-based citations on single images, without explicitly modeling more expressive citation structures. They note that while the system improves traceability, it does not guarantee the factual correctness of the retrieved content itself.

Technical Summary: MCite-RL

More like this