Technical Summary: Reasoning as Intersection: Consensus-Frame Alignment for Visual Focus in Video-MLLMs

1. Problem Statement

While Reinforcement Learning (RL), particularly Group Relative Policy Optimization (GRPO), has successfully enhanced the reasoning capabilities of Large Language Models (LLMs), its application to Video Multimodal Large Language Models (Video-MLLMs) faces a specific limitation. Current methods predominantly rely on outcome-only rewards (e.g., answer correctness, temporal consistency, or localization). While these signals verify if a response is correct, they fail to provide process-level guidance regarding which visual evidence frames support the answer.

In long or untrimmed videos, models may anchor their responses to frames that are visually plausible but irrelevant to the specific query, or they may rely on a diffuse pattern of frame usage that fails to isolate decisive evidence. Existing process supervision methods often focus on textual reasoning steps or image-conditioned traces, lacking a mechanism to align generation with temporally distributed visual evidence without requiring expensive human temporal annotations.

2. Methodology: Consensus Frame GRPO (CF-GRPO)

The authors propose Consensus Frame GRPO (CF-GRPO), a temporal-annotation-free process-level RL framework. The core innovation is the Consensus Frame Reward (CFR), which aligns the model's internal frame usage with a "consensus prior" constructed from intrinsic video cues.

2.1 Consensus Prior Construction

Instead of using human-annotated ground truth, the method constructs a soft, temporal-annotation-free prior $P(i)$ by fusing three complementary intrinsic cues:

1. Uniform Temporal Coverage: A baseline ensuring global sequence preservation.
2. Scene-Transition Cues: Identified via Bhattacharyya distance between HSV histograms of adjacent frames to capture visual shifts.
3. Query-Conditioned Visual Relevance: Derived from a pretrained visual-language encoder measuring image-text similarity between sampled frames and the user query.

These signals are weighted and normalized to create a probabilistic distribution $P(i)$ representing frames likely to contain useful evidence.

2.2 Model-Side Frame-Use Score Extraction

To align the model's behavior with the prior, the framework extracts a model-side frame-use score ($\alpha(i)$) from the model's internal representations:

• Salience-Aware Sparse Aggregation: Visual patch tokens for each frame are aggregated using channel-wise max pooling rather than average pooling. This preserves high-response regions and prevents dilution by background noise.
• Semantic Alignment: The aggregated frame vectors are compared with response hidden states via cosine similarity to generate raw weights.
• Distribution Sharpening: A low-temperature softmax is applied to the raw scores to create a high-contrast distribution, reducing overly diffuse frame scores and strengthening the policy gradient signal.

2.3 Reward Mechanism and Optimization

The Consensus Frame Reward ($R_{cf}$) is calculated as the dot product (overlap) between the consensus prior $P(i)$ and the model-side frame-use score $\alpha(i)$.
The total reward function combines standard outcome supervision with this process signal:
$$R_{total} = R_{acc} + R_{struct} + \lambda \cdot R_{cf} + R_{temp}$$
Where $R_{acc}$ is answer accuracy, $R_{struct}$ enforces format constraints, $R_{temp}$ encourages temporal consistency, and $R_{cf}$ is the new evidence-alignment term. The model is optimized using GRPO, updating the policy based on grouped rollouts to maximize this composite reward.

3. Key Contributions

1. CF-GRPO Framework: A novel RL framework for video reasoning that introduces frame-level evidence alignment without requiring human temporal annotations.
2. Consensus Frame Reward (CFR): A mechanism that constructs a consensus prior from intrinsic video cues (coverage, transitions, relevance) and aligns it with model-side frame usage via salience-aware sparse aggregation and distribution sharpening.
3. Performance and Interpretability: The proposed VideoCFR model achieves competitive performance on complex video reasoning benchmarks. The consensus prior provides an interpretable view of the evidence frames emphasized during training, and ablation studies confirm the necessity of the consensus prior, sparse aggregation, and sharpening components.

4. Experimental Results

The model was evaluated on a suite of video reasoning benchmarks (VSI-Bench, VideoMMMU, MMVU) and general video understanding benchmarks (MVBench, TempCompass, VideoMME).

• Main Results: VideoCFR (32-frame setting) achieved 52.4% on VideoMMMU, outperforming Video-R1-7B (47.2%) and VideoChat-R1 (48.7%). It also showed improvements on VSI-Bench and MMVU compared to representative RL baselines.
• Scaling: Increasing the frame budget from 16 to 64 improved performance on benchmarks requiring richer temporal context (e.g., VideoMME: 55.1% $\to$ 61.1%), though some benchmarks peaked at lower frame counts.
• Ablation Studies:
  • Replacing the consensus prior with a uniform distribution significantly degraded performance on reasoning benchmarks (e.g., -3.2% on VideoMMMU).
  • Removing sparse aggregation or distribution sharpening led to drops in performance across multiple benchmarks, confirming their role in preserving task-relevant signals and enhancing reward discriminability.
  • The inclusion of image QA data during training was found to be beneficial for preserving fine-grained visual recognition.
• Diagnostic Analysis:
  • Training Dynamics: The scaled consensus score increased from 0.47 to over 0.62 during training, indicating the model successfully learned to align with the prior.
  • Attention Alignment: Consensus frames received 19.5% of top-attention events while occupying only 14.1% of frame slots (1.48$\times$ enrichment), whereas non-consensus frames remained below baseline.
  • Qualitative Cases: Visualizations showed that VideoCFR successfully focused on decisive evidence (e.g., surgical markings, specific biological mechanisms) while ignoring visually salient but irrelevant distractors (e.g., gums in a medical video), unlike baseline models.

5. Significance and Limitations

Significance:
The paper argues that video reasoning requires more than just answer correctness; the generation process must be aligned with the visual evidence supporting the answer. By introducing a process-level reward that operates without human temporal labels, CF-GRPO bridges the gap between outcome supervision and the specific visual evidence used by the model. This approach offers an interpretable mechanism to guide Video-MLLMs toward more robust, evidence-based reasoning.

Limitations:

• Prior Reliability: The consensus prior is an estimate based on intrinsic cues. It may be less reliable when evidence is implicit, visually subtle, or distributed across many frames.
• Modality Coverage: The current prior relies only on visual and text-query cues, lacking audio, subtitles, or cross-modal event timing.
• Computational Cost: The method introduces additional computation during training due to the calculation of priors and frame-use scores over grouped rollouts.
• Frame Budget Sensitivity: Increasing the frame budget is not uniformly beneficial across all tasks, suggesting a need for task-specific adaptation rather than a universal scaling rule.
• Evaluation: Diagnostic analyses rely on internal statistics rather than human-annotated temporal evidence labels.

The authors conclude that future work could explore longer-context architectures, additional modalities, and online approximations to further broaden the applicability of CF-GRPO.