Technical Summary: VCap - Hypergeometric Rewards for Weak-to-Strong Visual Captioning

1. Problem Statement

Visual captioning aims to translate visual perception into natural language that faithfully captures salient facts (objects, attributes, relationships, events) while minimizing omissions and hallucinations. The quality of a caption is governed by two complementary dimensions: Correctness (every stated fact is grounded in the visual signal) and Completeness (all important visual content is sufficiently covered).

Existing approaches to improving caption quality face significant limitations:

• Scaling and Data Curation: Simply scaling models or curating high-quality reference data inevitably propagates existing omissions and hallucinations into the model, creating a performance ceiling.
• Reinforcement Learning (RL) Limitations: Current RL reward designs for captioning (e.g., VQA-based, VLM-as-judge, or imitation-style rewards like CIDEr/DPO) fail to provide fine-grained, reliable signals for factual verification. They typically constrain the reference signal to a single role (e.g., a target to imitate or a limited question pool), making it difficult to jointly supervise correctness and completeness at the fact level. The core obstacle is that the complete set of visual facts is unavailable in practice; no reference enumerates it, and no reward proxy fully recovers it.

2. Methodology: The Witness-Adjudicator Framework

The authors propose VCap, a novel reward mechanism that decouples the roles of the textual reference and the visual signal, treating them as distinct components in a "Witness-Adjudicator" framework.

2.1 Core Concept

• The Witness (Reference Caption): The reference caption is not treated as a gold-standard target to imitate. Instead, it is viewed as a stochastic witness that exposes a subset of visual facts. It acts as a recall anchor (requiring the policy to cover mentioned facts) and an error detector (flagging potential hallucinations).
• The Adjudicator (Visual Signal): The image or video acts as the adjudicator. It verifies matches and mismatches at the specific "slots" activated by the witness. Crucially, the visual signal does not act as an oracle enumerating all facts; it only resolves conflicts or confirms facts at the intersection of the witness and the policy.

2.2 Reward Computation

A frozen reward model (MLLM) processes the triple $(x, y_{ref}, y)$, where $x$ is the visual input, $y_{ref}$ is the reference, and $y$ is the policy-generated caption. The model performs a single-pass evaluation to:

1. Identify candidate omissions: Facts present in $y_{ref}$ but missing in $y$.
2. Identify candidate hallucinations: Facts in $y$ that contradict $y_{ref}$.
3. Verify against $x$:
   • An omission is penalized only if $x$ confirms the missing fact exists.
   • A hallucination is penalized only if $x$ confirms the policy's claim is false (if $x$ contradicts $y_{ref}$, the visual signal overrides the reference, preventing reference errors from penalizing the policy).

The model outputs three integer scores ($0-10$):

• $s_{corr}$: Correctness (penalizing conflicts with $x$).
• $s_{comp}$: Completeness (penalizing confirmed omissions of $y_{ref}$ facts).
• $s_{txt}$: Text Quality (fluency, coherence, absence of meta-text).

These are aggregated into a scalar reward using fixed weights. For video, a local-view reward is computed on random temporal segments and combined with a global-view reward to capture both fine-grained temporal facts and cross-segment events.

2.3 Theoretical Foundation: Hypergeometric Analysis

The paper formalizes visual captioning as a fact-level extraction problem over a latent fact set $F$ of size $N$.

• Let $R$ be the reference fact subset ($|R|=m$).
• Let the policy fact set be $\Phi(y) = C \cup E$, where $C$ are correct facts ($|C|=c$) and $E$ are hallucinated facts.

Under the assumption that $R$ is a uniform random subset of $F$, the probability of passing the completeness and correctness tests follows a hypergeometric distribution:

• Completeness Probability ($P_{comp}$): The probability that all $m$ reference facts fall within the $c$ correct facts. This increases strictly as $c \to N$.
• Correctness Probability ($P_{corr}$): The probability that the $n-c$ hallucinated facts avoid the $m$ reference slots. This decreases strictly as hallucinations grow.

Key Insight (Two-Axis Weak-to-Strong): The optimal policy state is $c \to N$ (maximal correct facts) and $n-c \to 0$ (zero hallucinations). Crucially, the location of this optimum is independent of the reference size $m$. The reference size $m$ only controls the steepness of the supervision gradient. This allows a model to learn from "weak" or imperfect references and still converge to the true visual-information ceiling, enabling a self-improvement loop where the policy regenerates stronger references to sharpen the gradient.

3. Key Contributions

1. Fact-Level Framework: The authors formalize captioning along two independent axes (Correctness and Completeness) and demonstrate why existing rewards struggle to supervise both simultaneously.
2. Witness-Adjudicator Reward (VCap): A novel reward design that treats the reference as a stochastic witness and the visual signal as an adjudicator. This separation yields hypergeometric reward signals that enable weak-to-strong generalization, allowing models to surpass the quality of their training references.
3. Empirical Validation: An 8B parameter model trained with VCap achieves state-of-the-art performance on multiple image and video captioning benchmarks, outperforming significantly larger open- and closed-source models (e.g., Qwen3.5-397B, GPT-5.4, Gemini 3.1 Pro).

4. Experimental Results

The authors evaluated VCap on image and video captioning benchmarks, as well as downstream QA tasks.

• Image Captioning: On CapMAS and DecapBench, the VCap (e2) model (8B parameters) achieved the highest scores across all metrics (e.g., 83.53 Avg on CapMAS, 73.67 Avg on DecapBench), surpassing GPT-5.4 and the 397B parameter Qwen3.5 model.
• Video Captioning: VCap achieved the best performance on the VDC benchmark (36.01 Avg) and competitive results on VCapsBench, outperforming the backbone and larger models.
• Human Evaluation: On a held-out 500-image set, VCap ranked first in per-image factual correctness. The VCap reward signal showed a 61.1% agreement with human pairwise preferences, significantly above the 50% random baseline.
• Self-Improvement: The transition from VCap (e1) to VCap (e2), where references were regenerated by the policy itself, resulted in monotonic improvements across all benchmarks, validating the theoretical self-improvement loop.
• Generalization: Models trained only on captioning (no QA data) showed significant gains on image and video QA benchmarks (e.g., Prism, MMVU), suggesting VCap improves underlying visual perception rather than just surface caption form.
• Ablation Studies:
  • Best-of-N Distillation: VCap-driven RL outperformed self-distillation from Best-of-N samples, proving the gains come from the reward signal, not just sample selection.
  • Modality/Dimension Ablation: Removing either the reference (witness) or the image (adjudicator) caused significant performance drops, confirming the necessity of the dual-role design. Removing Correctness or Completeness dimensions specifically degraded their respective metrics.

5. Significance and Claims

The paper claims that VCap fundamentally shifts the paradigm of RL-based captioning by decoupling the reward optimum from the reference quality.

• Breaking the Imitation Ceiling: Unlike imitation-style rewards where the policy is capped by the reference quality, VCap allows a model to exceed its training references by treating them as probes rather than targets.
• Factual Alignment: The method provides a mechanism for fine-grained factual verification that aligns closely with human judgment, addressing the "hallucination" problem in MLLMs.
• Perceptual Enhancement: The results suggest that optimizing for fact-level correctness and completeness via VCap enhances the model's general visual perception capabilities, benefiting downstream tasks like QA without specific fine-tuning.

Limitations Acknowledged by the Authors:

• Image and video policies were trained separately due to computational constraints.
• Human evaluation was limited to a 500-image set with internal annotators.
• The study focuses on GRPO; other on-policy RL algorithms were not empirically verified.
• Dedicated captioning specialists for baselines are scarce, forcing reliance on generalist MLLMs.