Counterfactual Sensitivity Is Not Repairability: Auditing Replay Probes for Video Evidence

Technical Summary: CARVE – Counterfactual Auditing of Retrieved Visual Evidence

Problem Statement
Tool-using video agents retrieve visual evidence (frames, temporal windows) to answer questions, yet their architectures do not enforce that the final answer depends on this retrieved evidence. Existing diagnostics often require annotated temporal intervals or a separate "judge" model to determine if an agent is hallucinating or relying on language priors rather than visual input. The authors identify a gap: practitioners holding a fixed, possibly closed-agent system need a method to audit evidence dependence without modifying the model, accessing gradients, or relying on external annotations.

Methodology: The CARVE Probe
The paper introduces CARVE (Counterfactual Auditing of Retrieved Visual Evidence), a black-box, matched counterfactual probe designed to measure whether an agent's committed answer changes when its retrieved visual evidence is semantically destroyed.

1. Intervention Design:

   • SHAM (Control): The agent's retrieved frames are preserved, but the pipeline is re-executed (regenerating pixel realization, parsing, and answering) to capture inherent pipeline variability.
   • DESTROY (Intervention): The same retrieved frames undergo Fourier phase randomization. This disrupts phase-dependent visual structures (destroying semantic content) while preserving layout, amplitude spectrum, and frame count.
   • Comparison: The agent's original answer ($\hat{a}$) is frozen. Both SHAM and DESTROY conditions are replayed $k$ times. The score $\hat{\Delta}$ is the difference in answer-change rates between DESTROY and SHAM: $\hat{\Delta} = \hat{\delta}_{destroy} - \hat{\delta}_{sham}$.

2. Theoretical Framework:

   • The estimator $\hat{\Delta}$ is modeled as the difference of two binomial proportions.
   • The authors derive the exact finite-replay law, showing that the score has a discrete support (spaced by $1/k$) and a non-zero probability of exact ties (zero score) even when the population score is non-zero.
   • This analysis highlights a critical distinction: a probe can be reliable in aggregate (showing a large mean effect) while remaining unstable as a per-question decision rule due to variance and discreteness at low replay budgets ($k$).

3. Routing Policy:

   • The primary policy defers a question to a secondary answerer if $\hat{\Delta} \leq 0$ (indicating the answer did not change significantly more under destruction than under intact replay, suggesting the original answer may not rely on the evidence).
   • The study evaluates this policy at different replay budgets ($k=3$ and $k=10$).

Key Contributions

1. A Matched Counterfactual Audit: CARVE isolates intervention sensitivity from pipeline variability without requiring model weights, logits, retraining, or annotated intervals.
2. Finite-Replay Theory: The paper provides a theoretical account of the variance, discreteness, and tie probability of counterfactual probes, explaining why aggregate stability does not guarantee stable per-question thresholds.
3. Decoupling Score Resolution from Decision Utility: The authors demonstrate that increasing the replay budget ($k$) resolves score ambiguity (reducing ties) but can paradoxically weaken a fixed routing policy. Higher precision in measurement does not necessarily yield a better controller if the distribution of actionable cases shifts relative to the decision boundary.
4. Empirical Characterization: The study disentangles temporal coverage, replay stability, and repairability, showing that CARVE is a routing signal rather than a direct grounding classifier.

Results

• Aggregate Effect: Across three independent runs on a frozen VideoExplorer-style agent using the LVBench dataset ($n=1,258$), DESTROY changed the answer 29.3 percentage points more often than SHAM. This confirms a large, reproducible aggregate intervention effect.
• Routing Performance:
  • At $k=3$, the inclusive policy ($\hat{\Delta} \leq 0$) selected 538 questions (42.8% of the set) and improved accuracy by 3.26 percentage points (from 45.47% to 48.73%).
  • The policy yielded 7.62 net repairs per 100 fallback calls, outperforming matched random routing subsets.
  • Stability vs. Utility: Increasing $k$ from 3 to 10 resolved most ties (reducing the tie mass) but weakened the routing policy. The $k=10$ policy selected fewer questions and produced lower accuracy gains, as many questions that were "tied" (and thus selected) at $k=3$ became positive (and excluded) at $k=10$.
• Construct Validity:
  • The score $\hat{\Delta}$ shows only a weak association with annotated temporal coverage (ROC-AUC $\approx$ 0.53).
  • The relationship is conditional: for correct answers, covered questions had higher scores; for incorrect answers, covered questions had lower scores. This indicates $\hat{\Delta}$ is not a direct measure of whether evidence overlaps the ground truth interval.
• Transferability: The routing rule showed positive mean transfer gains when applied to a different answerer backbone (Qwen2.5-VL) and on the Video-MME benchmark, though the magnitude was asymmetric.

Significance and Claims
The paper modestly claims that CARVE provides a practical, annotation-free method for auditing whether a frozen video agent's answer depends on its retrieved evidence.

• Not a Grounding Certificate: The authors explicitly state that CARVE is not a direct grounding classifier or a correctness label. A low score does not prove independence from the video, nor does a high score guarantee the answer is correct.
• A Routing Signal: The primary utility of CARVE is as a budget-dependent routing signal. It identifies a subset of questions where a fallback mechanism is likely to yield repairs, performing better than random selection.
• Policy Design Insight: A key finding is that "more precise" counterfactual measurement (higher $k$) does not automatically improve decision-making. The utility of the probe depends heavily on the specific operating point (e.g., the $k=3$ tie block) and how the distribution of cases aligns with the decision threshold.

In summary, CARVE offers a robust, black-box method to detect evidence dependence in video agents, revealing that while the aggregate effect of evidence destruction is strong, the per-question score is volatile and best utilized as a coarse routing mechanism rather than a fine-grained diagnostic tool.