💻 computer science

Conformal Coverage Guarantees for Any Video Temporal Grounder

This paper introduces COVER, a post-hoc, model-agnostic framework that transforms any video temporal grounder into a reliable predictor by emitting temporal regions with finite-sample, distribution-free guarantees of containing the true event moment, thereby addressing the inherent ambiguity of event boundaries and the lack of reliability metrics in current systems.

Original authors: Aseel Mohamed, Rasul Khanbayov, Erchin Serpedin, Hasan Kurban

Published 2026-08-10

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Aseel Mohamed, Rasul Khanbayov, Erchin Serpedin, Hasan Kurban

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a movie and someone asks, "When does the hero finally catch the villain?" You might point to a start time and an end time. But if you asked ten different friends to mark those exact moments on the same clip, you'd likely get ten different answers. Some would say the chase started a second earlier; others would say it ended a few seconds later. In the world of computer vision, this is called "video temporal grounding." It's the task of teaching computers to find specific moments in a video based on a text description. The tricky part is that the "true" answer isn't a single, perfect line on a timeline; it's a fuzzy cloud of possibilities because human perception is naturally ambiguous.

Currently, most computer programs act like they are absolutely certain. They draw a single box on the timeline and say, "This is it!" But if that box is wrong, the computer gives no warning. It's like a weather app that says, "It will rain at 2:00 PM," but doesn't tell you if it might actually start at 1:55 or end at 2:10. If you're a robot trying to edit a video or a search engine trying to find a specific clip, you need to know not just where the event is, but how sure the computer is. This paper tackles that problem by giving computers a way to say, "I'm 90% sure the event happens somewhere inside this wider, safer zone," without needing to retrain the computer or peek inside its brain.

The Problem: The "Confident" Mistake

The authors of this paper, working with the Kurban Intelligence Lab, noticed a big gap in how video AI works. When a computer tries to find a moment in a video, it usually spits out a single interval (a start time and an end time). The problem is that the "ground truth"—the actual correct answer—is often a distribution, meaning different people would mark slightly different times. Because the computer only gives one answer, a wrong prediction looks exactly the same as a right one. If you are building a tool that relies on these predictions, you have no way of knowing when to trust the computer and when to widen your search.

Some researchers tried to fix this by training models to guess their own uncertainty, or by just adding a fixed amount of time (a "margin") to the predicted start and end. The authors argue that these methods are flawed. Fixed margins are like wearing the same size shoes for everyone; they might fit a child perfectly but be too big for an adult or too small for a giant. In their tests, they found that simply picking a random margin could result in the computer being right anywhere from 10% to 100% of the time, depending on the video. Other methods that try to learn uncertainty from scratch often fail to deliver the reliability they promise, sometimes missing the true moment even when they claim to be 80% sure.

The Solution: The "Safety Wrapper" (Cover)

Enter Cover, a new tool described in this paper. Think of Cover not as a new brain, but as a smart safety wrapper that you can put over any existing video-finding program, whether it's a complex, trained model or a black-box AI you can't see inside.

Here is how it works, using a simple analogy: Imagine you are trying to catch a slippery fish (the true event) in a net (the computer's prediction). The computer throws a net that is usually close, but sometimes misses the tail or the head. Cover doesn't change the computer's throwing arm. Instead, it watches the computer practice on a set of known videos (a "calibration set"). It measures exactly how much the computer's net needs to be stretched to catch the fish every time.

Once it figures out the right amount of stretch needed to be, say, 90% sure of catching the fish, it applies that stretch to every new prediction. If the computer says the event is from 10 seconds to 20 seconds, Cover might say, "Okay, to be 90% sure, let's say it's actually between 8 seconds and 22 seconds." This new, wider zone is guaranteed to contain the true event with the probability you asked for.

What They Found

The researchers tested this "wrapper" on three different video datasets and five different types of video-finding programs. Their results were quite revealing:

It Works: When they asked for a 90% guarantee, the system delivered a 90% success rate. The "realized coverage" (how often they were actually right) tracked the target almost perfectly.
Fixed Margins Fail: They tested the old idea of just adding a fixed amount of time (like "add 10 seconds") without calibration. The results were chaotic. Depending on the video and the model, the success rate swung wildly, from as low as 0.096 (less than 10%!) to 1.000 (100%). This proves that you cannot just guess a safety margin; you must calibrate it.
The "Evidential" Trap: They tested a special type of AI designed specifically to guess its own uncertainty. Even though this AI claimed to be 80% sure, it was only right 66% of the time. Cover, however, took the exact same predictions from that AI and wrapped them in a new layer that did achieve the valid 80% guarantee. This showed that even models built to be uncertain can be wrong about their own uncertainty.
Asymmetry Matters: They discovered that some models are great at knowing when an event starts but terrible at knowing when it ends. For these models, Cover found that stretching the end of the interval needed to be much larger than the start. By allowing the wrapper to stretch each side differently, they could make the safety zone tighter and more efficient.

The Takeaway

The paper concludes that for video temporal grounding, the "right" answer isn't a single point in time, but a region with a confidence statement. Cover provides a way to get that region without needing to retrain the AI or have access to its internal code. It turns any video-finding tool into one that can say, "I'm not 100% sure, but I'm 95% sure the event is in this box," and it actually backs up that claim with math.

The authors emphasize that this guarantee holds true as long as the new videos are similar to the ones used for practice (a concept called "exchangeability"). If the videos are totally different from the practice set, the guarantee might weaken, but the method still offers a way to measure and adjust for that. Ultimately, Cover turns a guessing game into a reliable, calibrated process, ensuring that when a computer points to a moment in a video, we know exactly how much trust to place in it.

Technical Summary: Conformal Coverage Guarantees for Any Video Temporal Grounder

Problem Statement

Video temporal grounding involves identifying the start and end times of an event within a continuous video given a natural language query. A fundamental challenge in this domain is the inherent ambiguity of event boundaries: re-annotating the same query–video pair often yields overlapping intervals that agree by less than half on a large fraction of samples. Consequently, the ground truth is a distribution over intervals rather than a single point.

Current temporal grounders, whether trained localizers (e.g., DETR-style models) or black-box video–language models (VLMs), output a single interval without any statement of reliability. In deployment, a confident but incorrect interval is indistinguishable from a correct one, leaving downstream applications (retrieval, editing, evidence-seeking agents) without a principled way to decide when to defer or widen their search. Existing uncertainty quantification methods either require retraining specific architectures, rely on white-box access, or fail to provide finite-sample coverage guarantees that match their nominal targets.

Methodology: Cover

The paper proposes Cover, a post-hoc, model-agnostic wrapper that transforms any existing temporal grounder into a system that emits a temporal region containing the true moment with a user-specified probability of at least $1 - \alpha$ . Cover requires no retraining, no white-box access, and operates solely on the grounder's outputs and a held-out calibration set.

The method relies on conformal prediction principles adapted for temporal intervals:

Nonconformity Scores: Cover defines a score representing the minimal widening required for a predicted interval to contain the ground truth.
- Interval-Widening Score: For grounders emitting an interval $[\hat{s}, \hat{e}]$ , the score is the maximum of the normalized overshoots on the start and end boundaries: $s(X, I^*) = \max(\frac{\hat{s}-s^*}{\ell(X)}, \frac{e^*-\hat{e}}{\ell(X)})$ , where $\ell(X)$ is the predicted length (or a fixed scale). This allows for a two-sided widening.
- Per-Boundary Variant: A two-parameter variant calibrates start and end boundaries separately to handle asymmetric errors.
- Super-Level-Set Score: For grounders emitting a temporal relevance signal $f_X(t)$ , the score is $s(X, I^*) = -\min_{t \in I^*} f_X(t)$ . This generates adaptive, potentially disconnected regions spending temporal budget only where relevance is high.
Calibration: Given a labeled calibration set $\{(X_i, I^*_i)\}_{i=1}^n$ , Cover computes the nonconformity scores and selects the threshold $\hat{q}$ as the $(1-\alpha)$ empirical quantile (specifically the $\lceil(n+1)(1-\alpha)\rceil$ -th smallest score).
Prediction: For a new test input, the final region $\mathcal{C}(X)$ is constructed by widening the base prediction by the calibrated quantile $\hat{q}$ .
Variants: The framework includes a risk-control variant to bound the expected miss-rate and a stratified variant (Mondrian calibration) to ensure coverage within specific subpopulations (e.g., event length strata).

Key Contributions

Distribution-Free Uncertainty Quantification: The first method to provide finite-sample, distribution-free coverage guarantees for video temporal grounding. It converts any grounder (trained or black-box) into a set predictor with a guaranteed probability of containing the true interval.
Temporal Nonconformity Scores: The introduction of specific scores for both interval-based and relevance-signal-based grounders, including a two-parameter variant to address asymmetric boundary errors.
Theoretical Analysis:
- Efficiency: Proof that the calibrated widening is asymptotically length-optimal within the chosen family of regions.
- Conditional Coverage: Demonstration that coverage holds conditionally on event length under a scale model when using length-normalized scores.
- Exchangeability Bounds: An explicit bound on coverage degradation when moments from the same video violate exchangeability assumptions.
Empirical Certification: A comprehensive evaluation showing that calibration is essential for reliable coverage, exposing flaws in point-metric-based evaluations and "hand-picked" margins.

Experimental Results

The method was evaluated across three benchmarks (Charades-STA, ActivityNet-Captions, QVHighlights) and five grounders, including a trained localizer (QD-DETR) and a black-box VLM (Qwen2.5-VL-7B-Instruct).

Coverage Validity: Realized coverage closely tracks the target $1-\alpha$ across all datasets and models. For instance, at a target of 0.90, QD-DETR achieved $0.895 \pm 0.003$ on ActivityNet-Captions.
Efficiency: No single score family dominates uniformly. Length-normalized scores generally perform well, while super-level-set scores are most efficient for QVHighlights at high coverage targets.
Calibration vs. Fixed Margins: Hand-picked fixed margins failed to achieve consistent coverage, with realized coverage swinging from 0.096 to 1.000 depending on the constant chosen. Calibration was shown to be necessary to achieve the desired target efficiently.
Asymmetry Detection: The per-boundary variant revealed that the VLM (Qwen) had near-perfect onset detection but significant offset errors. A single-parameter score wasted budget on the accurate start boundary, whereas the two-parameter variant reduced region length by up to 9.1% on QVHighlights.
Uncertainty Quantification Failure: An evidential grounder designed for uncertainty quantification missed its nominal 0.80 target, achieving only 0.660 realized coverage. Cover restored a valid guarantee on the identical predictions.
Robustness: The method maintained validity under distribution shifts (cross-dataset transfer) when using weighted conformal prediction, though it hit a "positivity limit" where source data did not support the target distribution.

Significance and Claims

The paper claims that Cover fundamentally changes the output object of temporal grounding from a point estimate to a reliable region. Its significance lies in:

Model Agnosticism: It works on any grounder, including black-box VLMs, without retraining.
Reliability: It provides a rigorous, finite-sample guarantee that a wrong interval is distinguishable from a right one based on the coverage probability.
Diagnostic Power: The calibration process acts as a measurement tool, revealing hidden properties of grounders (e.g., asymmetric boundary errors, failure of native uncertainty estimates) that standard point metrics obscure.
Practical Deployment: It offers a principled mechanism for downstream agents to decide when to defer or widen searches, addressing a core open problem in continuous video understanding.

The authors remain modest, acknowledging that the guarantee relies on the exchangeability assumption (which can be violated within long videos or across datasets) and that exact, attribute-free conditional coverage is theoretically impossible distribution-free. However, they demonstrate that the method degrades gracefully under these conditions and provides the first practical framework for reliable temporal grounding.