Finding Optimal Video Moment without Training: Gaussian Boundary Optimization for Weakly Supervised Video Grounding

Technical Summary: Gaussian Boundary Optimization for Weakly Supervised Video Grounding

Problem Statement
Weakly supervised temporal video grounding (WSVG) aims to localize query-relevant segments within untrimmed videos using only video-sentence pairs, without ground-truth boundary annotations. While recent approaches utilize Gaussian-based temporal proposals to model these segments effectively, they suffer from a critical limitation in the inference stage. Existing methods typically rely on heuristic mappings to convert Gaussian parameters (mean and standard deviation) into segment boundaries. Specifically, they define the segment center as the Gaussian mean and the duration based on the Gaussian width. The authors argue that these predefined, heuristic strategies fail to fully leverage the structural information encoded in the proposals and often result in suboptimal localization performance.

Methodology: Gaussian Boundary Optimization (GBO)
To address the limitations of heuristic inference, the paper proposes Gaussian Boundary Optimization (GBO), a novel, training-free inference framework. GBO reframes segment boundary prediction as a principled optimization problem that balances two competing objectives:

1. Proposal Coverage: The area under the Gaussian proposal curve within the predicted segment, ensuring the segment aligns with high-confidence regions of the proposal.
2. Segment Compactness: The temporal length of the segment, penalized to avoid including irrelevant content.

The objective function is formulated as:
$$\max_{s,e} \left[ \int_{s}^{e} f(t) dt - \lambda(e - s) \right]$$
where $f(t)$ is the Gaussian proposal, $[s, e]$ is the predicted segment, and $\lambda \geq 0$ is a penalty weight controlling the trade-off between coverage and compactness.

Theoretical Derivation and Closed-Form Solution
The authors derive a closed-form solution for this optimization problem. By analyzing the stationary conditions, they establish that the optimal boundaries $s^*$ and $e^*$ occur where the Gaussian curve intersects the penalty threshold $\lambda$ (i.e., $f(s^*) = f(e^*) = \lambda$).

• Case $0 \leq \lambda < 1$: A unique, non-degenerate solution exists. The boundaries are symmetric around the Gaussian center $c$ and are given by $s^* = c - w\sqrt{-2\ln(\lambda)}$ and $e^* = c + w\sqrt{-2\ln(\lambda)}$, where $w$ is the Gaussian width.
• Case $\lambda \geq 1$: The optimal solution collapses to a degenerate interval at the center ($s=e=c$), as the penalty for length outweighs any potential coverage gain.

The framework is designed to be model-agnostic. It requires no modifications to the training procedure and is compatible with both single-Gaussian proposals (e.g., CPL) and Gaussian mixture proposals (e.g., PPS). For mixture proposals, the method aggregates the components to define an effective center and width before applying the optimization.

Key Contributions

1. Novel Inference Framework: The introduction of GBO, which replaces heuristic boundary prediction with a mathematically grounded optimization problem balancing coverage and compactness.
2. Theoretical Foundation: A rigorous mathematical analysis providing closed-form solutions for optimal boundaries under varying penalty regimes ($\lambda$), including proofs for the existence of non-degenerate solutions.
3. Model-Agnostic and Training-Free: GBO operates as a post-processing inference module that integrates seamlessly with existing Gaussian proposal-based architectures without additional training or architectural changes.
4. Empirical Validation: Extensive experiments demonstrating that GBO consistently improves localization performance across diverse architectures and datasets.

Experimental Results
The authors evaluated GBO on standard benchmarks: ActivityNet Captions and Charades-STA, applying it to three baseline methods: CNM, CPL, and PPS.

• Performance Gains: GBO significantly improved state-of-the-art results. For instance, on ActivityNet Captions, applying GBO to PPS improved R@5 at IoU=0.5 by +4.10% (from 71.32% to 75.42%) and R@5 at IoU=0.7 by +11.25%. On Charades-STA, GBO-enhanced PPS achieved a +6.52% gain in R@5 at IoU=0.5.
• Generalizability: The method demonstrated robustness across different proposal schemes (single vs. mixture) and different underlying proposal distributions (e.g., Laplace, Cauchy, Triangular), showing performance gains even when applied to non-Gaussian proposals.
• Efficiency: The framework introduces negligible inference overhead and relies on a single tunable hyperparameter ($\lambda$), which was found to have an optimal range (typically between 0.8 and 0.95) across different datasets and metrics.

Significance
The paper claims that GBO provides a theoretically sound and practically effective solution to the inference bottleneck in weakly supervised video grounding. By shifting the focus from training-time proposal generation to inference-time proposal utilization, GBO extracts maximal value from existing proposals without the cost of retraining. The authors position GBO as a lightweight, powerful enhancement that achieves state-of-the-art results by rigorously optimizing the trade-off between capturing relevant content and maintaining temporal precision.