Detecting Object Tracking Failure via Sequential Hypothesis Testing

1. Problem Statement

Real-time online object tracking is critical for applications like surveillance, robotics, and autonomous driving. However, deployed tracking systems often lack formal safety assurances. When a tracker fails (due to occlusion, motion blur, or appearance changes), existing systems typically rely on heuristic confidence measures (e.g., entropy or peak response values) to trigger alerts.

• Limitations of Current Approaches: Heuristics lack formal statistical grounding. They often produce delayed alerts, missed detections, or excessive false positives (unnecessary re-calibrations or human interventions).
• Goal: The authors propose a method to detect tracking failures with statistical guarantees, specifically controlling the False Positive Rate (Type-I error) at a desired level $\alpha$ while minimizing detection delay.

2. Methodology

The core contribution is framing object tracking failure detection as a Sequential Hypothesis Testing problem using e-processes (a concept from "safe testing" and "testing-by-betting" literature).

A. Problem Formulation

The tracking process is modeled as a stream of frames $Y_t$. A tracking quality metric $M_t \in [0, 1]$ is defined for each frame.

• Null Hypothesis ($H_0$): The expected tracking quality remains above a tolerance threshold $\epsilon$ at all times ($\mathbb{E}[M_t] \geq \epsilon$).
• Alternative Hypothesis ($H_1$): At some point, the expected quality drops below $\epsilon$ ($\exists t: \mathbb{E}[M_t] < \epsilon$).

B. The E-Process Framework

Instead of a fixed-sample test, the authors use an e-process ($X_t$), which is a non-negative supermartingale under $H_0$.

• Mechanism: The process starts at $X_0 = 1$. At each step $t$, it updates multiplicatively based on the discrepancy between the observed metric $M_t$ and the threshold $\epsilon$.
• Update Rule:
  $$X_t = \prod_{i=1}^t (1 + \lambda_i (\epsilon - M_i))$$
  Where $\lambda_i$ is a "betting rate" (a weight between 0 and 1) determined by past observations.
• Stopping Rule: An alert is triggered (rejecting $H_0$) as soon as $X_t \geq 1/\alpha$.
• Statistical Guarantee: By Ville's Inequality, this method guarantees that the probability of ever triggering a false alarm (when $H_0$ is true) is strictly bounded by $\alpha$, regardless of when the stopping decision is made ("optional stopping").

C. Tracking Quality Metrics ($M_t$)

The framework supports two modes of operation:

1. Supervised: Uses Ground Truth (GT) bounding boxes. The metric is Normalized Generalized Intersection over Union (NGIoU).
2. Unsupervised: Uses internal tracker signals (no GT required). Proposed metrics include:
   • Peak Correlation (PC): Max value in the response map.
   • Certainty Gain (CG): Normalized PC relative to a recent window.
   • Sharpness Gain (SG): Based on Average Peak-to-Correlation Energy (APCE), measuring the concentration of the response map.

D. Betting Rate Strategies ($\lambda_t$)

To ensure rapid detection of true failures, the betting rate $\lambda_t$ is dynamically adjusted to maximize the growth of the e-process under failure conditions. Two strategies are evaluated:

1. aGRAPA (Approximate Growth-Rate Optimal): Uses empirical mean and variance of past metrics to calculate an optimal closed-form update.
2. SF-OGD (Scale-Free Online Gradient Descent): Updates $\lambda_t$ using the normalized gradient of the log-loss, making it robust to scale variations.

3. Key Contributions

1. Formal Statistical Framework: First to formulate tracking failure detection as a sequential hypothesis test, providing anytime-valid Type-I error control.
2. Model-Agnostic Design: The method acts as a "wrapper" around any tracking model (e.g., KCF, SiamFC, Transformers) and requires no retraining or fine-tuning of the underlying tracker.
3. Flexible Implementation: Supports both supervised (using GT) and unsupervised (using internal signals) configurations, making it applicable to real-world scenarios where GT is unavailable.
4. Efficiency: The approach is computationally lightweight, involving simple arithmetic operations on streaming data.

4. Experimental Results

The method was evaluated on four benchmarks (OTB-100, LaSOT, TrackingNet, GOT-10k) using two distinct tracking models (KCF and SiamFC).

• False Positive Rate (FPR): The empirical FPR consistently stayed below the theoretical bound of $\alpha = 0.1$ (often significantly lower, e.g., 0% on GOT-10k), validating the statistical guarantees.
• Average Detection Delay (ADD): The system demonstrated low detection delays.
  • Supervised metrics (NGIoU) provided stable and reliable alerts.
  • Unsupervised metrics were more volatile but still effective, particularly when using the SF-OGD betting strategy which reacted faster to sudden changes.
• Ablation Studies:
  • Recency Windows: Using a fixed window for past data (truncating old history) improved responsiveness in fast-changing scenarios without compromising validity.
  • Metric Choice: Supervised metrics were more robust, but unsupervised metrics (like Sharpness Gain) successfully detected failures where simple peak correlation failed.

5. Significance and Impact

• Safety Assurance: This work bridges the gap between high-performance deep learning trackers and the rigorous safety requirements needed for deployment in critical systems (e.g., autonomous vehicles).
• Operational Efficiency: By mathematically bounding false alerts, the system reduces unnecessary human interventions or expensive re-calibration procedures.
• Generalizability: The "testing-by-betting" framework is not limited to tracking; the authors suggest it can be applied to other streaming vision tasks like anomaly detection, action recognition, and multi-object tracking.
• Practicality: It offers a plug-and-play solution that enhances existing pipelines without modifying the core tracking algorithms.

In summary, the paper proposes a statistically rigorous, lightweight, and flexible mechanism to monitor object trackers in real-time, ensuring that failures are detected quickly while strictly controlling the rate of false alarms.