⚡ electrical engineering

Occlusion-Aware Multi-Object Tracking via Expected Probability of Detection

This paper proposes an occlusion-aware multi-object tracking framework that enhances standard detection models by assigning each object an expected probability of detection based on its visibility relative to other objects, thereby systematically accounting for mutual occlusion uncertainties within a multi-Bernoulli mixture filter.

Original authors: Jan Krejčí, Oliver Kost, Yuxuan Xia, Lennart Svensson, Ondřej Straka

Published 2026-03-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jan Krejčí, Oliver Kost, Yuxuan Xia, Lennart Svensson, Ondřej Straka

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are standing in a busy, crowded room trying to keep track of a group of friends playing a game of tag. You are the "sensor" (your eyes), and your friends are the "objects."

In a perfect world, everyone stays in a straight line, and you can see everyone clearly at all times. But in reality, people walk in front of each other. When Friend A walks in front of Friend B, Friend B disappears from your view. This is called occlusion.

Most computer tracking systems are like a person with a very short attention span. If they lose sight of a friend because someone blocked them, they often give up and assume that friend has left the room. When the friend reappears, the system starts a new track, thinking it's a different person. This leads to confusion, lost identities, and safety risks (like a self-driving car thinking a pedestrian vanished).

This paper proposes a smarter way to handle this "crowded room" problem. Here is the breakdown using simple analogies:

1. The Problem: The "Guessing Game"

Current tracking methods often try to solve occlusion by making a quick guess: "I think Friend B is behind Friend A, so I'll just pretend Friend B is still there."

The Flaw: These guesses are often "ad-hoc" (made up on the spot). They don't mathematically account for how likely it is that Friend B is actually hidden. They might guess wrong, leading to tracking errors.

2. The Solution: The "Probability of Being Seen"

The authors introduce a concept called the Expected Probability of Detection (EPoD).

Think of this as a "Visibility Score" that updates in real-time.

Instead of just saying "I see Friend B" or "I don't," the system asks: "Given where everyone else is standing, what is the mathematical chance that I would see Friend B right now?"
If Friend B is partially hidden, the score drops (e.g., 40% chance of being seen).
If Friend B is fully hidden, the score drops to near zero.
If Friend B is in the open, the score is high (e.g., 90%).

3. The Secret Sauce: The "Reduced Palm Density"

This is the fancy math term in the paper, but here is the simple version:

Imagine you are trying to calculate the visibility of Friend B. To do this accurately, you need to know where everyone else is. But you don't know exactly where everyone else is; you only have a "best guess" (a probability distribution).

Old Way: The system picks one "best guess" for where everyone else is, calculates the visibility, and moves on. This is like looking at a blurry photo, squinting to pick one spot, and assuming that's the truth.
New Way (The Paper's Method): The system looks at all possible scenarios of where everyone else could be. It calculates the visibility score for every scenario and then averages them out.
- Analogy: Instead of guessing where the crowd is, the system runs a thousand simulations in its head: "If the crowd is here, visibility is X. If the crowd is there, visibility is Y." It then takes the average of all those possibilities.

This "averaging over all possibilities" is what the paper calls taking the expectation over the Reduced Palm Density. It ensures the system never gets fooled by a single bad guess about where the crowd is standing.

4. The Result: A Smarter Tracker

The authors tested this method (called MBM-PRO) on video footage of people walking in crowded streets.

The Outcome: The new tracker was much better at keeping track of people who were being blocked by others.
Why it matters: It didn't just "guess" that the person was there; it calculated the likelihood of them being there based on the physics of the crowd.
The Trade-off: The system became slightly more computationally heavy (it has to do more math), but it was much more accurate. It missed fewer people and didn't lose their identities as often as the old methods.

Summary Analogy

Old Tracker: A security guard who, when a person is blocked by a pillar, immediately stops looking for them. If they pop out the other side, the guard thinks, "Oh, a new person!" and starts a new file.
New Tracker (This Paper): A security guard who knows the layout of the room. When a person is blocked, the guard thinks, "Based on where the other people are standing, there is a 70% chance my target is still behind that pillar. I will keep my file open and wait."

In a nutshell: This paper teaches computers how to be better at "filling in the blanks" when objects hide behind each other, not by guessing, but by doing a rigorous mathematical calculation of how likely it is that the object is still there. This makes tracking systems safer and more reliable for things like self-driving cars and surveillance.

1. Problem Statement

The paper addresses Multi-Object Tracking (MOT) in scenarios where objects occlude one another relative to a sensor (e.g., a camera).

The Core Issue: Standard tracking models, such as the Standard Point-Object (SPO) model, assume that the Probability of Detection (PoD) for an object depends only on its own state. However, in occlusion scenarios, the PoD of an object $x_k$ depends on the presence and state of all other objects $X_k \setminus \{x_k\}$ .
Limitations of Existing Methods:
- Ad-hoc approaches: Many current methods (e.g., re-ID features, heuristic visibility variables) handle occlusion inconsistently with probabilistic principles, making them unsuitable for safety-critical applications where error quantification is required.
- Intractable Model-Based approaches: Fully model-based approaches (Bayesian) that account for occlusion dependencies often lead to intractable posterior distributions. Previous attempts to approximate these (e.g., substituting point estimates of other objects into the PoD function) fail to properly account for the spatial and existence uncertainties of the occluding objects.

2. Methodology

The authors propose a principled, fully model-based approximation that yields a tractable end-to-end algorithm. The core innovation is replacing the complex, state-dependent PoD with an Expected Probability of Detection (EPoD).

A. Theoretical Foundation: Reduced Palm Density (RPD)

To handle the dependency of an object's detection on others, the paper utilizes Palm conditioning.

Concept: The probability law of the set of other objects, given that a specific object $x$ exists.
Reduced Palm Density (RPD): Denoted as $p_{\Theta|x}(O|x)$ , this represents the density of the set of other objects $\Theta = \Xi \setminus \{x\}$ conditioned on $x \in \Xi$ .
Derivation: The authors derive the RPD analytically for common MOT densities, including Bernoulli, Poisson, and Poisson Multi-Bernoulli (PMB) distributions.

B. The Proposed Approximation (PRO Strategy)

The authors formulate the problem as minimizing the Kullback-Leibler Divergence (KLD) between the true intractable posterior (SPO-D model) and a tractable approximation (SPO model with a modified PoD).

Optimal Parameter: They prove that the best-fitting PoD for a specific object (or its mark) is the conditional expectation of the true PoD over the RPD:
$P^*(x_k) = \mathbb{E}_{\Theta|x_k} [ P_D(x_k, \Theta)]$
The PRO Strategy:
1. P (Palm): Condition the prior distribution on the existence of a specific object.
2. R (Reduce): Compute the EPoD by integrating the PoD function over the RPD of the other objects.
3. O (Obey): Use this computed EPoD as a constant parameter within a standard SPO model to enable tractable filtering.

C. Implementation: MBM-PRO Filter

The method is implemented using a Multi-Bernoulli Mixture (MBM) filter with marks (labels).

Mark-Based EPoD: Instead of computing EPoD for every continuous state (which is computationally expensive), the authors compute a single EPoD value for each mark (label) associated with a Bernoulli component.
Efficiency Approximation:
- The calculation involves a sum over all subsets of other objects (exponential complexity).
- The authors introduce a simplification based on an axiom: "Objects occlude individually." If an object does not occlude the target on its own, it is ignored in the combination.
- They further approximate high-existence-probability objects as certain (probability = 1) to reduce the combinatorial space, making the computation feasible for real-time tracking.

3. Key Contributions

Principled Occlusion Handling: A fully model-based derivation of occlusion handling that consistently accounts for the uncertainties (spatial and existence) of all objects, avoiding ad-hoc heuristics.
Analytical RPD Derivation: Explicit derivation of the Reduced Palm Density for standard MOT densities (Bernoulli, Poisson, PMB), enabling the calculation of EPoD.
Tractable MBM-PRO Filter: A practical implementation of the theory using MBM filters with marks, including an efficient approximation algorithm for computing EPoD.
Optimality Proof: Mathematical proof (via KLD minimization) that the proposed EPoD is the optimal parameter to approximate the intractable SPO-D model.

4. Experimental Results

The method was evaluated on the MOT-17 dataset (pedestrian tracking using monocular cameras) and compared against:

Baseline: Standard MBM with constant PoD.
Existing Model-Based: MBM with the "ESO" strategy (Estimate-Substitute-Obey), which uses point estimates of other objects.
Ad-hoc CV Methods: SORT and FastTracker.

Key Findings:

Performance: The MBM-PRO filter outperformed the standard MBM and the ESO strategy in terms of TGOSPA (Trajectory Generalized Optimal Sub-pattern Assignment) metrics, particularly in handling occluded pedestrians.
Visibility: MBM-PRO tracked more visible pedestrians with similar accuracy to ESO but significantly fewer missed detections (False Negatives) for occluded targets.
Comparison to Ad-hoc: While FastTracker (an ad-hoc deep learning method) achieved the highest scores overall (due to processing full images), MBM-PRO outperformed SORT and the basic MBM. Crucially, MBM-PRO provides probabilistic error quantification, which ad-hoc methods lack, making it superior for safety-critical applications.
Trade-offs: The PRO strategy resulted in slightly more False Positives and track switches compared to ESO, but this was a necessary trade-off for maintaining track continuity during occlusion.

5. Significance

Safety-Critical Applicability: By providing a rigorous probabilistic framework, this method allows for reliable uncertainty quantification in occlusion scenarios, which is essential for autonomous driving and surveillance.
Domain Independence: While demonstrated on visual tracking, the framework is sensor-agnostic and can be applied to LiDAR, radar, or any sensor where objects can occlude one another.
Theoretical Advancement: It bridges the gap between complex physical occlusion models and tractable Bayesian filtering, moving the field away from heuristic "workarounds" toward mathematically sound approximations.

In summary, the paper presents a robust, theoretically grounded solution to the multi-object tracking occlusion problem, demonstrating that accounting for the uncertainty of occluding objects via Expected Probability of Detection significantly improves tracking performance and reliability.