🤖 AI

CoAnchor: Robust Collaborative Perception under Spatio-Temporal Misalignment via Object-Level Anchors

The paper proposes CoAnchor, an anchor-centric framework that unifies spatial refinement, temporal propagation, and verification via sparse object-level anchors to robustly address joint communication delays and pose misalignments in collaborative perception.

Original authors: Chi Li, Rui Lin, Aobo Ji, Dongzhu Xu

Published 2026-08-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chi Li, Rui Lin, Aobo Ji, Dongzhu Xu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Autonomous vehicles are learning to see the world not just through their own eyes, but by sharing what they see with nearby cars. This practice, known as collaborative perception, allows a vehicle to detect objects hidden behind buildings or far down the road by combining its own sensor data with information sent from neighbors. It is a powerful idea that promises to make driving safer and more efficient. However, for this sharing to work, the information must arrive perfectly synchronized. In the real world, messages between cars are rarely perfect; they suffer from two main problems. First, there is a delay, meaning the information arrives a fraction of a second after it was captured, making it slightly outdated. Second, the cars often have small errors in knowing exactly where they are relative to one another, causing the shared images to be slightly shifted or misaligned. If a vehicle tries to combine these delayed and shifted messages with its own current view, the result can be confusing: the car might see a ghost vehicle where none exists, or it might miss a real one entirely.

Researchers at Beijing University of Posts and Telecommunications have developed a new method called CoAnchor to solve this specific problem of mismatched timing and position. Instead of trying to force the raw, complex data from all cars to fit together perfectly, the team changed the approach. They realized that trying to align every single pixel of the sensor data was too difficult and prone to error when the messages were delayed and the positions were noisy. Instead, they decided to focus on the objects themselves. They created a system that first identifies the specific vehicles and obstacles in the scene, treating each one as a distinct point of reference. These points act as anchors, or fixed markers, that the system can use to check and correct the information before it is combined.

The process works like a careful verification loop. When a car receives a message from a neighbor, it does not immediately trust the location of the objects described in that message. Instead, it first matches the neighbor's objects with its own current observations to create a rough alignment. Then, it uses a simple prediction model to move those neighbor objects forward in time, accounting for the delay, to see where they should be right now. Crucially, the system then checks if these predicted positions make sense against what the car sees at that exact moment. If a predicted object aligns well with a real observation, the system marks it as reliable. If it does not match, or if the position seems off, the system downgrades its confidence or discards it. This cycle of predicting, checking, and adjusting happens in a closed loop, allowing the system to refine the positions of the objects and the relative location of the cars simultaneously. Only after this rigorous checking is done does the system combine the data to make a final decision about what is on the road.

The researchers tested this method using both large-scale computer simulations and real-world driving data collected from actual vehicles. They found that while other methods could perform well in perfect conditions, they struggled significantly when both time delays and position errors were present at the same time. In these difficult scenarios, the new method proved much more robust. On a real-world dataset with a delay of 200 milliseconds and a position error of 0.6 meters, the CoAnchor system achieved a detection accuracy score of 67.04, which was notably higher than the next best method. This improvement meant the system was far less likely to create false alarms or miss dangerous obstacles. The study suggests that by focusing on the objects as anchors and verifying them against current reality, vehicles can share information effectively even when their communication is imperfect, keeping the road safer without requiring perfect synchronization.

Technical Summary: CoAnchor

Problem Statement

Collaborative perception enhances autonomous driving by fusing observations from multiple agents, extending the sensing range beyond a single vehicle's field of view. However, real-world deployments face two coupled challenges that degrade performance:

Temporal Misalignment: Communication delays cause received collaborator messages to be "stale" (captured at an earlier timestamp $u_j$ than the ego vehicle's current time $t$ ).
Spatial Misalignment: Localization noise and relative-pose errors place collaborator observations at incorrect locations within the ego frame.

Existing methods typically address these issues separately. Spatially oriented methods refine poses but fail when observations are temporally stale. Temporally oriented methods compensate for delays but assume the spatial reference is reliable. The paper argues that simply cascading these modules (e.g., spatial correction followed by temporal compensation) is insufficient because residual spatial errors propagate into the temporal stage, causing "motion ghosts," duplicated responses, and shifted detections. Furthermore, high-dimensional Bird's Eye View (BEV) features used in dense fusion lack explicit object correspondence, making it difficult to jointly reason about alignment and propagation under noise.

Methodology: CoAnchor

The authors propose CoAnchor, an anchor-centric spatio-temporal alignment framework. Instead of operating directly on dense BEV features, CoAnchor constructs a sparse, low-dimensional object-level interface (spatio-temporal anchors) to tightly couple spatial refinement, temporal propagation, and reliability verification within a unified closed loop.

The pipeline consists of four main stages:

1. Spatio-Temporal Anchor Initialization

Object State Representation: Agents detect objects and estimate motion states (position, velocity, heading) from short histories.
Bipartite Matching: Delayed neighbor detections are roughly warped into the ego frame. A bipartite matching algorithm (Hungarian algorithm) associates ego objects with delayed neighbor objects based on spatial distance, velocity similarity, and local neighborhood consistency.
Trajectory-Guided Pose Refinement: Unlike single-frame matching, CoAnchor utilizes the short motion history of matched pairs. It employs Iteratively Reweighted Least Squares (IRLS) to refine the relative pose transform ( $\Delta T$ ) by minimizing the error across matched trajectories. Pairs with lower matching costs start with higher reliability weights.
Anchor Construction: Matched pairs are converted into "delayed spatio-temporal anchors" with associated states, covariances, and reliability scores.

2. Object Temporal Propagation

State Prediction: Delayed anchors are propagated from the delayed timestamp $u_j$ to the current ego time $t$ using a rule-based constant-velocity motion model.
Ego-Guided Correction: If a reliable current-time ego observation exists for a propagated anchor, a Kalman-style measurement update is performed. The system computes the normalized innovation squared ( $d^2$ $d^{2}$ ) to verify consistency.
- If consistent, the anchor is updated.
- If inconsistent (e.g., due to occlusion or missed detection) or if no correspondence exists, the hypothesis is downgraded to an "ordinary" propagated object, which does not participate in subsequent closed-loop feedback.

3. Closed-Loop Posterior Scoring

Reliability Feedback: Before the next pose refinement round, the system evaluates the stability of matched pairs. It combines the current-time consistency ( $d^2$ ) with a short-history disagreement term ( $r_{hist}$ ) to compute a feedback residual.
Weight Update: Pair weights are exponentially downweighted based on this residual. Stable pairs retain high weights for the next IRLS pose update, while drifting or unreliable correspondences are suppressed. This creates a feedback loop where pose refinement is guided only by trustworthy anchors.

4. Anchor-Guided Feature Fusion

Feature Warping: The final refined pose is used to warp the delayed high-dimensional neighbor features into the current ego frame.
Box-wise Adjustment: A non-learnable, deterministic "box-wise feature mover" extracts the BEV rectangle of each delayed neighbor box and relocates it toward the corresponding propagated current-time box via bilinear sampling. This corrects the dominant foreground displacement before dense fusion.
Fusion: The corrected features are aggregated with ego features using a standard multi-scale fusion module (e.g., PointPillars backbone) for final detection decoding.

Key Contributions

Analysis of Joint Misalignment: The paper demonstrates that cascading separate spatial and temporal modules fails under coupled perturbations because residual spatial errors corrupt temporal propagation, leading to unstable fusion.
Anchor-Centric Framework: CoAnchor introduces a sparse object-level representation that serves as a shared interface for pose correction, temporal propagation, and reliability estimation, avoiding the complexity of implicit alignment in dense BEV spaces.
Unified Closed-Loop Design: The method tightly couples spatial refinement, temporal propagation, and current-time verification. It explicitly verifies the trustworthiness of collaborator hypotheses at the current time and feeds this reliability back to refine the pose, preventing the propagation of "ghost" detections.
Efficiency: By operating on sparse anchors rather than repeatedly processing dense features, the framework maintains a favorable accuracy-efficiency trade-off.

Experimental Results

The method was evaluated on OPV2V (simulated) and V2V4Real (real-world) datasets under various noise conditions, including "Joint-Hard" (200ms delay + 0.6m/0.6° pose noise).

Robustness: CoAnchor significantly outperforms state-of-the-art baselines (including V2X-ViT, CoAlign, TraF-Align, and a cascaded baseline baseST) under joint perturbations.
- On V2V4Real under Joint-Hard conditions, CoAnchor achieved 67.04 AP@0.5, surpassing the best baseline (baseST at 61.04) by 6.00 AP.
- On OPV2V under Joint-Hard, it reached 95.15 AP@0.5, outperforming baseST (90.19) by over 5 AP.
Clean Settings: In noise-free scenarios, CoAnchor remains competitive with existing methods, though not always the absolute top performer, indicating it does not sacrifice clean-case accuracy for robustness.
Qualitative Analysis: Visual results show CoAnchor produces fewer duplicate responses and motion ghosts compared to baselines, which often suffer from shifted boxes or missed detections when faced with coupled noise.
Efficiency: The closed-loop feedback adds minimal computational overhead. One round of feedback provides the majority of the robustness gain, with a second round offering only marginal improvements. CoAnchor is significantly faster than transformer-based baselines like V2X-ViT.

Significance and Claims

The paper claims that CoAnchor addresses a critical gap in collaborative perception: the inability of current methods to handle coupled spatio-temporal misalignment effectively. By shifting from dense feature alignment to a sparse, object-level anchor system with a closed-loop verification mechanism, CoAnchor achieves a robust balance between accuracy and efficiency.

The authors emphasize that their approach is particularly valuable for real-world deployments where communication delays and localization noise are inevitable. The method does not rely on perfect synchronization or precise relative poses; instead, it dynamically filters unreliable collaborator data and refines alignment based on current-time verification. The work suggests that future systems should move away from treating spatial and temporal alignment as independent stages, favoring instead a unified, anchor-driven approach that validates collaborator evidence before fusion.