💻 computer science

Comparing Trajectories from Positions Alone: Curvature-Based Time Alignment and Drift Error Metric

This paper proposes a standardized trajectory evaluation protocol for field robotics that combines a novel curvature-based temporal alignment method with a distance-normalized error metric to address the limitations of current assessment tools and ensure rigorous, reproducible accuracy comparisons.

Original authors: Effie Daum, Daniele De Martini, Claire Dune, François Pomerleau

Published 2026-09-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Effie Daum, Daniele De Martini, Claire Dune, François Pomerleau

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of robots that navigate the real world—hiking through forests, crossing snowy fields, or driving across deserts—knowing exactly where you are is the difference between success and failure. To build these machines, engineers create software that estimates a robot's path by combining data from cameras, lasers, and motion sensors. But to know if that software is any good, they must compare its guess against a "ground truth," a perfect record of where the robot actually went. In a laboratory, this record is easy to get, often using high-speed cameras that track every move. Out in the wild, however, getting such a perfect record is nearly impossible. The best we can usually do is a GPS signal or a laser tracker that tells us the robot's location in three-dimensional space, but it cannot tell us which way the robot is facing. Without knowing the direction, the standard tools used to measure robot accuracy break down, leaving engineers unable to fairly compare different navigation systems or understand why one might fail while another succeeds.

A team of researchers has developed a new way to solve this problem, allowing them to judge robot navigation using only these incomplete location records. Instead of trying to force a perfect match between two paths that might be slightly out of sync in time or space, they proposed a method that looks at the shape of the journey itself. Imagine two hikers walking the same trail; even if one starts a few seconds later or walks a slightly different route because of a loose shoe, they will both turn left at the same bend and right at the same fork. The researchers realized that the "curvature" of a path—how sharply it turns—acts as a unique signature that remains the same regardless of when the robot started or where its sensors are mounted. By matching these turning patterns, they can align two different records of the same journey with high precision, even if the clocks on the sensors were drifting apart or if the robot's sensors were not perfectly calibrated.

Once the paths are aligned, the team introduced a new way to measure error called "Drift Error." Traditional methods often fail when the reference data lacks direction, or they can be easily tricked by small mistakes in how the sensors are attached to the robot. The new metric simply compares the total distance traveled by the robot's estimate against the total distance traveled by the reference record over specific segments of the journey. If the robot thinks it walked ten meters but the reference shows it only walked nine, the system flags a drift. This approach is particularly powerful because it can reduce the impact of a common source of error known as a "lever arm" mistake. In robotics, sensors are often mounted at different points on the robot's body; if the software assumes they are in the wrong place relative to each other, the calculated path can be wildly inaccurate. The researchers demonstrated that their new method could mitigate these mounting errors under specific conditions, whereas standard tools would report massive, misleading errors.

To prove their method works, the team tested it on real-world data from two different robot missions. In one case, they looked at a robot navigating a forest and found that a tiny timing mismatch of just seventy milliseconds between sensors could inflate the reported error by sixty-five percent, completely changing which navigation system appeared to be the best. By using their curvature-based alignment, they corrected this timing and revealed the true performance of the systems. In another test, they simulated a scenario where the sensors were mounted with a significant error, shifting the robot's perceived position by half a meter. While a standard measurement tool reported a massive thirty-seven percent error, the new Drift Error metric, which accounted for the mounting shift, reported only a 4.5 percent error. This showed that the new protocol could distinguish between a robot that is actually failing and one that is simply suffering from a calibration mistake.

The researchers also carefully examined how different settings in their new system affected the results, ensuring that the method was reliable across various terrains and robot types. They found that the size of the path segments used for comparison mattered; if the segments were too short, the noise in the GPS signal would look like a failure, but if they were too long, the system might miss important details. They determined that using non-overlapping segments of about five meters provided the most honest picture of how much a robot was drifting. Their work does not claim to fix the robots themselves, but rather to fix the way we judge them. By providing a standardized, rigorous protocol that works even with imperfect data, they have opened the door to more honest comparisons of navigation software. This means that in the future, when engineers evaluate a new robot designed for harsh environments, they will have a clearer, fairer way to know if it is truly ready for the wild.

Technical Summary: Comparing Trajectories from Positions Alone

Problem Statement
In field robotics, evaluating state estimation, localization, and Simultaneous Localization and Mapping (SLAM) algorithms is hindered by the lack of independent, large-scale reference trajectories that are more accurate than the estimates themselves. While Absolute Trajectory Error (ATE) and Relative Pose Error (RPE) are standard metrics, they rely on assumptions often left unreported, such as perfect temporal synchronization, consistent spatial frame alignment, and full 6-Degrees-of-Freedom (DoF) reference poses. In outdoor environments, reference systems like Global Navigation Satellite Systems (GNSS) or Robotic Total Stations (RTS) typically provide only 3D positions without orientation. Attempting to reconstruct missing orientations using the same sensors (e.g., IMUs) used by the estimator compromises the independence of the ground truth. Furthermore, uncorrected temporal misalignments and extrinsic calibration errors (lever arms) can significantly distort error metrics, leading to misleading comparisons and unreliable benchmarking.

Methodology
The authors propose a standardized evaluation protocol designed to assess trajectories using 3D positions alone, without requiring known frame alignment or full 6-DoF reference poses. The framework consists of two primary components:

Curvature-Based Temporal Alignment:
To address temporal misalignment, the method exploits the rigid body property that all points on a moving platform share the same Instant Center of Rotation (ICR) at any given time. Consequently, the curvature signature ( $\kappa$ ) of the trajectory is invariant to the specific observation point on the body.
- Curvature is estimated for both the estimated and reference trajectories using a discrete approximation over a minimum arc length ( $\epsilon_{min}$ ).
- A normalized discrete cross-correlation is computed between the two curvature signals to find the lag ( $k^*$ ) that maximizes alignment.
- This lag is converted into a time offset ( $\Delta t$ ), which is applied to the estimated timestamps before evaluation. This process is robust to unknown spatial calibration and does not require speed profiles, which can be difficult to estimate accurately.
Drift Error (DE) Metric:
The paper introduces Drift Error (DE) as a metric normalized by the distance travelled, specifically designed for scenarios where only 3D positions are available.
- Definition: DE is defined as the ratio between the path lengths of the estimated and reference trajectories over a fixed window length $w$ (measured along the reference path): $DE = |1 - \frac{\Delta s_{est}}{\Delta s_{ref}}|$ .
- Lever Arm Correction: The metric accounts for errors in extrinsic calibration (lever arm). Since points at different radii from the ICR travel different arc lengths during rotation, the authors derive a correction factor $\alpha$ based on the ratio of curvature radii ( $r_{ref}/r_{est}$ ). This factor scales the estimated distance to match the reference, mitigating bias introduced by unknown sensor offsets.
- Degeneracy Handling: The method explicitly excludes segments with near-zero curvature (straight lines) or singular configurations (e.g., rotation around the reference point) where the lever arm correction becomes unbounded or undefined.

Key Contributions
The paper makes three specific contributions:

A Temporal Alignment Method: A novel approach matching trajectory curvature signatures to recover time offsets without requiring synchronized clocks or full pose data.
Drift Error (DE): A formal trajectory error metric expressed relative to travelled distance, suitable for 3D position-only references.
A Standardized Evaluation Protocol: A comprehensive framework integrating temporal alignment, spatial registration (lever arm correction), and the DE metric, validated on real-world datasets.

Results and Validation
The protocol was validated using the GrandTour and FoMo datasets, which contain diverse trajectories from legged and wheeled robots in various conditions.

Impact of Time Calibration: In a case study using the GrandTour HEAP-1 mission, the authors demonstrated that even small temporal misalignments (e.g., $\pm 0.1$ s) could drastically alter the relative ranking of algorithms. For instance, a 0.1 s offset inflated the error of a Direct Lidar Inertial Odometry (DLIO) solution by 65%, reversing its performance ranking against a proprioceptive estimator. The proposed curvature-based alignment successfully recovered the correct offset, whereas standard spatial alignment methods failed in this context.
Robustness to Lever Arm Errors: Using the GrandTour EIG-2 mission, the authors injected a 0.5 m lateral offset (simulating a lever arm error) into the estimated trajectory. The standard displacement error (Euclidean distance) increased to 37%, whereas the proposed DE metric, with lever arm correction, reported only a 4.5% drift. This demonstrates DE's ability to distinguish between algorithmic drift and calibration errors.
Parameter Sensitivity: Sensitivity analyses on parameters such as minimum displacement ( $\epsilon_{min}$ ), expected lever arm ( $a$ ), and window length ( $w$ ) revealed that the metric is stable within reasonable bounds. The authors suggest setting $\epsilon_{min}$ to three times the expected standard deviation of the reference trajectory and using disjoint windows ( $\beta = 0\%$ ) to avoid artificial inflation of observation counts.

Significance
The paper argues that rigorous benchmarking in field robotics requires protocols that explicitly account for the imperfections inherent in real-world data acquisition, such as clock drift and calibration uncertainty. By enabling evaluation based solely on 3D positions, the proposed protocol expands the range of usable datasets for rigorous testing, moving beyond the limitations of indoor motion capture systems. The authors claim that their approach contributes to more reproducible, standardized, and fair comparisons of state estimation algorithms, particularly in large-scale outdoor environments where full 6-DoF ground truth is unavailable. The work highlights that uncorrected temporal and spatial misalignments can fundamentally undermine the validity of benchmark results, and that curvature-based analysis offers a robust solution to these challenges.

Technical Summary: Comparing Trajectories from Positions Alone

More like this