🤖 machine learning

Physics-Regularized Machine Learning for Proprioceptive Vehicle Localization Using Onboard Sensors

This paper introduces PRML2, a hybrid framework that integrates machine learning with a differentiable Kalman filter to achieve robust, real-time vehicle localization using onboard sensors, particularly excelling in degraded sensing conditions and low-friction environments.

Original authors: Abinav Kalyanasundaram, Karthikeyan Chandra Sekaran, Wolfgang Utschick, Michael Botsch

Published 2026-07-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Abinav Kalyanasundaram, Karthikeyan Chandra Sekaran, Wolfgang Utschick, Michael Botsch

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are driving a car in a thick fog where you can't see the road signs, and your GPS signal has completely disappeared. How does the car know where it is?

Usually, cars rely on a mix of GPS (like a map) and IMUs (sensors that feel acceleration, like your inner ear). But when the GPS fails, the "inner ear" sensors start to get confused and drift off course, like a person spinning in circles and losing their sense of direction.

This paper introduces a new system called PRML2 to solve this problem. It teaches a car to know its location using only the standard sensors already built into the car (like wheel speed, steering angle, and brake pressure), without needing expensive extra cameras or lasers.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Drifting" Inner Ear

Think of a standard car sensor (IMU) like a person trying to walk in the dark while counting their steps. If they take 1,000 steps, they might think they walked 1,000 meters. But if they take a slightly crooked step, that tiny error adds up. After a few minutes, they might think they are in a different city than they actually are. This is called drift.

2. The Solution: A "Smart Coach" and a "Physics Coach"

The authors created a team of two "coaches" to help the car stay on track:

The Smart Coach (Machine Learning): This is a highly advanced AI (a "Transformer" model) that looks at the car's sensors (how fast the wheels are spinning, how hard the brakes are pressed) and guesses the car's speed and direction. It's great at spotting patterns in the noise, but sometimes it might guess something impossible, like a car accelerating faster than physics allows.
The Physics Coach (The Kalman Filter): This is a traditional math rulebook that knows the laws of physics. It knows, for example, that a car cannot instantly teleport or accelerate faster than the friction of the tires allows.

3. The Secret Sauce: "Physics-Regularized" Training

In the past, these two coaches worked separately. The AI would guess, and the math rulebook would fix the guess later.

In this new system (PRML2), the two coaches are glued together during the learning process.

Imagine the AI is a student taking a test.
Usually, the teacher (the loss function) just grades the final answer.
In this new system, the Physics Coach sits next to the student while they are writing the test. If the student starts writing an answer that breaks the laws of physics (like a car flying), the Physics Coach gently nudges them back to reality before they finish the answer.
Because the system is "differentiable," the AI learns from these nudges. It doesn't just get a bad grade at the end; it learns how to think like a physicist while it learns from the data.

4. The "Physics Guard"

The paper also mentions a "Physics Guard." Think of this as a safety net or a speed limiter. Even if the AI gets excited and predicts a car is moving at 200 mph, the guard checks the "friction circle" (how much grip the tires have on the road) and says, "Whoa, that's impossible on this icy road. Let's cap it at 50 mph." This keeps the predictions realistic.

5. The Results: Better in Snow and Rain

The researchers tested this system on a real car.

The Test: They drove the car for 60 seconds without GPS.
The Comparison: They compared their new system against older methods (like standard math filters or other AI models).
The Outcome: The new system was the most accurate. It drifted the least.
The "Low-Friction" Test: They also tested it on a slippery, snowy track (low friction) that the AI had never seen before during training. Even though it was a new, tricky environment, the system still performed better than the others. This proves that by teaching the AI the "rules of physics," it can handle situations it hasn't practiced for.

Summary

The paper claims that by combining a powerful AI with a math-based "physics coach" that teaches the AI during its training, cars can figure out exactly where they are using only the cheap sensors they already have. This makes autonomous driving safer and cheaper, especially when GPS fails or the weather is terrible.

Key Takeaway: You don't need expensive new sensors to make a car smart; you just need to teach the existing sensors to respect the laws of physics while they learn.

Technical Summary: Physics-Regularized Machine Learning for Proprioceptive Vehicle Localization (PRML2)

Problem Statement

Accurate vehicle localization is fundamental for autonomous mobility, typically relying on the fusion of Inertial Measurement Unit (IMU) data with Global Navigation Satellite System (GNSS) signals. However, during GNSS outages, dead reckoning based on commercial-grade IMUs suffers from rapid drift accumulation. While multi-sensor fusion (e.g., LiDAR, cameras) can mitigate this, it increases system cost and complexity. Conversely, standard onboard sensors (steering wheel angle, wheel speeds, yaw rate) are cost-effective and proprioceptive (independent of external signals) but are prone to drift when used with classical Bayesian filters due to noise accumulation and model limitations.

Recent data-driven approaches have shown promise in handling sensor noise but often lack physical consistency and generalization across diverse driving conditions. Furthermore, the potential of Machine Learning (ML) to enhance odometry using only standard onboard sensors—without relying on IMUs or exteroceptive measurements—remains underexplored. This paper addresses the challenge of achieving robust, accurate, and real-time vehicle localization in GNSS-denied environments using exclusively onboard sensors, while ensuring the solution adheres to physical vehicle dynamics.

Methodology: PRML2 Architecture

The authors propose Physics-Regularized Machine Learning for Localization (PRML2), a hybrid framework that integrates a data-driven ML model with a physics-based Extended Kalman Filter (EKF). The core innovation is the end-to-end training of the ML model through a differentiable EKF, which acts as a physics-informed regularizer.

1. Architecture Components

Transformer-Based ML Model: The system employs a transformer architecture to process onboard sensor data (observation window $L$ $L$ ). The model maps sensor inputs to vehicle dynamic states ( $z_t$ $z_{t}$ ), including velocities, accelerations, and angular rates.
- Physics Guard Layer: A domain-knowledge layer is embedded to bound predictions within physical limits (e.g., Kamm's circle for acceleration based on tire-road friction, maximum velocity, and angular velocity limits). This prevents physically implausible outputs.
- Uncertainty Quantification: The model outputs not only mean estimates but also variance, allowing for adaptive measurement noise covariance in the subsequent filter.
Differentiable Extended Kalman Filter (EKF): The estimated dynamic states from the ML model serve as "pseudo-measurements" for a kinematic EKF. The EKF recursively updates the vehicle pose ( $T_k \in SE(3)$ ) using a constant acceleration motion model.
End-to-End Training: Unlike traditional pipelines where the filter is fixed, the EKF in PRML2 is implemented in a differentiable manner. This allows loss gradients (computed from the difference between the final EKF pose and ground truth) to backpropagate through the filter to the ML model parameters.

2. Physics-Regularized Learning

The paper posits that the differentiable EKF acts as a physics regularizer. By minimizing the final localization error through the filter, the training process implicitly penalizes ML predictions that violate temporal consistency or physical motion models. The loss function is decomposed to show that minimizing the total error is equivalent to minimizing the prediction error while adhering to a regularization term derived from the Kalman filter's innovation and gain matrices.

3. Training Strategy

Training occurs in two stages:

Pretraining: The ML model is trained purely on data to estimate dynamic states using a negative log-likelihood loss.
Fine-tuning: The model is fine-tuned by backpropagating the localization error through the differentiable EKF, enforcing physical consistency.

Key Contributions

PRML2 Framework: Introduction of a novel hybrid architecture combining a transformer-based ML model with a differentiable EKF for vehicle localization using only onboard sensors (wheel speeds, steering angle, yaw rate, etc.), eliminating the need for IMUs or external signals.
Physics-Regularized Learning: Theoretical and empirical validation that integrating a differentiable Kalman filter into end-to-end training serves as a physics-based regularizer, penalizing physical and temporal inconsistencies and improving generalization.
Low-Friction Generalization: Demonstration that the framework generalizes effectively to challenging low-friction (low- $\mu$ ) conditions, a critical capability for safety-critical autonomous driving.
Novel Dataset: Publication of a new dataset collected under heavy snow and low-friction conditions to support research in vehicle localization under degraded sensing scenarios.

Experimental Results

The method was evaluated on the publicly available ReV-StED dataset and the authors' new low-friction dataset.

Dynamic State Estimation: PRML2 significantly outperformed five baselines (including classical models, GRU/LSTM networks, and other differentiable filter approaches) in estimating vehicle dynamic states ( $v_x, \omega_z$ , etc.). Specifically, it achieved a 28% improvement in longitudinal velocity ( $v_x$ ) and 25% improvement in yaw rate ( $\omega_z$ ) estimation compared to its pretraining-only variant, highlighting the efficacy of the physics regularization.
Localization Accuracy: Over 60-second GNSS outage simulations, PRML2 achieved the lowest position Root Mean Squared Error (RMSE) of 1.5 m (compared to 2.1 m for the next best baseline, RNN-EKF) and the lowest maximum drift.
Generalization: When tested on the unseen low-friction (snow) dataset without retraining, PRML2 maintained the highest accuracy (RMSE 2.5 m) and lowest standard deviation, demonstrating superior robustness compared to other methods.
Real-Time Capability: The system was evaluated on an NVIDIA Orin edge device, achieving an inference latency of 25–32 ms, enabling operation at approximately 30 Hz, which satisfies real-time requirements for vehicle localization.

Significance and Claims

The paper claims that PRML2 provides a robust and cost-effective solution for vehicle localization under degraded sensing conditions. By integrating learning with physics-based priors, the framework overcomes the limitations of purely data-driven methods (lack of physical consistency) and classical filters (sensitivity to noise and drift).

The authors emphasize that their work demonstrates the untapped potential of standard onboard sensors for high-precision localization. The key significance lies in proving that differentiable filtering can effectively regularize deep learning models, ensuring they adhere to physical laws and generalize to out-of-distribution scenarios (such as icy roads) without requiring expensive additional sensors. The work concludes that this approach offers a viable path toward reliable, proprioceptive autonomy in real-world environments where GNSS is unavailable.