💻 computer science

NeSAM: Neuro-Symbolic Kinodynamics with Soil Adaptation for Off-Road Mobility

The paper presents NeSAM, a neuro-symbolic framework that integrates differentiable Bekker-Wong terramechanics with learned residual dynamics and online soil parameter estimation to significantly improve off-road vehicle kinodynamic prediction accuracy and navigation performance on deformable terrain.

Original authors: Chenhui Pan, Tong Xu, Francesco Cancelliere, Xuesu Xiao

Published 2026-08-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chenhui Pan, Tong Xu, Francesco Cancelliere, Xuesu Xiao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Driving a vehicle across a flat, paved road is a predictable affair. The wheels grip the surface, the engine turns them, and the car moves forward exactly as the laws of physics on a hard surface dictate. But take that same vehicle off the pavement and onto a field of soft dirt, loose sand, or uneven mud, and the rules change instantly. The ground is no longer a solid stage but a living, shifting participant. As a wheel presses down, it sinks into the earth; as it turns, it shears the soil sideways. These deformations create forces that pull the vehicle off course, causing it to slide, tilt, or get stuck in ways that simple steering commands cannot easily correct. For robots and autonomous vehicles to navigate these rugged environments safely, they must be able to predict not just where they are going, but how the ground beneath them will react to their weight and movement. Without this foresight, a vehicle might plan a path that looks clear on a map but leads to a deep rut or a sudden loss of control the moment it touches the soil.

Researchers have long tried to solve this problem using two different approaches. One method relies on classical physics equations that describe how soil behaves under pressure, using parameters like stiffness and friction to calculate forces. While these models are grounded in real-world mechanics, they often struggle to capture the complex, messy reality of a vehicle's suspension, tire deformation, and the specific quirks of a particular patch of ground. The other method uses artificial intelligence to learn from data, watching how a vehicle moves and trying to guess the next step. These learning-based models are flexible but often act as "black boxes," offering little insight into why a vehicle is slipping or sinking, and they can fail when faced with soil conditions they have never seen before. Neither approach alone has been sufficient to provide both high accuracy and a clear understanding of the physical forces at play.

To bridge this gap, a team of researchers has developed a new system called NeSAM, which stands for Neuro-Symbolic Soil Adaptive Mobility. This framework does not choose between physics and learning; instead, it weaves them together into a single, cohesive model. The system operates by first looking at the terrain ahead, using cameras to see both the shape of the ground and what kind of material it is made of, such as grass, dirt, or sand. It then uses a specialized component based on established soil mechanics to estimate how much the wheels will sink and how the soil will resist the vehicle's movement. This provides a solid, physics-based foundation for the prediction. However, because real-world conditions are rarely perfect, the system also employs a learning-based "residual" model. Think of this as a fine-tuning mechanism that learns the small, systematic differences between what the physics equations predict and what the vehicle actually does, correcting the estimate to match reality.

The true power of NeSAM lies in its ability to adapt in real time. As the vehicle moves, the system constantly compares its predictions with the actual motion it observes. If the vehicle sinks deeper than expected or slides more than the model predicted, the system uses a mathematical filter to update its internal understanding of the soil's properties. It effectively asks, "If the ground is behaving this way, what must the soil parameters be?" and then adjusts its model accordingly. This allows the vehicle to learn about the specific ground it is traversing as it goes, refining its predictions for the next few seconds of travel. The researchers tested this system in two ways: first, in a highly detailed computer simulation that mimics the complex physics of soil and vehicle interaction, and second, on a physical, four-wheeled robot driving over a real-world test course with varying terrain.

The results showed that this hybrid approach significantly outperformed existing methods. In the computer simulations, NeSAM reduced prediction errors by up to 30 percent compared to the best alternative models. When tested on the physical robot, it improved accuracy by nearly 29 percent. More importantly, the system proved its value when the vehicle had to navigate a path where the soil conditions were different from what it had been trained on. In these challenging scenarios, the version of the system without soil adaptation failed to complete the course in most trials, often losing control or getting stuck. In contrast, the adaptive version successfully completed the vast majority of its runs, keeping the vehicle much closer to its intended path and maintaining a more stable orientation. By combining the reliability of physical laws with the flexibility of machine learning, NeSAM offers a robust way for autonomous vehicles to understand and navigate the unpredictable, deformable world of off-road travel.

Technical Summary: NeSAM – Neuro-Symbolic Kinodynamics with Soil Adaptation for Off-Road Mobility

Problem Statement
Accurate prediction of off-road vehicle motion over deformable terrain is challenging due to complex interactions involving sinkage, slip, and traction that vary with local soil conditions. Conventional planar kinematic models fail to account for pressure-dependent sinkage and shear deformation, leading to discrepancies between predicted and realized vehicle motion. While classical terramechanics models (e.g., Bekker-Wong) offer physical interpretability, they rely on prescribed constitutive relationships that may not capture vehicle-specific effects like suspension compliance or tire deformation, and their soil parameters are difficult to estimate and vary across heterogeneous terrains. Conversely, purely learning-based kinodynamic models can capture complex nonlinearities but often treat terrain as rigid, lack explicit representation of soil mechanics, and offer limited physical interpretability, making online adaptation difficult.

Methodology
The authors propose NeSAM (Neuro-Symbolic Soil Adaptive Mobility), a framework that integrates differentiable terramechanics with learned terrain representations and a Transformer-based residual dynamics model. The system operates through the following components:

Neuro-Symbolic Architecture:
- Input: The model takes vehicle states ( $x_t$ ), control inputs ( $u_t$ ), and local terrain observations ( $m_t$ ), consisting of elevation patches ( $m^e_t$ ) and semantic patches ( $m^s_t$ ).
- Terrain Encoding: Separate U-Net autoencoders process elevation and semantic data to produce latent representations. A semantic soil-prior head ( $h_\theta$ ) maps semantic features to nominal soil parameters ( $\bar{\theta}_t$ ).
- Transformer Encoder: A causal Transformer processes a sequence of interaction tokens (combining past states, controls, and terrain contexts) to capture temporal dependencies.
- Dual-Head Prediction:
  - Sinkage Head: Predicts wheel sinkages ( $\hat{s}_t$ ) based on the interaction history, providing the deformation variables required by the terramechanics model.
  - State-Residual Head: Predicts a chassis-level correction ( $\Delta x^{res}_t$ ) to account for systematic errors not captured by analytical models (e.g., suspension dynamics, load transfer).
- Differentiable Terramechanics: The predicted sinkages, wheel angular velocities, and current soil estimates instantiate a differentiable Bekker-Wong model to compute wheel-terrain contact forces ( $f^{BW}_{i,t}$ ).
- Newton-Euler Propagation: These forces are aggregated and propagated through Newton-Euler equations to generate a physics-based state increment ( $\Delta x^{NE}_t$ ).
- Final Prediction: The next state is computed as $\hat{x}_{t+1} = x_t + \Delta x^{NE}_t + \Delta x^{res}_t$ .
Online Soil Adaptation:
- During deployment, the learned neural components remain fixed.
- An Extended Kalman Filter (EKF) updates the soil parameters ( $\theta$ ) online.
- The EKF uses a buffer of observed vehicle velocities versus predicted velocities to refine the soil correction ( $\Delta \theta$ ), allowing the system to adapt to mismatches between the nominal soil prior and actual terrain conditions.

Key Contributions

Neuro-Symbolic Framework: A novel architecture integrating differentiable Bekker-Wong terramechanics, learned terrain representations, and Transformer-based residual dynamics for long-horizon, six-degree-of-freedom (6-DoF) motion prediction.
Online Adaptation: An EKF-based method that updates physically interpretable terramechanics parameters using the discrepancy between predicted and measured motion, enabling adaptation to heterogeneous soil without retraining the neural network.
Comprehensive Evaluation: Validation in both the Chrono-based Verti-Bench simulator and on a physical Verti-4-Wheeler platform, demonstrating improvements over state-of-the-art learning-based baselines and ablation studies confirming the necessity of both the residual branch and the sinkage prediction.

Experimental Results
The paper evaluates NeSAM against pure end-to-end Transformer predictors and Terrain-Attentive Learning (TAL) variants:

Simulation (Verti-Bench): NeSAM reduced prediction errors by up to 30% compared to the strongest baselines. Specifically, it reduced x, y, and z errors to 0.80m, 0.78m, and 0.06m, respectively, and yaw error to 4.18°.
Physical Experiments (Verti-Arena): NeSAM improved prediction accuracy by 29% on real-world data relative to baselines, reducing x, y, and z errors by 17.1%, 15.7%, and 36.8% respectively compared to a Transformer baseline.
Closed-Loop Control: When integrated with a Model Predictive Path Integral (MPPI) controller under out-of-distribution (OOD) soil conditions:
- Simulation: Online soil adaptation increased trajectory completion from 0/5 to 4/5 and reduced the Hausdorff distance to the reference trajectory by 69.4%.
- Physical: Completion rates improved from 1/5 to 3/5, with mean absolute roll and pitch errors reduced by 21.2% and 23.0%, respectively.

Significance and Claims
The authors claim that NeSAM addresses the limitations of existing methods by explicitly modeling soil mechanics while retaining the flexibility of data-driven learning. The significance lies in the ability to:

Provide physically interpretable predictions that account for deformable terrain effects (sinkage, shear) which rigid-surface models miss.
Enable online adaptation of soil parameters, allowing the vehicle to maintain accurate trajectory tracking even when the actual soil properties differ from the semantic prior (OOD conditions).
Achieve superior long-horizon prediction accuracy in both simulation and real-world scenarios, facilitating more reliable predictive planning for off-road mobility.

The paper acknowledges a limitation: the framework assumes wheel-terrain interactions can be adequately represented by the Bekker-Wong formulation. While parameters can be adapted, the underlying constitutive relationships cannot be modified, potentially limiting performance on terrains that deviate significantly from these assumptions. Future work is suggested to explore richer differentiable terrain models.

Technical Summary: NeSAM – Neuro-Symbolic Kinodynamics with Soil Adaptation for Off-Road Mobility

More like this