⚡ electrical engineering

Multi-Step Deep Koopman Network (MDK-Net) for Vehicle Control in Frenet Frame

This paper proposes a Multi-Step Deep Koopman Network (MDK-Net) that utilizes deep learning to create a globally linear representation of nonlinear vehicle dynamics in the Frenet frame, enabling a computationally efficient Model Predictive Control (MPC) strategy that significantly outperforms traditional linear models in complex maneuvers like double lane changes.

Original authors: Mohammad Abtahi, Mahdis Rabbani, Armin Abdolmohammadi, Shima Nazari

Published 2026-01-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mohammad Abtahi, Mahdis Rabbani, Armin Abdolmohammadi, Shima Nazari

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot car how to drive itself. The biggest problem isn't the steering wheel or the gas pedal; it's the car's personality. Cars are complicated. When you turn the wheel, the car doesn't just turn instantly; it slides, the tires grip and slip, the engine revs, and the suspension bounces. These are "nonlinear" behaviors, meaning they don't follow a simple, straight-line rule.

Trying to predict exactly how a car will move is like trying to predict the weather: it's messy, chaotic, and hard to get right.

The Old Way: The "Local Map"

Traditionally, engineers tried to solve this by making a simplified, linear map of how the car moves. Think of this like drawing a map of your neighborhood that only works if you stay on the sidewalk. If you step off the sidewalk (drive fast, turn sharply, or hit a bump), the map becomes useless. This is why old methods often fail when a car needs to make a quick, dramatic maneuver, like a double lane change to avoid an obstacle.

The New Way: The "Koopman Lens"

This paper introduces a new tool called the Koopman Operator. Imagine you have a pair of special glasses (a "lens"). When you look at the car through these glasses, the messy, chaotic, non-linear movements suddenly look like a simple, straight line.

In the real world, the car's movement is a tangled knot. The Koopman operator is a mathematical trick that "lifts" the car's state into a higher dimension (a higher floor of reality) where that knot untangles itself into a straight line. Once it's a straight line, it's very easy to predict where the car will be in the future.

The Innovation: The "Deep Learning Translator"

The tricky part of using these special glasses is figuring out how to make them. Usually, engineers have to guess which "lenses" to use, often by trial and error.

The authors of this paper built a Deep Learning Translator (called the MDK-Net). Instead of guessing, they fed the computer thousands of hours of driving data from a high-fidelity simulator (CarSim). The AI learned on its own how to create the perfect "lens" to turn the car's messy reality into a simple, predictable line.

Key features of their translator:

It sees the road: Most car models just look at the car. This one also looks at the road's curves (curvature) as a separate input, like a passenger pointing out a turn ahead.
It feels the pedals: It doesn't just look at "steering." It understands the raw inputs: how hard the driver is pressing the gas, the brake, and turning the wheel. This lets it understand the engine and drivetrain, not just the wheels.
It looks ahead: Instead of just guessing the next second, it learns to predict a whole sequence of future steps at once, ensuring the car stays stable.

The Test: The "Double Lane Change"

To see if this worked, the researchers put the car through a "double lane change" test. This is like a slalom course where the car has to swerve left, then right, very quickly.

The Old Model (Linear): It got confused. It thought the car would go straight when it should have turned, causing the car to drift off the path.
The New Model (MDK-Net): It predicted the car's movement with high precision. It knew exactly how the car would slide and recover.

The Result: Fast and Accurate

The best part? Even though the new model is smarter and more complex, it runs just as fast as the old, simple models.

They used this new model to build a Model Predictive Control (MPC) system. Think of MPC as a co-pilot that constantly asks, "If I turn the wheel this much, where will I be in 5 seconds? If I brake now, where will I be?"

The Old Co-pilot (using the linear model) made mistakes and drifted off the track.
The New Co-pilot (using the MDK-Net) stayed perfectly on the line, tracking the desired path with almost zero error.

Summary

In short, the authors created a smart AI system that learns to translate the chaotic, messy physics of a real car into a simple, predictable language. This allows a self-driving car to make complex, high-speed maneuvers safely and accurately, without needing a supercomputer to do the math. It's like giving the car a crystal ball that works even in the most chaotic driving conditions.

Technical Summary: Multi-Step Deep Koopman Network (MDK-Net) for Vehicle Control in Frenet Frame

Problem Statement
The practical implementation of optimal and Model Predictive Control (MPC) strategies for autonomous vehicles is hindered by the highly nonlinear dynamics of vehicle systems, which arise from complex interactions between lateral/longitudinal motion, tire forces, and powertrain dynamics. While nonlinear control techniques (e.g., NMPC, iLQR) offer high accuracy, their computational complexity often renders them unsuitable for real-time applications. Conversely, traditional linearization methods (e.g., Taylor series) provide computational efficiency but fail to capture dynamics across a wide range of operating points, limiting their validity to local regions. Existing data-driven approaches, such as Extended Dynamic Mode Decomposition (EDMD), struggle with the manual selection of observable functions, which often requires extensive trial and error to accurately capture system dynamics.

Methodology
The authors propose a Multi-Step Deep Koopman Network (MDK-Net), a deep learning framework designed to learn a global linear representation of nonlinear vehicle dynamics within a curvilinear Frenet frame.

Framework Architecture: The model employs a deep neural network to learn the Koopman operator and the lifting (observable) functions simultaneously.
- State Representation: The system operates in a Frenet frame where states include longitudinal velocity ( $v_x$ ), lateral velocity ( $v_y$ ), yaw rate ( $\dot{\psi}$ ), progress differential ( $\Delta s$ ), lateral deviation ( $e_y$ ), and heading error ( $e_\psi$ ).
- Inputs: The control inputs consist of low-level actuators: throttle, brake pressure, and steering angle. Crucially, road curvature ( $\kappa$ ) is treated as an exogenous input, a distinction the authors note is novel for Koopman-based vehicle modeling.
- Encoder: A fully-connected Multilayer Perceptron (MLP) acts as an encoder $\phi(\cdot)$ , mapping original states $x_k$ to a latent space $Z_k$ . The observable vector $\psi(x_k)$ is constructed by concatenating the original states with these latent states: $\psi(x_k) = [x_k^\top, Z_k^\top]^\top$ .
- Linear Dynamics: The dynamics in the lifted space are approximated as $\psi(x_{k+1}) = A\psi(x_k) + Bu_k$ , where matrices $A$ and $B$ are learned parameters representing the Koopman operator.
Training and Loss Functions: Unlike traditional Koopman methods that focus on single-step prediction, MDK-Net utilizes a composite loss function to ensure long-term accuracy and stability:
1. Single-Step Loss ( $L_{ssl}$ ): Penalizes deviation in one-step lifted state predictions.
2. Multi-Step Loss ( $L_{msl}$ ): Penalizes errors over a prediction horizon, using a forgetting factor $\beta$ to prioritize near-future accuracy.
3. Stability Loss ( $L_{sl}$ ): Explicitly penalizes any eigenvalue of matrix $A$ with a magnitude greater than unity, ensuring bounded multi-step predictions.
4. Regularization Loss ( $L_{regl}$ ): Prevents overfitting of encoder and operator parameters.
Control Application: The trained Koopman model is deployed within an MPC framework. The optimization problem is formulated in the lifted observable space, treating curvature as a constant exogenous input over the horizon.

Key Contributions
The paper identifies four primary contributions:

Frenet Frame Integration: This is the first application of deep learning-based Koopman operators to vehicle dynamics in Frenet-frame coordinates, explicitly incorporating road curvature as an exogenous input.
Low-Level Input Modeling: The model utilizes raw control inputs (steering, brake, throttle) to capture full powertrain and driveline dynamics, moving beyond the simplified tire/powertrain representations common in prior studies.
End-to-End Architecture: The authors developed a novel architecture that jointly learns observable functions and the Koopman operator without requiring an auxiliary decoder to map control inputs back to the original state space.
Stability Enforcement: A specific stability loss term was introduced to penalize unstable eigenvalues, ensuring the learned dynamics remain bounded for multi-step predictions.

Results
The model was trained and tested using data from a high-fidelity CarSim simulation environment (C-Class hatchback, 7,000 episodes).

Prediction Accuracy: In open-loop testing over 2,400 trajectories (80-step horizon), the MDK-Net significantly outperformed a Linear Time-Invariant (LTI) model identified via MATLAB's System Identification Toolbox.
- For critical lateral dynamics ( $v_y$ and $e_y$ ), the MDK-Net achieved Mean Squared Error (MSE) values of 0.020 and 2.504, respectively, compared to the LTI model's 0.548 and 39.303.
- The MDK-Net accurately captured the evolution of all state variables in a double lane-change maneuver, whereas the LTI model exhibited significant drift, particularly in lateral and heading errors.
Control Performance: An MPC controller utilizing the MDK-Net model successfully tracked a reference trajectory in a double lane-change scenario with minimal error. In contrast, the LTI-based MPC controller showed noticeable drift and failed to adequately track lateral and angular velocities.
Computational Efficiency: Despite the higher dimensionality of the lifted state space, the convexity of the Koopman-based MPC optimization problem ensured computational efficiency comparable to linear MPC. For a 10-second simulation, the MDK-Net MPC took 7.4 seconds to compute, compared to 3.2 seconds for the LTI MPC, on an Intel Core i7-1165G7 processor. The authors note this is viable for real-time implementation, especially when contrasted with the high resource demands of conventional Nonlinear MPC.

Significance and Claims
The paper claims that the MDK-Net framework successfully bridges the gap between the accuracy of nonlinear modeling and the computational efficiency required for real-time control. By leveraging deep learning to automatically discover observables and enforcing stability through specific loss terms, the approach provides a robust, globally linear representation of vehicle dynamics that is superior to traditional linearization and more computationally feasible than nonlinear control methods. The authors position this work as a foundational step toward more efficient and accurate autonomous vehicle control, particularly in scenarios involving complex road geometries (curvature) and full vehicle dynamics.