⚡ electrical engineering

A Hybrid End-to-End and Modular Control Architecture Toward Safe Vehicle Lateral Control: Combining Soft Actor-Critic with Model Predictive Control

This paper proposes a hybrid vehicle lateral control architecture that integrates a Soft Actor-Critic reinforcement learning policy with a constrained Model Predictive Controller via a monotone blending coefficient, achieving the adaptability of end-to-end learning while maintaining the safety guarantees and interpretability of model-based control.

Original authors: Farzaneh Tatari

Published 2026-08-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Farzaneh Tatari

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Driving a car involves a constant, subtle negotiation between the road and the machine. To stay in a lane, a vehicle must constantly adjust its steering, reacting to curves, wind, and the uneven grip of tires. For decades, engineers have relied on two main ways to teach a computer how to do this. The first is a rule-based approach, where the car follows a strict set of physics equations and pre-programmed limits. This method is predictable and safe, but it can be rigid; if the road conditions change in a way the rules didn't anticipate, the car might struggle. The second approach is learning-based, where a computer program watches thousands of driving examples and figures out the best moves on its own. This method is incredibly flexible and can handle complex, messy situations, but it is a black box: it is hard to know why it made a specific choice, and it offers no guarantee that it won't make a dangerous mistake in a situation it hasn't seen before.

The challenge for the future of autonomous driving is to combine the best of both worlds. We need a system that is as adaptable as a human learner but as reliable and understandable as a rule-based engineer. A new study by independent researcher Farzaneh Tatari tackles this exact problem by creating a hybrid controller for the side-to-side movement of a car. The research does not rely on a single method but instead fuses a learning-based policy with a rule-based safety net. The goal is to see if a car can learn to drive smoothly and accurately while still being bound by a safety system that prevents it from ever exceeding physical limits or making erratic moves.

To test this idea, the researcher built a digital simulation of a car moving at a steady speed of 15 meters per second. In this virtual world, she set up four different drivers to see how they performed. The first was a standard, simple controller that has been used in cars for years. The second was a sophisticated, rule-based system that constantly calculates the best path forward while respecting strict limits on how fast the steering wheel can turn. The third was a pure learning system, trained from scratch to minimize errors without any pre-written rules. The fourth, and most important, was the new hybrid system. This system takes the steering command from the learning AI and the steering command from the rule-based calculator and blends them together into a single instruction. It uses a single dial to adjust the mix: turning the dial one way gives more weight to the flexible learning AI, while turning it the other way gives more weight to the safe, rule-based calculator.

The results of the simulation showed that the learning AI was the most accurate at keeping the car in the center of the lane, making fewer errors than the traditional rule-based systems. However, the pure learning AI had a weakness: in a few extreme situations that were very different from its training, it occasionally made a mistake that caused the car to drift too far off course. The hybrid system retained the high accuracy of the learning AI while ensuring that every steering command stayed within the safe physical limits of the car. In the vast majority of tests, the hybrid car performed almost as well as the pure learning AI. However, in the rare, extreme cases where the pure learning AI failed, the hybrid system inherited those exact same failures. Because the learning AI was so dominant in the mix, the rule-based safety component could not override it quickly enough to prevent the error in those specific corner cases.

The study also explored what happens when the car encounters conditions it was not trained for, such as slippery roads or sudden, large errors in its starting position. The hybrid system proved robust, maintaining its accuracy even when the tire grip was weaker than expected or when the car started with a significant sideways drift. However, the researcher found a specific limit to this safety. In a few very rare corner cases, where the car started with a large error and the learning AI tried to steer in a direction that was already at its physical maximum, the blend could not correct the mistake fast enough. This happened because the learning AI was so dominant in the mix that the rule-based safety component could not override it quickly enough to prevent a crash.

This finding led to a crucial insight about how such a system should work in the real world. The researcher proposed that the "dial" controlling the mix should not be fixed. Instead, it should be connected to outside information, such as data from other cars or road sensors. If the system detects that the car is entering a situation it has never seen before, or if the road conditions are dangerous, the dial should automatically shift to give more power to the safe, rule-based calculator. This would allow the car to remain flexible and efficient on normal roads but instantly become more conservative and predictable when the situation becomes uncertain.

The work demonstrates that it is possible to build a vehicle controller that learns from experience without losing the safety guarantees of traditional engineering. The hybrid approach successfully retained the smooth, accurate tracking of the learning system while keeping the car within its physical boundaries. While the current version is a simplified blend of two signals, the research points toward a future where a more advanced version could solve the remaining rare failures. By connecting the controller to a network of vehicles and road infrastructure, the system could dynamically adjust its behavior, ensuring that the car remains safe even when the road throws a curveball it has never seen before. This represents a significant step toward vehicles that are not only smart enough to learn but also wise enough to know when to rely on their safety rules.

Technical Summary: A Hybrid End-to-End and Modular Control Architecture Toward Safe Vehicle Lateral Control

Problem Statement
Connected and automated vehicles (CAVs) require lateral controllers that balance accuracy, low control effort, and safety under conditions of model error, sensor noise, and unmodeled disturbances. The literature is currently dominated by two distinct approaches, each with significant limitations:

Modular Controllers (e.g., MPC, PID): These rely on physics-based models and explicit constraints, offering interpretability and safety guarantees. However, they are sensitive to model inaccuracies and require extensive hand-tuning.
End-to-End Learned Policies (e.g., Deep RL): These adapt to nonlinearities and unmodeled dynamics without hand-designed control laws but lack intrinsic safety guarantees, interpretability, and robustness under distribution shifts.

The paper addresses the challenge of combining the adaptability of learned policies with the safety and predictability of modular control without sacrificing the strengths of either.

Methodology
The author proposes a hybrid architecture that linearly blends a trained Soft Actor-Critic (SAC) policy with a constrained linear Model Predictive Control (MPC) controller.

Vehicle Model: Experiments utilize a four-state linearized lateral bicycle model at a constant longitudinal speed of 15 m/s.
Modular Component (MPC): A linear MPC is formulated as a quadratic program (QP) over a 15-step horizon with explicit input ( $|\delta| \leq 0.4$ rad) and input-rate ( $|\dot{\delta}| \leq 3$ rad/s) constraints. It is tuned via grid search over 243 weight configurations.
End-to-End Component (SAC): A SAC policy is trained from scratch on the same plant using a reward function penalizing lateral error, heading error, steering magnitude, and steering rate. Training occurs in an episodic simulator with randomized initial conditions.
Hybrid Architecture: At each time step $k$ , the controller computes a convex combination of the MPC's first-step optimum ( $\delta_{MPC}$ ) and the SAC policy's output ( $\delta_{SAC}$ ). The final steering command is:
$\delta_k = \text{sat}_{\delta_{max}} \left( (1 - \alpha(\lambda)) \delta_{MPC} + \alpha(\lambda) \delta_{SAC} \right)$
where $\alpha(\lambda) = \lambda / (1 + \lambda)$ is a monotone blending coefficient. The result is saturated to the same actuator limits used by the MPC.
Evaluation: The system is benchmarked against PID, tuned linear MPC, and stand-alone SAC across nominal conditions, robustness cases (large initial errors, process noise, tire-stiffness mismatch), and a multi-initial-condition ensemble (5x5 grid crossed with 5 training seeds).

Key Contributions

Benchmarking: A comprehensive comparison of four controllers (PID, MPC, SAC, Hybrid) on a linearized lateral bicycle model using shared metrics (RMSE, peak error, mean absolute steering, final error).
Hybrid Formulation: The proposal of a closed-form blending mechanism that interpolates between a constrained model-based anchor and a learned policy, ensuring the output remains within the actuator envelope.
Robustness Analysis: Evaluation of the hybrid's performance under significant perturbations, including process noise and tire-stiffness mismatches.
Limitation and Extension Analysis: A detailed discussion of the current implementation's inability to guarantee recursive feasibility or prevent all corner-case divergences, motivating a connectivity-aware extension where the blending coefficient is scheduled based on Vehicle-to-Everything (V2X) signals.

Results

Nominal Performance: The SAC policy achieved the lowest lateral-error RMSE (0.0167 m), outperforming PID by 48% and linear MPC by 41%. The hybrid controller (with $\lambda=12$ ) closely matched SAC's tracking quality (RMSE 0.0171 m) while maintaining the MPC's actuator envelope and providing a deterministic, model-based contribution to every command.
Robustness: In all four robustness cases (large lateral error, large heading error, process noise, and tire-stiffness mismatch), SAC and the hybrid achieved the lowest RMSE, reducing errors by 32–49% compared to PID and MPC.
Corner-Case Divergence: In a multi-initial-condition ensemble (125 pairs), PID and MPC remained convergent on all trials. SAC and the hybrid diverged in 3 out of 125 cases (2.4%), specifically at the extreme boundaries of the training distribution (e.g., $\pm 0.4$ m lateral error, $\pm 0.10$ rad heading error).
Failure Mode: The analysis revealed that at the chosen blending weight ( $\alpha \approx 0.92$ ), the model-based contribution is bounded to $\approx 0.031$ rad. This is insufficient to counteract a saturated SAC command in the wrong direction within a single sample, leading to divergence.

Significance and Claims
The paper claims that the hybrid architecture successfully retains the tracking quality of stand-alone SAC while preserving the safety constraints and interpretability of the modular MPC. The primary value of the hybrid is not necessarily a reduction in nominal RMSE (where SAC already excels) but in the provision of a tunable interface (the blending coefficient $\lambda$ ) that allows system integrators to adjust the balance between adaptability and safety.

The author modestly notes that the current implementation is a closed-form convex blend and not a full predictive safety filter. Consequently, it does not guarantee recursive feasibility or terminal invariance, and it cannot prevent failure under severe distribution shifts, only attenuate their magnitude.

The paper concludes that the architecture is a stepping stone toward a more rigorous safety filter. The proposed path forward involves:

Replacing the closed-form blend with a full constrained-QP predictive safety filter (where the SAC action enters as a soft preference in the MPC objective).
Implementing a connectivity-aware extension where $\lambda$ is dynamically scheduled based on V2X signals (e.g., road friction, weather) to shift authority back to the model-based controller when the vehicle operates outside the training distribution.

The work is presented as an independent study, distinct from any employer's products, and serves as a foundation for future work involving higher-fidelity nonlinear models and real-vehicle validation.

More like this