⚡ electrical engineering

An Actor-Critic-Identifier Control Design for Increasing Energy Efficiency of Automated Electric Vehicles

This paper proposes an actor-critic-identifier reinforcement learning framework that uses a neural-network identifier to learn the complex mapping between control inputs and power consumption, enabling optimized control commands that improve electric vehicle energy efficiency and energy recovery.

Original authors: Hamed Faghihian, Arman Sargolzaei

Published 2026-02-11

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Hamed Faghihian, Arman Sargolzaei

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a student how to drive an electric car (EV) to get the absolute most mileage out of a single charge.

Most people would give the student a manual that says, "If you press the pedal this much, you will use this much energy." But in the real world, that manual is often wrong. Wind, hills, tire pressure, and even how hot the battery is can change how much energy you use. If the manual is wrong, the driver becomes inefficient.

This paper proposes a "Super Driver" system that doesn't need a manual at all. Instead, it uses a three-part brain called the Actor-Critic-Identifier (ACI).

Here is how it works, using a metaphor of a Professional Chef learning to cook a complex new dish:

1. The Identifier (The "Taste Tester")

Before the chef can cook perfectly, they need to understand the ingredients. The Identifier is like a taste tester who constantly samples the sauce. It doesn't know the recipe, but it observes: "When I add a pinch of salt, the flavor changes by this much."

In the car: This part of the brain constantly watches how the car actually reacts to the motor. It learns the "hidden" physics of the car (like how much energy is lost to friction or wind) in real-time, without needing a pre-written math formula.

2. The Critic (The "Food Critic")

The Critic is like a harsh judge sitting in the corner. It doesn't cook; it just watches the chef and calculates a "score." It looks at the final dish and says, "That was too salty, and you wasted too much expensive saffron."

In the car: The Critic looks at the driving performance. It calculates a "cost" based on two things: Are we staying at the speed limit? And are we wasting battery? It provides the mathematical "score" that tells the chef if they are doing a good job.

3. The Actor (The "Chef")

The Actor is the one actually doing the work—pressing the pedals. The Actor wants to cook a perfect meal, but it's constantly being guided by the Critic's scores and the Identifier's observations.

In the car: The Actor decides exactly how much torque to send to the wheels. It tries to find the "sweet spot"—accelerating just enough to keep up with traffic, but braking in a way that "recycles" energy back into the battery (regenerative braking).

Why is this a big deal? (The Results)

Usually, to make a car efficient, engineers have to build incredibly complex mathematical models of every single part of the car. This is expensive, difficult, and often inaccurate because every car is slightly different.

The "ACI" approach changes the game because:

It’s Self-Learning: It doesn't need to know the car's "manual." It learns the car's personality while it's driving.
It’s Highly Efficient: In the researchers' simulations, this "Super Driver" was much better at managing energy than a standard controller (the "PID" controller, which is like a driver who just follows a set of rigid, unthinking rules).
The "Win": The researchers found that this method could reduce energy consumption by a massive amount (the paper mentions a 42% reduction in net traction energy compared to the baseline).

In short: Instead of giving a car a map and a set of rules, this paper gives the car a brain that learns how to drive perfectly by trial, error, and constant self-correction.

Technical Summary: Actor-Critic-Identifier Control Design for Increasing Energy Efficiency of Automated Electric Vehicles

1. Problem Statement

The primary challenge addressed in this paper is the limited range of Electric Vehicles (EVs), which remains a significant barrier to market penetration. While increasing battery capacity is a primitive solution, it introduces environmental and cost concerns. Therefore, improving energy efficiency through advanced control is essential.

Existing control strategies for Electric Automated Vehicles (EAVs) often struggle because:

Model Complexity: Linking control inputs (motor torque) to power consumption/regeneration is difficult due to time-varying factors like State of Charge (SOC), temperature, and efficiency maps.
Model Dependency: Classical optimal control methods (like MPC or Dynamic Programming) require highly accurate, explicit models of the powertrain and vehicle dynamics, which are often unavailable or computationally expensive to run in real-time.
Control Gaps: Most existing research focuses on regenerative braking or eco-driving in isolation, failing to simultaneously manage speed tracking, traffic smoothness, and energy maximization within a unified framework.

2. Methodology: The Actor-Critic-Identifier (ACI) Architecture

The authors propose a novel Actor-Critic-Identifier (ACI) reinforcement learning framework. Unlike traditional methods that separate system identification from control, the ACI architecture couples them to learn the vehicle's unknown dynamics and optimal control policy simultaneously.

The architecture utilizes three coordinated Neural Networks (NNs):

Identifier Network: Learns the unknown drift term $g(x)$ (the system dynamics) online. It uses a RISE (Robust Integral of the Sign of the Error) feedback term to ensure accurate estimation of the vehicle's longitudinal behavior.
Critic Network: Approximates the optimal value function $J^*(x)$ , which represents the minimum cost-to-go (energy consumption and tracking error).
Actor Network: Generates the optimal control policy $\hat{u}(x)$ (motor torque) by minimizing the Hamilton-Jacobi-Bellman (HJB) residual.

Mathematical Foundation:
The controller is designed by minimizing the HJB residual ( $\delta HJB$ ), which acts as a bridge between the three networks. The stability of the entire closed-loop system is rigorously proven using Lyapunov stability analysis and Filippov’s framework for discontinuous systems, ensuring that the tracking errors and weight estimation errors are bounded and converge toward zero.

3. Key Contributions

Model-Free Adaptability: The ACI controller eliminates the need for explicit, pre-defined power-consumption models, making it highly adaptable to different EV models and varying environmental conditions.
Integrated Learning: It is one of the few frameworks that simultaneously performs system identification and optimal policy computation in a continuous control space.
Stability Guarantees: The authors provide a formal mathematical proof (via Lyapunov/Filippov analysis) that the update laws for the actor, critic, and identifier ensure the system remains stable and achieves asymptotic tracking.
Multi-Objective Optimization: The cost function $\ell(x, u)$ is specifically designed to penalize both speed deviations and high power draw during acceleration, directly targeting energy efficiency.

4. Results and Validation

The proposed method was validated through simulations using a realistic, "black-box" integrated EV powertrain-dynamics model. The ACI controller was compared against a tuned Proportional-Integral-Derivative (PID) baseline controller.

Speed Tracking: The ACI controller demonstrated superior tracking performance, maintaining the desired speed profile more accurately than the PID controller (as seen in the speed error trajectories).
Energy Efficiency: The ACI controller significantly reduced net traction energy consumption. While the PID baseline consumed 13.835 kJ, the ACI controller consumed only 7.99 kJ, representing a 42% reduction in energy consumption over the drive cycle.
Regenerative Braking: The controller effectively managed the transition between power draw and energy recovery (regenerative braking), optimizing the net power flow to the battery.

5. Significance

This research demonstrates that machine learning-based control can outperform classical methods in complex, non-linear systems like EVs. By removing the requirement for high-fidelity mathematical models of the powertrain, the ACI framework offers a scalable path toward more efficient automated driving. This has profound implications for extending the range of electric vehicles and improving the overall sustainability of automated transportation systems.