Foundation Models and Agentic AI for Autonomous Driving: A Systematic Review
This systematic review synthesizes over 100 works to present a unified nine-theme taxonomy that integrates foundation models, agentic AI, and cooperative V2X systems, distinguishing itself from prior surveys by jointly addressing the reasoning–efficiency trade-off, simulation-to-reality gaps, and safety governance within a comprehensive cognitive-architecture perspective for autonomous driving.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The road to a truly self-driving car has long been paved with a specific kind of engineering: a rigid assembly line where one computer part sees the world, another part decides what to do, and a third part steers the wheel. For years, this modular approach worked well enough to keep cars moving, but it struggled when faced with the messy, unpredictable nature of real life. A human driver does not just see a red light; they understand the context of a construction zone, the intent of a jaywalker, and the urgency of an approaching ambulance, all while explaining their actions to a passenger. To bridge the gap between rigid code and human-like intuition, researchers have begun turning to a new class of artificial intelligence known as foundation models. These are massive systems trained on vast amounts of text and images, capable of understanding language and vision together. When applied to driving, they promise to transform vehicles from simple automated machines into cognitive partners that can reason about complex traffic situations, explain their choices in plain English, and adapt to scenarios they have never seen before.
A comprehensive review published in September 2026 by a team of researchers from institutions across India, Malaysia, the United Arab Emirates, and Saudi Arabia maps out this rapidly shifting landscape. The authors, led by A. Srinivaas and Jeswin K F, did not simply list new gadgets; they conducted a systematic analysis of over one hundred recent studies to understand how these powerful language and vision models are being woven into the fabric of autonomous driving. Their work reveals that the field is moving away from isolated perception systems toward a unified "cognitive architecture." In this new vision, the car is not just reacting to sensors but is actively reasoning about the world, using tools to gather information, and collaborating with other vehicles and city infrastructure to navigate safely.
The review highlights a significant evolution in how these systems are built. Early attempts relied on separate modules for seeing and thinking, but the latest research focuses on Vision-Language-Action models. These systems combine the ability to see a scene, understand the language describing it, and directly control the vehicle's movements in a single, cohesive process. For instance, instead of just detecting a pedestrian, a system using these models can understand a sign that says "School Zone" and reason that it should slow down, even if the pedestrian is not yet visible. The researchers found that frameworks like DriveMLM and TeLL-Drive are already demonstrating the ability to improve decision-making and reduce the time it takes for the car to think. In one specific test using a simulation environment called CARLA, a model named DriveMLM improved driving scores by 4.7 points specifically on the CARLA Town05 Long driving score by better aligning its visual understanding with its planning decisions.
However, the path to a fully autonomous future is not without significant hurdles, and the review is careful to distinguish between what works in a computer simulation and what works on a real street. The authors point out a persistent "simulation-to-reality gap." While many of these advanced models perform impressively in virtual worlds like CARLA or nuScenes, they often struggle when transferred to real-world conditions. The gap is substantial; in some cases, the performance of cooperative perception systems drops by as much as 40 percent when moving from a simulated environment to the real world. This is because the perfect, predictable data of a simulation cannot fully capture the chaotic unpredictability of actual traffic, weather, and human behavior. Furthermore, the very features that make these models smart—their ability to reason through complex chains of thought—also make them slow. The time it takes for a large language model to "think" can be too long for the split-second decisions required in an emergency, creating a difficult trade-off between intelligence and speed.
Another critical finding of the review is the lack of a unified safety framework. Currently, safety checks, communication between vehicles, and the artificial intelligence itself are often treated as separate problems. The researchers argue that for these systems to be trusted, they need a holistic approach that integrates safety verification directly into the reasoning process. They note that while some progress has been made in using digital twins—highly detailed virtual replicas of cities—to test these systems, there is no standardized way to measure how closely these digital worlds match reality. Without this standard, it is difficult to certify that a car is truly safe for public roads. The review also touches on the emerging concept of "agentic AI," where the car acts as an independent agent with memory and the ability to use tools. While this offers the promise of vehicles that can learn from experience and collaborate with one another, the authors caution that these systems are still in their infancy and face challenges in interoperability and reliability.
The paper concludes by outlining a clear roadmap for the future, emphasizing that the next generation of autonomous driving will not be defined by a single breakthrough but by the integration of several complex technologies. The researchers suggest that the future lies in hybrid systems that combine the speed of traditional control systems with the reasoning power of large language models, all supported by robust communication networks that allow cars to talk to each other and to city infrastructure. They stress that for these systems to become a reality, the industry must solve the problem of making these powerful models fast enough for real-time use, safe enough to be certified, and transparent enough for humans to trust. The journey from a car that simply follows a lane to one that understands the world and reasons about it is well underway, but the final mile requires solving some of the most difficult challenges in artificial intelligence and engineering.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.