🤖 AI

Reason--Imagine--Act: Closed-Loop LLM Decision Making with World Models for Autonomous Driving

The paper proposes Reason--Imagine--Act (RIA), a closed-loop framework that integrates a large language model reasoner with an action-conditioned world model to enable online safety verification and significantly improve decision-making performance in autonomous driving.

Original authors: Zhengqi Sun, Yiwen Sun, Boxuan Liu, Tailai Chen, Tianxu Guo, Jiabin Liu

Published 2026-05-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhengqi Sun, Yiwen Sun, Boxuan Liu, Tailai Chen, Tianxu Guo, Jiabin Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a very smart, well-read robot to drive a car. This robot has read every traffic law book, understands the nuances of human behavior, and can explain why it wants to make a turn. However, there's a problem: this robot is like a brilliant chess player who has never actually played on a physical board. It knows the rules of the game, but it doesn't quite understand how heavy the pieces are or how fast they slide.

If you ask this robot to "swerve quickly to avoid a pedestrian," it might agree logically, but it doesn't realize that its car is too heavy to stop in time, or that the turn is too sharp and the car will flip over. In the paper, the authors call this a "physical hallucination"—the robot's brain says "go," but the car's physics say "crash."

To fix this, the researchers created a system called Reason–Imagine–Act (RIA). Think of it as giving the robot a "virtual test drive" before it actually moves the car.

Here is how the three steps work, using a simple analogy:

1. Reason (The Brain)

First, the robot's "Brain" (a Large Language Model) looks at the road. It sees a red light and a car cutting in front. It thinks, "I need to slow down and maybe change lanes to the right." It comes up with a plan and a few specific ways to do it (e.g., "brake gently," "brake hard," or "swerve slightly").

2. Imagine (The Simulator)

Before the robot touches the brake pedal, it runs a World Model. Think of this as a high-speed, crystal-ball simulator.

The robot asks the simulator: "If I brake gently, what happens in the next 2 seconds?"
The simulator runs a quick "movie" of that future. It sees that a gentle brake isn't enough; the car will still hit the other vehicle.
Then it asks: "What if I brake hard?"
The simulator runs that movie too. It sees that braking hard stops the car safely without hitting anyone.

The simulator acts like a safety inspector who checks the robot's plans against the laws of physics. It doesn't care about the robot's "feelings" or "logic"; it only cares about numbers: distance, speed, and collision risk.

3. Act (The Driver)

Finally, the robot looks at the results from the "Imagine" step. It sees that the "gentle brake" plan failed the safety check, but the "hard brake" plan passed. So, it chooses the hard brake and executes the action.

Crucially, after the car actually moves, the robot tells the simulator, "Hey, I just braked hard. Here is what actually happened." This feedback loop helps the robot learn and adjust its next "Imagine" step, making it smarter over time.

The Results: A Safer Driver

The researchers tested this system in a very realistic video game world called CARLA (which simulates city driving with thousands of cars and pedestrians). They compared their new "Reason–Imagine–Act" robot against:

A robot that only uses its "Brain" (no simulator).
A standard, rule-based driving system (like a basic cruise control).
Another advanced AI system.

The findings were clear:

The "Brain-only" robot was smart but clumsy. It often crashed or got stuck because it didn't check if its ideas were physically possible.
The "Reason–Imagine–Act" robot was the winner. It completed 80% of its trips successfully (compared to 61% for the brain-only version) and crashed only 0.2% of the time (compared to 0.4% for the brain-only version).
It also drove much more smoothly, braking less violently and making fewer jerky movements.

The Bottom Line

The paper argues that for self-driving cars to be truly safe, they can't just be "smart talkers." They need a way to "imagine" the physical consequences of their actions before they do them. By pairing a smart language brain with a physics-based simulator, the system creates a "think-before-you-act" loop that prevents the car from making dangerous mistakes that look good on paper but fail in the real world.

Technical Summary: Reason–Imagine–Act (RIA) for Autonomous Driving

1. Problem Statement

The paper addresses a critical gap in applying Large Language Models (LLMs) to autonomous driving: the lack of physical grounding. While LLMs excel at semantic reasoning, interpreting traffic rules, and handling long-tail scenarios, they often function as a "brain in a vat." They can generate logically coherent high-level commands (e.g., "execute emergency avoidance") but fail to precisely capture vehicle dynamic constraints or anticipate the numerical consequences of these commands in continuous physical space.

This decoupling leads to physical hallucinations, where generated intentions are kinematically or dynamically infeasible, posing severe safety risks. Existing approaches either rely on online language reasoning without explicit dynamics verification or use World Models (WMs) primarily in offline pipelines. Consequently, there is a disconnect between "rule-aware" semantic logic and "dynamics-aware" physical execution.

2. Methodology: Reason–Imagine–Act (RIA)

The authors propose Reason–Imagine–Act (RIA), a closed-loop framework that tightly couples an LLM reasoner with an action-conditioned World Model (WM) for online safety verification. The system operates without parameter updates (no fine-tuning) during inference.

Core Loop Components

Reason (LLM):
- The LLM ingests the current structured state (ego features, road topology, social features) and a memory buffer of recent behaviors.
- It outputs a high-level discrete action template (e.g., "lane change left," "speed up," "stop") and a set of candidate sub-actions (parameterizations like specific speed increments or steering biases).
- The LLM is constrained to structured JSON outputs to ensure a compact reasoning space.
Imagine (World Model):
- An action-conditioned WM performs short-horizon rollouts (1-second prediction) for each candidate sub-action generated by the LLM.
- Architecture: The WM uses a latent dynamics formulation inspired by Recurrent State-Space Models (RSSM). It employs:
  - Social Attention: To prioritize interaction risks (e.g., high closing-speed cut-ins) over raw distance.
  - Temporal Recurrence: To track motion continuity (acceleration, yaw-rate evolution).
  - Residual Decoding: To predict state changes ( $\hat{s}_{t+1} = s_t + \text{residual}$ ) for numerical stability.
- The model predicts future trajectories and interactions, specifically focusing on safety boundaries (lane departure, front-gap collapse, stop-line risk).
Act (Safety Scorer & Execution):
- A safety scorer evaluates the rollouts using a cost function ( $C$ ) that penalizes critical risks (e.g., distance $<3.0$ m), proximity to surrounding agents, and lane violations.
- The system selects the sub-action with the minimum cost (highest safety score).
- Hard Constraints: If a rollout indicates imminent collision or if all candidates violate thresholds, a hard-rule fallback (e.g., emergency braking) overrides the selection.
- The selected action is executed via the CARLA Traffic Manager (TM) backend, which maps high-level intents to low-level control signals.
- Feedback: The execution summary and verification outcomes are fed back to the LLM's memory buffer, enabling iterative self-correction in subsequent steps.

World Model Training

The WM is trained on a dataset combining expert policies (CARLA TM) with controlled perturbations to cover nominal and near-failure regimes (e.g., hard braking, near-collisions). The training objective focuses on short-horizon prediction fidelity and boundary-transition coverage rather than long-horizon video synthesis, ensuring the model is optimized for ranking candidate safety rather than pixel-level realism.

3. Key Contributions

The paper claims three primary contributions:

First Closed-Loop Coupling: To the authors' knowledge, this is the first work to tightly couple an LLM reasoner with an action-conditioned world model specifically for online "think-before-acting" verification in a closed-loop driving pipeline.
RIA Mechanism: The proposal of a decision mechanism that evaluates template-conditioned candidate sub-actions via short-horizon rollouts and feeds physical verification signals back to the LLM for iterative correction.
Performance Gains: Demonstrating that under a unified closed-loop interface, the method consistently outperforms training-free baselines. Specifically, compared to an LLM without WM verification, the full RIA method improves Route Completion (RC) from 61.21% to 80.05%, Arrival Rate (AR) from 30.20% to 51.10%, and reduces Collision Rate (ColR) from 0.40% to 0.20%.

4. Experimental Results

Experiments were conducted in the CARLA 0.9.15 simulator under a randomized point-goal navigation protocol across 1,000 episodes in complex urban environments.

Comparative Performance (vs. Baselines):

vs. LLM without WM: RIA significantly improves safety and completion metrics.
vs. CARLA Traffic Manager (TM): RIA achieves a much higher RC (80.05% vs. 46.47%) and AR (51.10% vs. 13.40%) with a drastically lower collision rate (0.20% vs. 16.00%).
vs. MADA (LLM-driven baseline): RIA outperforms MADA on all core metrics, including RC (80.05% vs. 21.88%) and ColR (0.20% vs. 53.00%).

Safety and Comfort Indicators:

RIA shows substantial reductions in hard-brake events (HB), high-jerk events (HJ), and mean absolute jerk (MAJ) compared to the LLM-only baseline.
While RIA shows a slight increase in off-road events compared to the native TM (0.19 vs. 0.00), it maintains a superior balance of route completion and safety outcomes overall.

5. Significance and Claims

The paper posits that the integration of semantic reasoning with physically grounded imagination is essential for safe autonomous driving. The significance of RIA lies in its ability to:

Bridge the Gap: Connect high-level semantic intent with low-level physical feasibility without requiring additional policy training or fine-tuning.
Enable Iterative Correction: Use physical feedback to refine LLM decisions in real-time, mitigating the risk of "physically hallucinated" commands.
Demonstrate Practical Value: Prove that a training-free, inference-time framework can achieve robust performance in dynamic, safety-critical environments, outperforming both traditional rule-based systems and other LLM-based approaches.

The authors emphasize that their goal is not long-horizon pixel realism but reliable candidate ordering within a short reaction window to ensure safety. The framework is designed to be reproducible, with code and evaluation scripts made available.