LLM-Assisted Dynamic Threat Analysis for Attacker-Reachable Software Weaknesses in Autonomous Vehicles
This paper evaluates the feasibility of using large language models to automate the generation of dynamic exploit artifacts for Autoware, revealing that while reasoning models outperform code-specialized models in initial compilation, the primary barrier to confirming software weaknesses is not candidate generation or fuzzing but rather the high failure rate in build integration caused by dependency wiring and the reliance on stubbed code.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the software inside a self-driving car as a massive, bustling city. This city has millions of tiny workers (lines of code) who talk to each other to decide when to turn the steering wheel or hit the brakes. To keep this city safe, engineers act like detectives. First, they use "static analysis," which is like a super-fast map-reader that scans the entire city blueprint to find spots where a stranger could sneak in a bad message and cause chaos. But a map isn't the real city. Just because a path looks open on the blueprint doesn't mean you can actually walk down it; maybe there's a locked gate or a bridge that doesn't exist. To be sure, you need to send a real explorer into the city to try walking that path. This is called "dynamic analysis."
For years, the hope has been that Artificial Intelligence, specifically Large Language Models (LLMs)—the same kind of tech that writes stories or solves math problems—could act as these explorers. The idea was that instead of hiring a human to build a custom "test car" for every single suspicious spot on the map, we could just ask the AI to build it for us. If the AI could automatically build these test cars, drive them into the software, and see if they crash, we could check the safety of self-driving cars at lightning speed. This paper asks a simple, high-stakes question: Can these AI detectives actually build the test cars well enough to prove if a self-driving car is truly safe, or do they get stuck building fake cars that look real but don't work?
The Great AI Test-Drive Experiment
In this study, researchers set up a massive experiment using Autoware, a popular, open-source software stack that powers many self-driving cars. Think of Autoware as the operating system for a robot car, made up of 185 different packages (like different neighborhoods in our city) and thousands of files.
The Setup: The Map and the AI Builders
First, the researchers used their "map-reader" (static analysis) to find 740 specific spots in the Autoware code where a bad input from an attacker could potentially reach a safety-critical decision, like telling the car to stop or go. These were the "suspects."
Next, they handed these 740 suspects to two different AI models (one specialized in coding, the other a general reasoning model) and asked them to build a "test harness." In plain English, a test harness is a little program designed to poke the specific spot in the code to see if it breaks. The researchers gave the AI the code around the suspect, the description of the problem, and the rules of the road (the build environment).
The Journey: Where the AI Got Lost
The researchers then tried to compile (build) these AI-generated test programs against the real Autoware software. This is where the story takes a twist.
Out of 2,960 attempts to build these test programs (740 targets × 4 different AI conditions), the results were sobering:
- The "Build" Wall: Most of the AI's first attempts failed to compile. About 80% of the failures weren't because the AI wrote bad logic; they were because the AI didn't know how to wire the test program to the rest of the car's software. It was like the AI building a car engine but forgetting to attach the wheels or the fuel line.
- The "Stub" Trap: The researchers gave the AI a second chance. They showed it the error messages and asked it to fix the code (a process called "compiler-in-the-loop repair"). The AI got better at fixing the errors, eventually making 100% of the programs compile.
- However, there was a catch. To make the code compile, the AI often replaced the real, complex parts of the car's software with "stubs." A stub is like a cardboard cutout of a door. It looks like a door, and the test program can "open" it, but it's not a real door, and it doesn't lead anywhere. The AI was essentially building test cars that drove into cardboard cutouts instead of the real software.
The Result: No Crashes Found (Because No Real Driving Happened)
After all the fixing and compiling, the researchers tried to run the tests.
- Only 652 of the original 2,960 attempts actually linked up with the real Autoware software and reached the fuzzer (the part that tries to break the code).
- Zero of the original 740 suspects were confirmed as dangerous.
- The only 37 crashes that did happen? They all happened inside the AI's own "stub" code—the cardboard cutouts—not in the real Autoware software.
What This Means
The paper concludes that while AI is great at writing code snippets, it currently cannot automatically build the complex, integrated test environments needed to safely test a full self-driving car stack.
The main barrier isn't that the AI can't write the logic; it's that the AI can't figure out how to connect its test program to the massive, real-world software ecosystem without breaking it or faking the connection. The researchers found that the "build integration" (getting the test to actually talk to the real car software) is the bottleneck, not the generation of the test itself.
The Bottom Line:
The study suggests that we cannot yet rely on AI to autonomously confirm if self-driving software is safe. The AI tends to build "fake" tests that compile but don't actually test the real thing. Until we can teach AI to build test cars that drive into the real city and not just cardboard cutouts, human engineers will still need to do the heavy lifting of verifying these safety-critical paths. The static analysis (the map) is still useful for finding where to look, but the dynamic confirmation (the test drive) remains a job that AI, on its own, isn't ready to do yet.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.