💻 computer science

TRACE: Topology-aware Reconstruction of Accidents in CARLA for AV Evaluation

TRACE is an automated pipeline that reconstructs real-world NHTSA crash reports into high-fidelity CARLA simulations by integrating OpenStreetMap data, Large Language Models, and trajectory generation to create a diverse benchmark for evaluating autonomous vehicle safety.

Original authors: Nahian Salsabil, Sebastian Elbaum

Published 2026-04-27

📖 3 min read☕ Coffee break read

CC0 1.0

Original authors: Nahian Salsabil, Sebastian Elbaum

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a driving instructor for a student learning to drive a self-driving car. To make sure that car is truly safe, you can’t just have it practice driving on sunny, empty suburban streets. If you only practice the "easy stuff," the student will be totally unprepared when they encounter a chaotic, real-world accident.

The problem is that real accidents are rare. You can't just wait for a real car crash to happen to test your software. Scientists usually solve this by making up "fake" accidents in a computer, but these are often too simple—like practicing driving in a video game where the roads are perfectly straight and the cars always behave predictably.

This paper introduces TRACE, a way to turn real-world police reports into high-fidelity "digital rehearsals" for self-driving cars.

Here is how it works, using a few analogies:

1. The "Digital Detective" (Data Extraction)

Think of a police accident report like a messy, handwritten note from a witness. It says things like, "A blue sedan hit a truck at the corner of Main and 5th, traveling north at 40mph." It’s not a perfect blueprint, but it has clues.

TRACE acts like a digital detective. It reads these messy reports and pulls out the vital clues: Where exactly did it happen? How fast were they going? Which way were they turning?

2. The "Google Maps Architect" (Map Reconstruction)

Most simulators use "toy" maps—generic roads that don't look like anything real. TRACE is different. It uses OpenStreetMap (think of it as the "Wikipedia of Maps") to find the exact street where the real accident happened.

If the real crash happened at a weirdly shaped, curvy intersection in Virginia, TRACE doesn't just build a generic intersection; it builds that specific curvy intersection, complete with the same lane widths and signs. It’s the difference between practicing a dance in a generic gym versus practicing it on the actual, slippery stage where the performance will happen.

3. The "Mind Reader" (Scenario Reconstruction)

This is the cleverest part. A police report tells you where the cars ended up after the crash, but it rarely tells you exactly where they were 10 seconds before the impact.

To solve this, the researchers used a Large Language Model (LLM)—the same kind of technology behind ChatGPT. They essentially ask the AI to "fill in the blanks." The AI looks at the map and the final crash details and says, "Based on the physics and the road layout, the car must have started right here to end up there." It’s like looking at a spilled glass of water on a table and being able to tell exactly how fast the person was walking when they bumped into it.

Why does this matter?

The researchers used this pipeline to create a "Stress Test Benchmark"—a collection of 52 different, highly realistic accident scenarios.

Instead of testing a self-driving car on a "sunny day in a simulator," they are throwing it into a "digital replay of a real-world disaster." By forcing the car to navigate the exact same tricky turns and sudden collisions that real humans have faced, we can find out if the car's "brain" is actually ready for the unpredictable chaos of the real world.

In short: TRACE turns the "lessons learned" from real-world tragedies into a high-tech training ground to prevent future ones.

Technical Summary: TRACE (Topology-aware Reconstruction of Accidents in CARLA)

Problem Statement

The safe deployment of Autonomous Vehicles (AVs) requires rigorous validation against "long-tail" events—rare, safety-critical accidents that are seldom encountered in routine driving data. Current evaluation benchmarks face a significant fidelity gap:

Synthetic/Abstract Scenarios: Many existing tools generate accidents using simplified road geometries or abstract maps that fail to capture the complex topological nuances (e.g., specific lane curvatures, signage, and intersection geometries) of real-world crashes.
Data Scarcity: While real-world crash reports (like those from NHTSA) exist, they are semi-structured, often incomplete, and difficult to translate directly into high-fidelity simulation environments like CARLA.
Limited Reproducibility: Existing frameworks often lack open-source availability or rely on domain-specific languages that lack geospatial accuracy.

Methodology

The authors propose TRACE, an automated pipeline that transforms raw NHTSA crash reports into high-fidelity CARLA simulations. The pipeline consists of three primary modules:

Crash Data Extraction:
- An Extractor module parses semi-structured XML reports from the NHTSA database.
- It isolates three critical data categories: Geospatial Coordinates (latitude/longitude), Crash Dynamics (type, environment, topology, and event sequence), and Vehicle States (velocity, impact points, and trajectory hints).
- A completeness constraint is applied to ensure the report contains enough data (location, topology, and trajectory) to be reconstructible.
Map Reconstruction:
- Retriever: Queries OpenStreetMap (OSM) to pull the exact road network of the accident site.
- Converter: Uses the OSMIUM library to prune the map and converts OSM data into the OpenDRIVE format required by CARLA.
- Topology Unification: To bypass CARLA’s inherent restrictions (which often prevent "illegal" maneuvers like driving against traffic or crossing solid lines), the pipeline transforms disjointed lane segments into unified, multi-lane road objects. This allows for the simulation of realistic, non-compliant behaviors like wrong-way driving.
Scenario Reconstruction:
- State Estimator (LLM-based): Since reports rarely provide exact starting coordinates, the pipeline uses a Large Language Model (LLM) to perform "backward trajectory" reasoning. The LLM infers the vehicles' initial positions and orientations by calculating backward from the reported collision point, speed, and angle, while ensuring the positions remain within valid road boundaries.
- Launcher: Instead of using standard route planners (which follow legal road graphs), the Launcher generates custom waypoints that allow for the specific, often "illegal," trajectories described in the report.
- Validation Loop: The pipeline validates the reconstruction by checking if the simulated crash occurs within 5 meters of the reported location, matches the reported impact points (within a $\pm$ 2 clock position margin), and follows the reported directional maneuvers.

Key Contributions

Automated Pipeline: A complete end-to-end workflow from raw government crash reports to executable CARLA simulation scenarios.
Topology-Awareness: Unlike previous methods, TRACE preserves the exact road geometry and signage of the actual accident site using OSM data.
LLM-Driven State Estimation: A novel use of LLMs to bridge the gap between qualitative accident descriptions and quantitative simulation parameters (position, velocity, orientation).
Open-Source Benchmark: A curated dataset of 52 diverse, verified accident scenarios.

Results

The authors processed an initial batch of 100 NHTSA reports, resulting in 52 verified scenarios after filtering for map compatibility and simulation success. The benchmark demonstrates high diversity across three dimensions (based on the FARS Manual):

Collision Types: Includes Angle (19), Front-to-Front (23), Front-to-Rear (3), and various Sideswipe scenarios.
Road Topologies: Covers non-intersections (36), T-intersections (8), and Four-way intersections (8).
Vehicle Trajectories: Includes same-direction (4), opposite-direction (27), turning maneuvers (9), and intersecting paths (7).

Significance

TRACE provides a much-needed, high-fidelity resource for the AV community. It enables three critical use cases:

Collision Avoidance Testing: Allowing researchers to test how well their control systems avoid real-world crash patterns.
Perception and Prediction Stress-Testing: Evaluating how visual occlusions or complex intersection geometries affect an AV's ability to anticipate erratic behavior.
Automated Test-Case Generation: Providing a scalable way to generate the "long-tail" training data necessary to make autonomous systems robust against rare but catastrophic failures.