Empowering Microscopic Traffic Simulators with Realistic Perception using Surrogate Sensor Models

This paper introduces MIDAR, a computationally efficient surrogate LiDAR detection model that leverages a geometry-aware Graph Transformer to generate realistic perception data from microscopic traffic simulators, thereby enabling high-fidelity, large-scale evaluation of autonomous vehicle applications without the heavy computational costs of traditional game-engine-based simulators.

The Gist

Imagine you are trying to teach a self-driving car how to drive safely in a busy city. To do this, you need to run millions of simulations on a computer. But here's the catch: you have two very different tools for the job, and neither is perfect on its own.

The Two Tools: The "Movie Studio" vs. The "Traffic Spreadsheet"

1. The Movie Studio (Game-Engine Simulators like CARLA):
   Think of this like a high-end video game or a Hollywood movie set. It creates a stunningly realistic 3D world. It can simulate exactly how a laser scanner (LiDAR) on a car would "see" a truck, including shadows, reflections, and exactly which parts of the truck are hidden behind a bus.

   • The Problem: It's incredibly heavy. Running this "movie" for just one car is easy. But if you want to simulate a whole city with 500 cars at once, your computer crashes. It's like trying to render a blockbuster movie in real-time for every single car in traffic; it's too slow and expensive.

2. The Traffic Spreadsheet (Microscopic Traffic Simulators like SUMO):
   Think of this as a giant, super-fast spreadsheet. It doesn't care about 3D graphics or lasers. It just knows: "Car A is at this spot, moving at this speed, and is 15 feet long." It can simulate thousands of cars in a city instantly.

   • The Problem: It's too simple. It doesn't know what the car "sees." In this world, if a car is on the road, the spreadsheet assumes the self-driving car sees it perfectly, even if a giant truck is blocking the view. It's like playing a game where you can see through walls.

The Solution: MIDAR (The "Smart Translator")

The researchers created a new tool called MIDAR. Think of MIDAR as a super-smart translator that sits between the "Traffic Spreadsheet" and the "Movie Studio."

MIDAR's job is to take the simple data from the spreadsheet (where cars are, how big they are) and guess what a realistic laser scanner would have seen, without actually running the heavy 3D graphics.

How MIDAR Works (The Creative Analogy)

Imagine you are standing in a crowded room (the self-driving car) trying to spot your friend (a target vehicle) across the room.

1. The "Line of Sight" Chain:
   In the old, simple models, the spreadsheet just said, "Is your friend in the room? Yes? Then you see them."
   MIDAR is smarter. It asks: "Is there a giant person standing between you and your friend?" If yes, it checks: "Is that giant person standing in front of a wall?"
   MIDAR builds a chain of relationships. It traces the path from your eyes, through any blocking people, all the way to your friend. It understands that if a bus blocks the view, you can't see the car behind it.

2. The "Laser Ray" Guess (The Ray-Hit Feature):
   Real laser scanners shoot thousands of tiny invisible beams (like a spider web of light). If a beam hits a car, it's seen. If it hits a wall first, the car behind is hidden.
   Since MIDAR doesn't have the actual laser beams, it uses a clever trick called "Ray Casting." It mentally shoots thousands of invisible lines from the car's position. It counts how many lines actually hit the target car versus how many hit a blocker first.

   • Analogy: Imagine throwing a handful of spaghetti at a target. If a big box blocks the path, most noodles hit the box. MIDAR counts the noodles that hit the target to decide, "Okay, the target is partially visible," or "Nope, totally hidden."

3. The "Brain" (Graph Transformer):
   MIDAR uses a special type of AI brain (a Graph Transformer) that is really good at looking at these chains of people and spaghetti. It learns the patterns: "Oh, when a truck is 20 meters away and slightly to the left, it usually hides the car behind it."

Why Does This Matter? (The Real-World Impact)

The paper proves that using MIDAR changes the results of traffic simulations in two big ways:

1. It's Not Just "Fake" Data:
   If you use the simple "perfect vision" model, you might think a traffic light system works great because it "sees" everyone. But in reality, if the light system can't see a car hidden behind a bus, it might turn green too early, causing a crash. MIDAR shows these hidden dangers, making the simulation results much more realistic.

   • Result: When they tested traffic lights, the "perfect vision" model said traffic would flow smoothly. The "MIDAR" model (which accounts for blocked views) showed that traffic would actually get stuck because the system missed cars. This helps engineers build safer systems.

2. It's Super Fast:
   MIDAR is so lightweight that it can run on a standard computer while simulating thousands of cars. It's like swapping a supercomputer for a smartphone app. You get 90% of the realism of the "Movie Studio" but with the speed of the "Spreadsheet."

The Bottom Line

MIDAR is the bridge. It allows researchers to simulate massive, realistic traffic networks with hundreds of self-driving cars without needing a supercomputer. It teaches the computer to "guess" what is hidden behind other cars, just like a human driver would, making our future traffic systems safer and smarter.

In short: It turns a boring spreadsheet of car locations into a realistic simulation of what those cars can actually see, saving time, money, and potentially lives.

Technical Summary

1. Problem Statement

The evaluation of Intelligent Transportation Systems (ITS) and Autonomous Vehicle (AV) technologies relies heavily on simulation. However, a significant trade-off exists between scalability and perception fidelity:

• Game-Engine Simulators (e.g., CARLA): Generate high-fidelity 3D environments and raw sensor data (LiDAR point clouds), allowing for realistic perception modeling. However, they suffer from high computational overhead, making them infeasible for large-scale network simulations involving hundreds of AVs.
• Microscopic Traffic Simulators (e.g., SUMO): Efficiently simulate large traffic networks with thousands of vehicles but lack native perception modeling. They typically rely on simplified detection models (e.g., "perfect detection" or "random dropout") that fail to capture realistic occlusion and physical sensing constraints.

This gap leads to unrealistic simulation outcomes, where simplified perception models produce biased results for downstream ITS applications (e.g., signal control, trajectory reconstruction). The paper addresses the need for a lightweight, scalable surrogate sensor model that mimics realistic LiDAR detection using only high-level vehicle features available in microscopic simulators.

2. Methodology: The MIDAR Framework

The authors propose MIDAR (Microscopic Inference of Detection and Awareness for Realism), a graph-transformer-based surrogate LiDAR detection model.

A. Core Concept

MIDAR predicts whether a ground-truth vehicle is a True Positive (TP) or False Negative (FN) based on the relative positions and dimensions of surrounding vehicles, without generating raw point clouds. It does not model False Positives (FPs) as they are statistically rarer and outside the primary scope.

B. Key Components

1. Refined Multi-hop Line-of-Sight (RM-LoS) Graph:

   • Instead of a simple visibility check, MIDAR constructs a directed graph where edges represent potential occlusion chains.
   • For a target vehicle, the graph traces a chain of "blockers" (intermediate vehicles) back to the ego AV.
   • This structure explicitly encodes the sequence of occlusions, serving as the input sequence for the transformer.

2. Height-Aware Azimuthal Ray Casting (HARC):

   • To incorporate physical priors regarding LiDAR beam coverage, MIDAR introduces a ray-hit feature.
   • It discretizes the 360° field of view into azimuthal rays.
   • It employs a $k$-depth peeling strategy across multiple height slices (e.g., 1.8m for cars, 3.6m for trucks) to account for vehicle height differences and LiDAR beam tilt.
   • The feature quantifies how many rays reach a target vehicle without being blocked by closer vehicles, acting as a lightweight proxy for point-cloud density.

3. LoS-Graphormer Architecture:

   • Input: Node features include vehicle dimensions (width, length, height), position ($x, y, z$), heading ($yaw$), distance to ego, and the calculated ray-hit score.
   • Attention Mechanism: The model uses a Transformer encoder. Crucially, it generates an attention bias based on pairwise geometric relations (distance and relative heading) between vehicles in the LoS chain. This injects physical priors (e.g., closer vehicles have higher influence) into the attention mechanism.
   • Output: A binary classification (TP or FN) for the target vehicle.

3. Key Contributions

1. Bridging Scalability and Fidelity: MIDAR is one of the first works to close the gap between the scalability of microscopic simulators and the detection realism of high-fidelity simulators.
2. Lightweight Efficiency: The model requires orders of magnitude fewer GPU/CPU resources than game-engine simulators (e.g., CARLA) and can be seamlessly integrated into real-time traffic simulators.
3. Physical Priors: The introduction of the ray-hit feature and RM-LoS graph allows the model to learn complex occlusion patterns (including partial occlusions and height-dependent visibility) that rule-based models miss.
4. Generalizability: While designed for LiDAR, the framework is adaptable to other sensors (cameras, radar) and cyber-physical systems (e.g., robotics).

4. Experimental Results

The model was trained and evaluated on two datasets: nuScenes (real-world data) and a CARLA-SUMO co-simulation dataset. The target detection model mimicked was CenterPoint.

A. Detection Performance

• AUC Scores: MIDAR achieved an AUC of 0.94 on the CARLA dataset and 0.86 on the nuScenes dataset.
• Baseline Comparison: MIDAR significantly outperformed baselines including MLP, Graph Convolutional Networks (GCN), and Vanilla Transformers.
• Ablation Study: Removing the ray-hit feature caused a significant performance drop on the CARLA dataset, confirming the importance of physical visibility priors.

B. Application-Level Evaluation

The authors integrated MIDAR into two ITS applications to prove its practical value:

1. Cooperative-Perception Adaptive Signal Control (TSC):
   • Metric: Average vehicle delay.
   • Finding: Simplified models (Perfect Detection, Random Dropout) underestimated delays by assuming unrealistic detection patterns. MIDAR revealed that realistic occlusion leads to significantly higher delays (e.g., 76.79s vs. 58.12s for Perfect Detection), highlighting the risk of over-optimistic control strategies.
2. Vehicle Trajectory Reconstruction:
   • Metric: Reconstruction accuracy (MAE, RMSE) of vehicle positions and lane changes.
   • Finding: MIDAR's reconstruction metrics were statistically indistinguishable from those using real LiDAR detection (CenterPoint). In contrast, simplified models produced significantly different (and less accurate) trajectory reconstructions due to unstructured missing data patterns.

C. Computational Efficiency

• Resource Usage: MIDAR uses <0.5 GB GPU memory** and **~1 GB CPU memory**, whereas the conventional CARLA pipeline uses **>21 GB GPU memory and ~8 GB CPU memory.
• Latency: MIDAR achieves a per-AV detection time of ~6–7 ms, compared to ~18–19 ms for the detection module alone in the conventional pipeline (excluding rendering overhead).
• Scalability: MIDAR enables large-scale, real-time simulations that are impossible with high-fidelity sensor simulation.

5. Significance

This paper demonstrates that high-fidelity perception modeling does not require high-fidelity sensor simulation. By leveraging geometric relationships and physical priors via a surrogate model, researchers can simulate realistic AV perception in large-scale traffic networks. This is critical for:

• Validating ITS Applications: Ensuring that signal control and traffic management strategies are robust to realistic sensing limitations (occlusion).
• Cost Reduction: Drastically reducing the computational cost of AV fleet simulations.
• Future Research: Providing a foundation for simulating other cyber-physical systems where perception is imperfect but critical for coordination.

The code and data are publicly available, facilitating further research into scalable, perception-aware traffic simulation.