💻 computer science

Guiding Neuro-Symbolic Scenario Generation with Spatio-Temporal Logic

This paper introduces STRELGen, a scalable neuro-symbolic framework that combines a multi-agent trajectory-generation diffusion model with differentiable Spatio-Temporal Logic specifications to efficiently generate safety-critical, realistic driving scenarios for robust autonomous vehicle testing.

Original authors: Lorenzo Bonin, Francesco Giacomarra, Luca Bortolussi, Jyotirmoy V. Deshmukh, Francesca Cairoli

Published 2026-05-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Lorenzo Bonin, Francesco Giacomarra, Luca Bortolussi, Jyotirmoy V. Deshmukh, Francesca Cairoli

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a self-driving car how to handle every possible situation on the road. The old way of doing this is like trying to learn how to swim by jumping into the ocean and hoping you don't drown. You wait for millions of real-world accidents to happen naturally so you can study them. But the problem is, the really dangerous, scary accidents (the "edge cases") are so rare that you might wait a lifetime to see one. It's too slow, too expensive, and statistically useless.

The authors of this paper, STRELGen, propose a smarter way: instead of waiting for accidents to happen, they build a "digital simulator" that can invent them on demand.

Here is how they do it, broken down into simple concepts:

1. The "Dreaming" Machine (Diffusion Models)

Think of a Diffusion Model as a very talented artist who has studied millions of photos of traffic. If you ask this artist to draw a traffic scene, they can create a picture that looks incredibly realistic. However, if you just say, "Draw a traffic scene," they will draw a boring, normal one. They won't draw a car swerving dangerously or a pedestrian stepping out unexpectedly unless you specifically tell them to.

The challenge is: How do you tell the artist to draw something specific and dangerous without breaking the laws of physics or making the picture look like a cartoon?

2. The "Rulebook" (Spatio-Temporal Logic)

To solve this, the authors created a special rulebook called STREL (Spatio-Temporal Reach and Escape Logic).

Standard Logic (STL): Imagine a rulebook that says, "The car must be close to the pedestrian." This is okay for one car, but if you have 50 cars, 10 pedestrians, and 5 bikes, the rulebook becomes a tangled mess of instructions that is impossible to read or follow.
The New Rulebook (Colored STREL): The authors invented an upgrade called Colored STREL. Imagine the traffic scene is a giant board game. Instead of treating every piece the same, they paint the pieces different colors: Red for cars, Blue for bikes, Green for pedestrians.
- Now, the rulebook can say things like: "Find a Red piece that is moving fast and is about to get too close to a Blue piece."
- Because the rules are based on colors, the computer can easily understand complex interactions between different types of road users without getting confused.

3. The "Guidance System" (Gradient Search)

Here is the magic trick. Usually, once the artist (the AI) finishes drawing, you can't change it without starting over. But because the authors made their rulebook mathematically smooth (differentiable), they can use it as a GPS.

They start with a random "dream" (a latent input) that the AI turns into a traffic scene.
They check the scene against their Colored STREL rulebook.
If the scene isn't dangerous enough (e.g., the fast car is still too far from the bike), the system calculates exactly how to tweak the original "dream" to make the car closer.
It repeats this thousands of times, nudging the AI to generate a scene that is perfectly realistic (it looks like real traffic) but specifically designed to be a safety test (it forces a near-miss).

4. The Safety Net (Regularization)

There is a risk: if you push the AI too hard to create a crash, it might start drawing cars flying in the sky or driving on the sidewalk. That's not a useful test.
To prevent this, the authors added a "reality check" (regularization). It's like a bouncer at a club. The system says, "You can create a dangerous scenario, but only if it still looks like something that could happen in the real world." This ensures the generated accidents are plausible, not hallucinations.

The Result

The paper shows that this method works. They used it to generate three specific types of scary scenarios:

Fast vehicles getting dangerously close to bikes or pedestrians.
Fast cars tailgating slow cars (rear-end risk).
Aggressive drivers getting surrounded by slow traffic.

In their tests, the unguided AI rarely created these dangerous situations. But with their STRELGen guidance, they could reliably generate these specific "safety-critical" scenes that are underrepresented in real-world data.

In short: They built a tool that lets engineers ask a self-driving car simulator, "Show me a realistic scenario where a fast car almost hits a cyclist," and the simulator instantly generates that exact scenario for testing, without needing to wait for it to happen in real life.

Technical Summary: Guiding Neuro-Symbolic Scenario Generation with Spatio-Temporal Logic

Problem Statement
The rapid advancement of autonomous driving (AD) technologies has outpaced the development of robust safety evaluation methods. Conventional testing relies on brute-force exposure to vast numbers of real-world traffic scenes, a strategy that is prohibitively expensive and statistically ineffective at capturing rare, safety-critical edge cases. While deep generative models, particularly Diffusion Models (DMs), offer a promising alternative for synthesizing realistic multi-agent trajectories, they lack mechanisms to explicitly steer generation toward specific, safety-critical behaviors without retraining. Existing guidance approaches often struggle with the spatio-temporal and multi-agent complexity of traffic domains. Furthermore, formal methods like Signal Temporal Logic (STL) face scalability and interpretability challenges in multi-agent systems due to the combinatorial explosion required to specify interactions between heterogeneous agents. A critical gap remains in defining quantitative robustness for properties involving categorical attributes (e.g., vehicle types), which limits the use of formal logic as a differentiable objective for optimization.

Methodology
The authors propose STRELGen, a scalable framework that synergistically combines a multi-agent trajectory-generation diffusion model with Colored Spatio-Temporal Reach and Escape Logic (CSTREL). The methodology operates in three main stages:

Trajectory Generation Baseline: The system utilizes a Latent Diffusion Model (LDM) based on the architecture from [23]. This model generates future trajectories for all agents jointly, conditioned on scene context (maps, history, interactions) encoded via a pre-trained QCNet. The authors employ the implicit formulation (DDIM) to ensure the reverse process is deterministic, facilitating stable gradient backpropagation.
Colored STREL (CSTREL): To address the limitations of standard Spatio-Temporal Logic (STREL) regarding categorical predicates, the authors introduce CSTREL. This logic partitions the spatial graph into color-specific subgraphs based on agent attributes (e.g., car, pedestrian, bike).
- Differentiable Semantics: CSTREL extends STREL by defining a coloring function $p: L \times \mathbb{R}^+ \to C$ . It introduces "Colored Atoms" and "Colored Reach/Escape" operators that restrict path searches to nodes matching specific color sets. Crucially, this extension allows for the derivation of continuous robustness values even from discrete categorical attributes, making the logic fully differentiable.
- Implementation: A PyTorch library was developed to compute these robustness values, enabling gradient-based optimization.
Latent Space Guidance: At inference time, the framework optimizes the latent input $z$ $z$ of the pre-trained diffusion model to maximize the satisfaction of a CSTREL formula $\phi$ $ϕ$ .
- Objective Function: The optimization maximizes a robustness metric $\rho_\phi(G_\theta(z))$ . For safety-critical scenarios, the maximum robustness across agents is targeted; for realism, the minimum is used.
- Regularization: To prevent the optimization from drifting into low-probability regions of the latent space (off-manifold samples), a regularization term proportional to the log-likelihood of the latent input (assuming a standard Gaussian prior) is added. The final objective is $J(z) = \rho_\phi(G_\theta(z)) - \lambda \frac{1}{2}\|z\|_2^2$ .

Key Contributions

Colored STREL: A novel extension of STREL that partitions spatial graphs by node attributes. This enables the monitoring of complex spatial properties across specific classes of objects while maintaining continuous robustness semantics, solving the issue of undefined quantitative semantics for categorical data.
Differentiable Guidance Framework: A method that uses CSTREL specifications as objective functions to directly optimize the latent space of generative models. This allows for the production of trajectories satisfying spatio-temporal requirements without model retraining.
Safety-Critical Data Augmentation: A strategy to generate diverse, realistic, and safety-critical scenarios that are underrepresented in real-world data, ensuring consistency with traffic rules and preventing unrealistic outputs through likelihood regularization.

Results
The approach was evaluated on the Argoverse 2 Motion Forecasting Dataset using three representative safety-critical scenarios: a fast vehicle reaching a vulnerable road user, a fast vehicle finding a slow vehicle ahead, and a fast car surrounded by slow cars.

Steering Effectiveness: STRELGen successfully steered the diffusion model to produce trajectories that satisfied the target CSTREL specifications. In unguided sampling, only ~13.6% of scenarios were challenging (positive robustness), whereas the guided approach achieved 100% satisfaction.
Safety Metrics: The guided generation consistently produced lower median minimum inter-agent distances and a higher number of potential collisions compared to the vanilla baseline, confirming the generation of safety-critical behaviors.
Plausibility: Qualitative analysis and the use of likelihood regularization ensured that the generated trajectories remained physically plausible and aligned with road geometry, avoiding off-manifold hallucinations. The method successfully balanced specification-driven optimization with the preservation of physical constraints.

Significance
The paper claims that STRELGen provides a flexible, interpretable, and powerful tool for stress-testing autonomous driving systems. By bridging the gap between data-driven generative models and formal logic, the framework moves beyond the limitations of brute-force data collection. It enables the systematic assessment of generative model coverage regarding rare events and allows for the targeted generation of safety-critical scenarios that lie within the learned data distribution. The authors posit that this approach facilitates the creation of rigorous testing environments essential for validating real-world robustness, offering a path toward more reliable autonomous vehicle deployment.