🤖 machine learning

Reinforcement Learning Enhancement Using Vector Semantic Representation and Symbolic Reasoning for Human-Centered Autonomous Emergency Braking

This paper proposes a neuro-symbolic reinforcement learning framework for autonomous emergency braking that integrates vector semantic representations of scene context with a Soft First-Order Logic reward function to enhance policy robustness and align decision-making with human safety values.

Original authors: Vinal Asodia, Iman Sharifi, Saber Fallah

Published 2026-02-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Vinal Asodia, Iman Sharifi, Saber Fallah

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine teaching a self-driving car to brake for pedestrians. The old way of doing this is like teaching a child to drive by showing them a million raw photos and saying, "If you see something red, stop; if you see something blue, go." The problem is that the car sees millions of pixels, not "pedestrians" or "crosswalks." It struggles to understand the story of the scene, and it relies on a rigid rulebook (a reward function) that doesn't know how to balance safety with getting to the destination on time.

This paper proposes a new way to teach the car, using a mix of super-organized memory and common-sense logic. Here is how it works, broken down into simple parts:

1. The "Super-Organized Memory" (Vector Semantic Representation)

Instead of just looking at a blurry photo, the car's brain creates a special "mental snapshot" of the scene.

The Old Way: It looks at the pixels.
The New Way: It breaks the scene down into a list of characters and their relationships, like a script for a play. It identifies: "There is a pedestrian (Character A) standing on a crosswalk (Location B)."
The Magic: It uses a technique called Hyperdimensional Computing. Think of this as a giant filing cabinet where every object (pedestrian, truck, road) has its own unique, high-tech ID card. The car doesn't just store the ID; it also stores where the object is and what it looks like, all bundled into one super-efficient "memory vector."
The Safety Filter: The system is smart enough to know that a pedestrian is more fragile than a truck. So, it gives the pedestrian's ID card a "loudness" setting (a weight) that is much higher than the truck's, ensuring the car pays extra attention to vulnerable people.

2. The "Common-Sense Judge" (Soft First-Order Logic Reward)

In Reinforcement Learning, the car learns by getting points (rewards) for good behavior and losing points for bad behavior.

The Old Way: The rulebook was rigid. "If you hit a pedestrian, -1000 points. If you drive fast, +10 points." This is too simple. What if you are driving fast but there is a hidden pedestrian behind a parked car? The old rules might not catch it.
The New Way: The paper introduces a Soft First-Order Logic (SFOL) judge. Instead of a rigid rulebook, this is like a wise judge who looks at the context.
- The judge asks questions based on the "mental snapshot": "Is there a crosswalk? Is it clear? Is a car blocking the view of the crosswalk?"
- Based on the answers, the judge decides how much the car should care about Safety vs. Efficiency.
- If the judge sees a pedestrian on a crosswalk, it says, "Safety is 100% important right now; efficiency doesn't matter."
- If the road is empty, it says, "Safety is still important, but let's also get to the destination quickly."
- The "Soft" part means the judge isn't binary (yes/no); it understands uncertainty. If the view is blocked, the judge says, "I'm not 100% sure, but I'm 80% sure there might be a pedestrian, so let's be cautious."

3. The Training Ground (The Experiment)

The researchers tested this in a virtual city (CARLA simulator) with a specific, tricky scenario: The Hidden Pedestrian.

The Setup: A car is driving down a road. A large van is parked, hiding a crosswalk. A pedestrian steps out from behind the van.
The Test: They tested the car in light, medium, and heavy traffic, with the pedestrian sometimes fully hidden and sometimes partially visible.

4. The Results: What Worked Best?

The paper found that the "Super-Organized Memory" and the "Common-Sense Judge" worked best when used together, but only if they were combined in a specific way:

Just the Memory (No Logic): The car was too aggressive and crashed often. It knew what was there but didn't know how to react.
Just the Logic (No Memory): The car was okay in light traffic but got confused and crashed in heavy traffic because it couldn't "see" the details well enough.
The Winner (Memory + Logic + Addition): When they combined the memory of the scene with the logical reasoning using a simple "addition" method, the car became the most robust. It successfully stopped for hidden pedestrians, avoided crashes in heavy traffic, and didn't drive too conservatively (stalling the traffic).

The Bottom Line

This paper shows that to make self-driving cars safer, we shouldn't just feed them more photos. We need to give them a way to understand the scene (who is where) and a flexible brain to decide when to be safe and when to be efficient. By combining a structured memory of the road with a logic-based judge, the car learns to make decisions that are safer and more aligned with human values, even when the view is blocked.

Technical Summary: Reinforcement Learning Enhancement Using Vector Semantic Representation and Symbolic Reasoning for Human-Centered Autonomous Emergency Braking

Problem Statement

Existing camera-based Deep Reinforcement Learning (DRL) approaches for Autonomous Driving (AD) face two primary limitations:

Feature Representation: They rarely integrate high-level scene context into feature representations, relying instead on raw images or attention mechanisms that highlight regions without explicit semantic understanding of object relations (e.g., a pedestrian's proximity to a crosswalk).
Reward Function Design: They typically rely on rigid, fixed reward functions (weighted sums of penalties and incentives). These fail to capture nuanced trade-offs between competing human values (safety, efficiency, comfort) in complex, dynamic scenarios. Adaptive reward functions exist but often use "hard switching" between objectives, leading to brittle behavior.

Methodology

The paper proposes a neuro-symbolic pipeline that combines Vector Semantic Representation (VSR) for state encoding and Soft First-Order Logic (SFOL) for reward shaping.

1. Vector Semantic Representation (VSR)

The authors extend the VSR framework to encode both static and dynamic entities using Hyperdimensional Computing (HDC), specifically Holographic Reduced Representations (HRR).

Input Processing: Camera images are processed through a ResNet18-based UNet to generate semantic segmentation maps and a ResNet18 autoencoder to produce latent feature embeddings.
Entity Encoding: Each dynamic entity ( $E_i$ $E_{i}$ ) is represented as a hypervector combining:
- Spatial Semantic Information (SSI): Encodes the semantic class (e.g., pedestrian, vehicle), spatial relationships with neighbors (e.g., "pedestrian below crosswalk"), and shape/location within a grid. This is constructed by binding semantic class, directional, and shape hypervectors.
- Spatial Features (SF): Spatially attended feature embeddings derived from the latent features, weighted by the semantic map.
Fusion: The final VSR vector is a weighted sum of all entity hypervectors. Crucially, weighting schemes prioritize vulnerable road users (e.g., pedestrians on the road/crosswalk are weighted higher than those on sidewalks or vehicles).
Integration: The SSI hypervector and SF embeddings are fused via element-wise addition ( $\oplus$ ) or multiplication ( $\otimes$ ) to create a compact, context-rich descriptor fed into a Proximal Policy Optimization (PPO) agent.

2. Soft First-Order Logic (SFOL) Reward Function

To address the rigidity of fixed rewards, the authors integrate a symbolic reasoning module using PyReason.

Predicate Extraction: Semantic and spatial predicates (e.g., Pedestrian(X), OnCrosswalk(X), Occludes(X,Y)) are extracted from the segmentation maps and spatial analysis. Uncertainty is handled by assigning soft truth values in $[0, 1]$ .
Rule Base: A set of Horn clauses defines rules for Safety (e.g., prioritizing safety if a pedestrian is on the road or a crosswalk is occluded) and Efficiency (e.g., prioritizing speed if the lane is clear).
Dynamic Weighting: The system infers confidence levels for these rules. These confidences are used to dynamically calculate weights ( $\bar{w}_{saf}$ , $\bar{w}_{eff}$ ) for the base reward components (safety, efficiency, and smoothness). This allows the reward function to balance human values based on the specific context of the driving scene rather than using hard switches.

Key Contributions

Extended VSR: Adapting VSR to encode dynamic entities with weighting schemes that emphasize vulnerable road users, fused with spatially attended features.
Neuro-Symbolic Fusion: Creating a compact scene descriptor by fusing symbolic hypervectors (SSI) with deep learning feature embeddings (SF).
SFOL Reward Function: Proposing a reward function that uses symbolic reasoning to dynamically balance safety, efficiency, and comfort, moving beyond fixed or hard-switching reward formulations.

Experimental Results

The pipeline was evaluated in the CARLA simulator using an occluded pedestrian crossing scenario with varying traffic densities (Low, Medium, High) and occlusion levels (Partial, Full).

VSR Integration:
- SSI Only: Performed poorly, indicating symbolic information alone is insufficient for policy learning.
- SF Only: Converged quickly but degraded significantly in high-density traffic, leading to higher collision rates.
- SSI $\oplus$ SF (Element-wise Addition): Achieved the most robust performance, balancing high success rates, low collision rates, and stable stopping distances across all densities. It outperformed the multiplication variant ( $\otimes$ ) in stability.
SFOL Reward Ablation:
- Complete Rule Set (CE & CS): Delivered the strongest overall performance with the least degradation in high-density scenarios.
- Partial Rule Sets: Removing subtle safety rules (CE & PS) led to aggressive driving and delayed reactions. Removing subtle efficiency rules (PE & CS) resulted in overly conservative behavior and high timeout rates.
- Baseline Comparison: The proposed SFOL approach outperformed a standard risk-factor adaptive baseline, particularly in high-density, occluded scenarios where the baseline showed brittleness.

Significance and Claims

The paper claims that integrating holistic neuro-symbolic representations and soft reasoning into Reinforcement Learning enables more context-aware and value-aligned decision-making for autonomous driving.

Robustness: The combination of symbolic context (VSR) and spatial features allows the agent to handle complex, occluded scenarios better than methods relying solely on visual features or symbolic data.
Human Alignment: The SFOL reward function successfully balances competing human values (safety vs. efficiency) dynamically, avoiding the brittleness of hard-switching logic.
Limitations: The authors modestly note that no configuration achieved a 100% success rate. They identify potential instability in the reasoning process (fluctuations in reward weights) during heavy congestion as an area for future refinement, suggesting the need for temporal symbolic information or abductive reasoning to handle hidden hazards.