💻 computer science

Belief-Space Quantum-Inspired Reinforcement Learning for Partially Observable Autonomous Cyber Defense in the Internet of Vehicles

This paper proposes Q-BIRD, a quantum-inspired reinforcement learning framework that utilizes amplitude-based belief states to enhance defender decision-making under partial observability in Internet of Vehicles, significantly outperforming classical Bayesian methods in reducing attack success rates and cumulative damage against adaptive adversaries.

Original authors: Anwar Shah, Rohan Farooq, Sajid Anwer, Tallha Akram, Usman Ghous, Sajid Ullah Khan

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Anwar Shah, Rohan Farooq, Sajid Anwer, Tallha Akram, Usman Ghous, Sajid Ullah Khan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: A Game of "Hide and Seek" in a Smart City

Imagine the Internet of Vehicles (IoV) not just as cars on a road, but as a massive, high-speed game of "Hide and Seek" played in a smart city.

The Defenders: These are the security systems in the cars and traffic networks. Their job is to keep everyone safe.
The Attackers: These are hackers trying to crash cars, steal data, or jam traffic. But they aren't stupid; they are adaptive. If they see a security guard (the defender) looking one way, they sneak around the other. If they get caught, they change their disguise.

The Problem:
Most current security systems are like a security guard who only looks at a static photo of a known criminal. If the criminal puts on a new hat or walks differently, the guard doesn't recognize them. Furthermore, these guards often get overconfident. If they see a shadow that looks like a criminal, they might shout "Thief!" immediately, even if it's just a cloud. When the criminal is actually hiding, the guard might be too sure they are safe and miss the real danger. This leads to panic (false alarms) or getting caught off guard (real attacks).

The Solution: "Quantum-Inspired" Thinking (Q-BIRD)

The authors propose a new system called Q-BIRD (Quantum Belief-Integrated Reinforcement Defense). They don't use actual quantum computers (which are huge and expensive). Instead, they use math inspired by quantum physics to make the security guard smarter at guessing what the hacker is doing.

Here is how it works, broken down into simple concepts:

1. The "Superposition" of Suspicion

In the old way (Classical Bayesian), the security guard tries to pick one answer: "Is the person a criminal? Yes or No?"

If the evidence is shaky, the guard forces a guess. They might say, "It's 90% a criminal!" and act aggressively.
The Flaw: If the criminal is actually faking it, the guard made a bad guess based on shaky evidence.

In the Q-BIRD way (Quantum-Inspired), the guard keeps all possibilities open at once.

Think of it like a spinning coin. While it's spinning, it isn't "Heads" or "Tails." It is a mix of both.
The system keeps a "superposition" of thoughts: "Maybe they are probing, maybe they are attacking, maybe they are just driving normally."
It doesn't force a decision until it has enough solid evidence. This prevents the guard from panicking over a fake shadow.

2. The "Amplitude" vs. The "Probability"

Old Way: The guard updates a score (Probability). If the score hits 99%, they act. But if the hacker tricks the guard with a fake signal, the score crashes, and the guard becomes confused or overconfident.
New Way: The guard updates a "wave" (Amplitude). Imagine the uncertainty is a wave in a pond. When a hacker tries to trick the system, the waves can cancel each other out or boost each other up in complex ways.
The Magic: This wave math allows the system to say, "I'm not sure yet, and that's okay," without making a rash decision. It holds onto the uncertainty longer, waiting for the hacker to slip up.

3. Learning from Mistakes (Reinforcement Learning)

The system uses a method called PPO (Proximal Policy Optimization). Think of this as a video game character learning to play.

The character tries different moves: "Alert the police," "Slow down traffic," or "Isolate the suspicious car."
If the move stops an attack, they get a "point." If they slow down traffic unnecessarily, they lose a point.
Over 600 training sessions (simulated games), the system learns the perfect balance between being safe and not annoying the drivers.

What Happened in the Experiments?

The researchers built a computer simulation where a smart hacker tried to attack a network of cars. They tested their new "Quantum" guard against an "Old School" guard.

The Results:

Less Damage: The Quantum guard caused 60% less damage overall.
Stability: This is the biggest win. The old guard's performance was all over the place (sometimes great, sometimes terrible). The Quantum guard was 10 times more stable. It didn't have wild swings in performance.
Zero Success for Hackers: In the test runs, the Quantum guard stopped 100% of the attacks from succeeding, whereas the old guard let about 25% of them through.
Why? The old guard got "overconfident" when the hacker tried to trick it. The Quantum guard stayed calm, kept its options open, and waited for the truth to reveal itself.

The "Black Box" Check (Explainability)

The authors didn't just trust the results; they looked inside the "brain" of the AI using tools like SHAP and LIME.

They found that the Quantum guard was actually listening to its "uncertainty" signals.
When the hacker was being tricky (evasion), the Quantum guard's internal "wave" stayed spread out, telling the system: "Don't act yet, I'm still figuring this out."
The old guard, by contrast, had collapsed its thinking into a single wrong guess, leading to bad decisions.

Summary

The paper claims that by using math inspired by quantum physics (keeping uncertainty "alive" like a spinning coin instead of forcing a guess), the new security system is much better at defending smart cars against hackers who try to trick it. It is smarter, calmer, and more consistent than current methods, all without needing any special quantum hardware—just better math running on regular computers.

Technical Summary: Belief-Space Quantum-Inspired Reinforcement Learning for IoV Security

Problem Formulation
The paper addresses the security challenges inherent in the Internet of Vehicles (IoV), where the dynamic and adversarial nature of the environment renders traditional static intrusion detection systems (IDS) ineffective. Existing approaches rely on offline datasets and fixed classifiers, failing to account for sequential decision-making, attacker adaptation, and the partial observability of real-world deployments. Specifically, the authors identify a critical limitation in current Reinforcement Learning (RL) defenses: they rely on classical Bayesian belief models to track hidden attacker intent. Under partial observability, particularly when attackers employ probing or evasion strategies, these classical models tend to "collapse" into overconfident estimates. This collapse corrupts the policy input, leading to erratic, high-variance defensive actions precisely when reliable reasoning is most needed.

Methodology: Q-BIRD
To overcome these limitations, the authors propose Quantum Belief-Integrated Reinforcement Defense (Q-BIRD). The framework models IoV security as a sequential attacker-defender interaction under partial observability, formulated as a Partially Observable Markov Decision Process (POMDP).

Quantum-Inspired Belief Representation: Unlike classical approaches that maintain a probability vector over attacker intent hypotheses, Q-BIRD utilizes an amplitude-based belief state ( $\psi_t \in \mathbb{C}^H$ ). The defender maintains a state vector of complex numbers, where each entry corresponds to a possible attacker intent (e.g., benign, probing, attack, evasion).
- Evolution: The belief state evolves via a linear transformation matrix ( $U_t$ ) in the amplitude space, rather than direct probability updates.
- Probability Extraction: Probabilities are derived only when needed using the Born rule ( $b_t(\theta_i) = |\psi_t(i)|^2$ ).
- Mechanism: This approach allows uncertainty to persist in a "superposition" of hypotheses under ambiguous or deceptive observations. It prevents the premature collapse of belief that occurs in classical Bayesian updates, enabling non-Bayesian belief evolution that is more robust to noise and adversarial manipulation.
Integration with RL: The quantum belief state is integrated into a Proximal Policy Optimization (PPO) defender. The policy ( $\pi_\phi$ ) takes as input both the observable system state ( $s_t$ ) and the belief state ( $b_t$ or $\psi_t$ ). The defender selects cost-aware mitigation actions (alerting, throttling, isolation) to minimize a reward function defined as the negative sum of cumulative damage and operational costs.
Threat Model: The environment features an adaptive attacker with a hidden intent state that evolves based on the defender's actions. The attacker dynamically shifts between probing, active attacks, and evasion strategies to evade detection, creating a non-stationary environment.

Key Contributions
The paper outlines four primary contributions:

Formulation: IoV security is formulated as a sequential decision process with an adaptive attacker operating under a hidden intent state.
Quantum-Inspired Belief: A novel amplitude-based belief representation is proposed that preserves distributed uncertainty across intent hypotheses under ambiguous observations, contrasting with the collapse-prone nature of classical Bayesian beliefs.
Policy Integration: The quantum belief state is integrated into a PPO-based defender, allowing for cost-aware mitigation actions conditioned on both observable states and belief uncertainty.
Validation: The framework is validated through controlled simulations and post-hoc explainability analyses (SHAP, LIME, Grad-CAM), confirming that the quantum belief state serves as the primary decision signal during attacker strategy transitions.

Experimental Results
The approach was evaluated in a simulated IoV environment using 600 training episodes and 80 test episodes, with an adaptive attacker model. The results were compared against baselines including PPO with classical Bayesian belief, Q-Learning, and random policies.

Damage Reduction: The proposed method achieved a 60.4% reduction in cumulative mean damage compared to PPO with classical Bayesian belief (27.495 vs. 69.348).
Stability: Crucially, the method demonstrated a 90.2% reduction in damage variance (3.636 vs. 37.054), representing a 10.2-fold improvement in stability. This indicates the defender's behavior is significantly more predictable and reliable.
Attack Success Rate (ASR): The quantum PPO achieved an ASR of 0.000 in test episodes, a 50% improvement over the classical PPO baseline (0.247).
Survival Probability: Survival probability increased by 46.4% (1.000 vs. 0.683).
Ablation Findings: The study noted that a random policy actually outperformed PPO with classical belief in mean damage. This suggests that the classical belief model actively harms the policy by providing false, overconfident state data during evasion, whereas the quantum belief maintains uncertainty, allowing the policy to remain consistent.

Significance and Claims
The paper claims that the primary source of performance gain is the quantum-inspired belief representation, not the underlying RL algorithm (PPO), as confirmed by ablation studies where only the belief update mechanism differed. The authors assert that non-classical uncertainty representations provide superior security for IoV networks without requiring quantum hardware; the "quantum" aspect is a mathematical framework for representing uncertainty on classical hardware.

The significance lies in addressing the specific failure mode of classical belief models in adversarial settings: the "collapse" of uncertainty under deceptive observations. By preserving uncertainty in amplitude space, Q-BIRD enables robust decision-making during strategy transitions, offering a more stable and effective defense against adaptive adversaries in safety-critical IoV environments. The work suggests that future autonomous defense systems must move beyond static detection and classical probabilistic modeling to handle the ambiguity and adaptability of modern cyber threats.