⚡ electrical engineering

Predictive Fidelity-Aware Routing and Adaptive Scheduling in Distributed NISQ Quantum Networks via Actor–Critic Reinforcement Learning

This paper proposes a Deep Reinforcement Learning-based framework (DRL-QCOF) that integrates a Bi-LSTM fidelity predictor with an Actor–Critic scheduling agent to jointly optimize routing and scheduling in distributed NISQ quantum networks, significantly outperforming existing methods in link fidelity, latency, and resource utilization under dynamic conditions.

Original authors: Prabu Pachiyannan

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Prabu Pachiyannan

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive, high-speed delivery network, but instead of trucks carrying pizza, it's a fleet of invisible "quantum couriers" zipping around a city made of light and atoms. These couriers carry something incredibly fragile: entanglement, the spooky connection that powers future quantum computers. The problem? The roads they travel on are bumpy, the weather changes instantly, and the couriers get tired (decoherence) or lose their packages (noise) very easily.

For a long time, the traffic controllers for this quantum city have been using old-fashioned rulebooks. They check the map, see a road is clear, and send the courier. But if a sudden storm hits or a bridge gets jammed, these rulebooks are too slow to react. They keep sending couriers down broken roads, leading to lost packages and frustrated customers.

Enter a new, super-smart traffic system called DRL-QCOF. Think of it as a team of two genius AI assistants working together to manage the quantum delivery service.

The Crystal Ball and The Traffic Cop

The first assistant is the Quantum Fidelity Prediction Network (QFPN). Imagine this assistant has a crystal ball made of math. Instead of just looking at the road right now, it looks at the history of the road—the bumps, the noise, and the traffic jams from the last few minutes. Using a special brain called a Bi-LSTM (which is like a memory that remembers things from the past and the future), it predicts what the road will look like in the next few seconds. It tells the system, "Hey, that road is going to get bumpy in 5 seconds; don't send the courier there yet!"

The second assistant is the DRAQS-Agent, the actual traffic cop. This agent uses a "Actor-Critic" system. The Actor is the decision-maker who chooses which road to take and how fast to go. The Critic is the judge that watches the Actor and says, "Good choice!" or "Bad choice!" based on a scorecard. This scorecard rewards the agent for keeping the packages safe (high fidelity), getting them there fast (low latency), and not clogging the roads (good resource utilization).

Together, they don't just react to traffic; they predict it. If the crystal ball says a road is about to get noisy, the traffic cop reroutes the courier before the noise even happens.

The Race Against the Old Rules

The authors tested this new system in a giant computer simulation of a quantum city (using tools called NetSquid and OMNeT++). They pitted their new AI against the old rulebooks, like HWFR (which just weighs past history) and DMPQS (which tries to juggle priorities but gets confused by chaos).

Here is what the simulation showed:

Keeping the Packages Safe: The old methods managed to keep the quantum connection strong about 0.78 to 0.86 of the time. The new AI, however, kept the connection strong up to 0.88. That might sound like a small number, but in the fragile world of quantum physics, it's a huge leap in reliability.
Speeding Up: The old systems took about 104 milliseconds to get a package from start to finish. The new AI shaved that down to 83 milliseconds. Other methods were stuck above 110 milliseconds.
Fewer Lost Packages: The old systems had to say "no" to about 7% of the delivery requests because the roads were too bad. The new AI only said "no" to 3% of requests, meaning it successfully delivered 97% of the time.
Using the Roads Better: The new system kept the quantum roads busy and efficient, using 71% to 83% of the available capacity, whereas the old systems were either too lazy or too chaotic, hovering around 67%.

The Learning Curve

The most exciting part is how fast the AI learned. The old methods took a long time to figure out the best routes, often needing hundreds of practice runs just to get stable. The new AI, however, figured out the perfect strategy in just 300 training episodes. By the time it finished its training, it was consistently delivering more than 121 entangled pairs per second, compared to the 87 to 93 pairs per second the old methods managed.

What This Means (and What It Doesn't)

The authors are very clear: this isn't a magic wand that fixes the real world today. They didn't build a physical quantum network in a lab and run this on real hardware. Instead, they built a highly detailed simulation that mimics the messy, noisy reality of current "NISQ" (Noisy Intermediate-Scale Quantum) devices.

In this simulated world, the new system proved it could handle the chaos much better than the old rules. It suggests that if we ever build a real, large-scale quantum internet, we shouldn't rely on static rulebooks. Instead, we need a system that can predict the noise and adapt its routing in real-time, just like this AI did.

So, while we aren't ordering quantum pizza delivery just yet, this study suggests that the future of the quantum internet will belong to the traffic cops who can see around corners and predict the storm before it hits.

Technical Summary: Predictive Fidelity-Aware Routing and Adaptive Scheduling in Distributed NISQ Quantum Networks via Actor–Critic Reinforcement Learning

1. Problem Statement

Distributed Noisy Intermediate-Scale Quantum (NISQ) systems rely on quantum communication to function; however, practical deployment is hindered by link fluctuations, limited quantum resources, and entanglement degradation. These factors create significant challenges for reliable routing and scheduling, particularly in large-scale, dynamically changing environments.

Existing classical approaches, such as Historical Weighted Fidelity Routing (HWFR) and Dynamic Multi-Priority Quantum Scheduling (DMPQS), perform adequately only when network conditions are stable. Their performance degrades significantly when entanglement success rates vary or relay nodes become congested. A primary limitation of current literature is the isolation of routing and scheduling optimization, often failing to explicitly account for future fidelity variations. This fragmentation leads to inefficient resource usage, increased end-to-end latency, and an inability to maintain stable link fidelity under time-varying conditions. Furthermore, many learning-based strategies struggle with generalization in environments with irregular traffic variations and suffer from slow convergence speeds.

2. Methodology

To address these challenges, the authors propose the Deep Reinforcement Learning-based Quantum Communication Optimization Framework (DRL-QCOF). This framework jointly optimizes routing and scheduling through two interconnected components:

A. Quantum Fidelity Prediction Network (QFPN)

The QFPN serves as the predictive backbone, estimating future link fidelity ( $F_{pred}(t)$ ) to enable proactive decision-making.

Architecture: It utilizes a Bi-directional Long Short-Term Memory (Bi-LSTM) network combined with a Temporal Self-Attention mechanism.
Inputs: The network processes time-series sequences of link features, including link length, noise coefficient, entanglement success probability, and relay utilization.
Mechanism: The Bi-LSTM captures temporal dependencies in both forward and backward directions, while the attention mechanism assigns significance weights to specific time steps (e.g., sudden noise spikes). A regression layer with a sigmoid activation outputs the predicted fidelity bounded between 0 and 1.
Output: A scalar predicted fidelity value used to guide the routing agent.

B. Deep Reinforcement-based Adaptive Quantum Scheduling Agent (DRAQS-Agent)

The DRAQS-Agent acts as the decision-making core, utilizing an Actor–Critic architecture to learn an adaptive policy for routing and scheduling.

State Representation: The state vector ( $S_t$ ) integrates QFPN-predicted fidelities, current queue lengths, relay utilization, latency estimates, and hop availability.
Action Space: The agent selects discrete actions comprising routing path selection, scheduling priority assignment, or request deferral/rejection based on resource scarcity.
Reward Function: A continuous reward formulation guides the agent to maximize link fidelity while minimizing latency, queue length, and relay overload. The reward is defined as:
$R_t = \alpha F_{eff} - \beta L_t - \gamma Q_t - \delta U_t$
where $F_{eff}$ is the effective multi-hop fidelity, $L_t$ is latency, $Q_t$ is queue length, and $U_t$ is utilization.
Training: The agent is trained using Temporal Difference (TD) learning with gradient clipping and entropy regularization to ensure stability. The Actor network updates the policy, while the Critic network estimates the value function.

3. Key Contributions

The paper outlines three primary contributions:

Unified Control Strategy: The proposal of an RL-based control strategy that jointly optimizes routing and resource decisions across long training horizons, demonstrating consistent performance improvement as the system evolves.
Multi-Metric Reward Structure: The design of a reward function that aligns the learning process with practical operational goals, simultaneously optimizing link fidelity, latency, blocking rate, and throughput.
Adaptive State Formulation: The introduction of a state representation that captures both transient and cumulative network dynamics, supported by the QFPN's predictive capabilities, to facilitate more accurate and stable decision patterns.

4. Experimental Results

The framework was evaluated using a co-simulation environment combining NetSquid (for quantum network dynamics) and OMNeT++ (for classical control and traffic). Simulations were conducted on a 25-node NISQ topology (based on NSFNET) over 1,000 control epochs with 30 random seeds.

Performance Comparisons:
The proposed DRL-QCOF was benchmarked against HWFR, DMPQS, Purification-Aware routing, DRL-RRA, Multi-Objective routing, and TQD-Secure routing.

Link Fidelity: The proposed method achieved a maximum average link fidelity of 0.88, outperforming existing methods which ranged between 0.78 and 0.86.
Latency: End-to-end latency was reduced from an initial 104 ms to 83 ms at convergence. Baseline schemes remained above 110 ms under similar conditions.
Response Time: Average request response time decreased to approximately 147 ms, a significant improvement over baseline schemes (e.g., HWFR at 217 ms).
Resource Utilization: The framework achieved a utilization rate of 71–83%, compared to 67–81% for existing schemes.
Success Ratio: The success ratio improved from 93% to 97%, while the blocking rate was reduced to as low as 0.04.
Throughput: The system achieved a throughput of 118–121 entangled pairs per second.
Convergence: The cumulative reward stabilized at approximately 16.0, with convergence observed within 300 training episodes, significantly faster than competing methods.

5. Significance and Claims

The paper claims that the DRL-QCOF framework offers a robust solution for the dynamic and uncertain nature of NISQ networks. By integrating predictive fidelity modeling with adaptive reinforcement learning, the system overcomes the limitations of static or semi-adaptive strategies that fail to anticipate link degradation.

The significance of this work lies in its unified, multi-metric optimization approach. Rather than prioritizing a single metric, the framework balances interdependent network properties (fidelity, latency, throughput) through a stability-driven reward shaping mechanism. The authors assert that this approach enables the network to maintain high reliability and efficiency even under fluctuating traffic loads and environmental noise, providing a practical direction toward constructing reliable, efficient, and adaptable network control solutions for distributed quantum computing. The results suggest that the method is effective in dynamic scenarios and converges faster than existing learning-based or heuristic approaches.