Graph Reinforcement Learning for Calibration-Aware Quantum Circuit Routing

This paper presents a calibration-aware graph reinforcement learning approach for quantum circuit routing that, by leveraging real-time hardware calibration data and proximal policy optimization, achieves significantly higher simulated fidelity than standard SABRE-based methods on small-to-medium circuits, despite incurring higher two-qubit gate counts.

The Gist

Imagine you are trying to guide a team of delivery drivers (the quantum data) through a massive, chaotic city (the quantum computer) to deliver packages (perform calculations).

In the past, navigation apps for these quantum cities only cared about one thing: distance. They would tell the drivers, "Take the shortest route, even if it means driving over a pothole-ridden bridge or through a construction zone." The logic was simple: fewer miles driven equals less wear and tear.

However, this paper argues that in the real world of quantum computers, distance isn't everything. Sometimes, a slightly longer route that avoids a broken bridge is actually much better because it gets the package to its destination in better condition.

Here is a breakdown of what the researchers did, using simple analogies:

The Problem: The "Perfect" Route vs. The "Real" Route

Quantum computers are like cities where the roads (connections between parts of the computer) are constantly changing quality. Some roads are smooth and fast; others are bumpy and prone to breaking down. This quality is called "calibration."

Old navigation systems (like the standard SABRE algorithm mentioned in the paper) are like GPS apps that only look at a map. They say, "Go this way because it's 5 miles." They don't know that the 5-mile road is currently flooded, while the 6-mile road is dry.

The Solution: A "Calibration-Aware" GPS

The authors created a new, smarter navigation system using Graph Reinforcement Learning. Think of this as a GPS that doesn't just look at the map, but also checks the live traffic report and the weather forecast for every single road before making a decision.

• The "Brain": They trained an AI (using a method called Proximal Policy Optimization) to act as the navigator.
• The Input: Before telling the drivers where to go, the AI looks at:
  1. The remaining delivery list (the circuit).
  2. Where the drivers are currently parked (the placement).
  3. The live health report of every road (the calibration data from IBM's Heron r2 chip).
• The Strategy: The AI is willing to take a slightly longer route (adding more "SWAP" operations, which are like detours) if it means avoiding a road that is known to be broken or noisy.

The Experiment: A Race Against the Old Way

The researchers tested their new AI navigator against two established "old school" GPS systems:

1. SABRE-best20: The standard, distance-focused navigator.
2. Target-aware SABRE: A slightly smarter version that knows the map but doesn't use live traffic data as effectively.

They ran the test on nine different "delivery routes" (quantum circuits) of varying sizes (5, 8, and 10 stops) using real-time data from IBM's quantum hardware.

The Results: Quality Over Quantity

The results were a clear win for the new AI, but with a twist:

• The Big Win: On smaller and medium-sized routes (5 and 8 stops), the AI's routes were much more successful. The "packages" arrived in much better condition.
  • The Score: The AI achieved a "fidelity" (success rate) of 0.727, while the old methods scored around 0.440 and 0.481. That's a huge jump in quality.
• The Trade-off: To get this high quality, the AI did take more steps. It added about 8 extra detours (two-qubit gates) and made the route slightly deeper.
  • The Lesson: Taking a few extra steps to avoid a broken bridge is worth it if it saves the cargo.
• The Limitation: On the largest routes (10 stops), the AI didn't do as well. Why? Because the "city map" they were given was a rigid tree shape with very few alternative paths. When there are no good detours available, the AI couldn't outsmart the old distance-focused GPS.

The Bottom Line

This paper proves that for quantum computers, knowing the current health of the hardware is more important than just counting the number of steps.

By teaching an AI to look at the "live traffic" (calibration data) and choose routes that avoid "broken bridges" (noisy couplers), even if those routes are slightly longer, we can get much better results. It's a shift from asking "What is the shortest path?" to asking "What is the safest path?"

Technical Summary

Technical Summary: Graph Reinforcement Learning for Calibration-Aware Quantum Circuit Routing

Problem Statement
Quantum circuit routing is a critical compilation step for Noisy Intermediate-Scale Quantum (NISQ) devices, where logical circuits must be mapped to physical hardware with sparse connectivity. Traditional routing strategies often optimize for standard overhead metrics, such as minimizing the number of SWAP operations or circuit depth. However, the authors argue that on calibrated hardware, these metrics are insufficient. Two routes with similar overheads may traverse physical couplers with vastly different error rates, leading to significant differences in the final state fidelity. A route with more gates might actually preserve the ideal state better if it avoids poorly calibrated couplers. The core challenge is to develop a routing policy that utilizes same-day calibration data to maximize exact simulated state fidelity, even if it requires inserting additional two-qubit gates.

Methodology
The authors propose a calibration-aware graph reinforcement learning (RL) router trained using Proximal Policy Optimization (PPO). The approach models routing as a sequential decision-making process on a calibrated backend graph $G_B = (P, E, \kappa)$, where $P$ represents physical qubits, $E$ executable couplers, and $\kappa$ snapshot calibration data (including readout, one- and two-qubit errors, and coherence times).

• State Representation: The observation state $s_t$ includes the remaining logical circuit, the current non-identity placement of logical qubits, and the calibration snapshot. This is encoded as a graph where node features capture readout error, coherence, incident two-qubit error, and lookahead demand distance. Edge attributes include calibrated two-qubit error probabilities and a legal-action mask.
• Policy Architecture: The policy utilizes a Graph Neural Network (GNN) with two message-passing layers to generate node embeddings. A Multi-Layer Perceptron (MLP) scores legal SWAP edges based on these embeddings and edge attributes, outputting a probability distribution over valid SWAPs via a masked softmax.
• Training Protocol: The agent is trained on IBM Heron r2 calibration snapshots (Fez, Kingston, Marrakesh) using nine MQT Bench circuits (5q, 8q, and 10q families).
  • Reward Function: To avoid the high cost of exact density-matrix simulation during training, the authors employ a low-cost proxy reward based on Estimated Success Probability (ESP). The reward function includes terms for reducing shortest-path distance, routing progress, gate counts, and penalties for invalid actions or timeouts. A terminal reward compares the agent's proxy fidelity against a baseline (SABRE-best20) and penalizes excess cost.
  • Evaluation: Final evaluation uses exact density-matrix simulation with a noisy model (including depolarizing errors and thermal relaxation) to compute the true state fidelity $F = \langle \psi | \rho | \psi \rangle$.
• Baselines: The proposed method is compared against two reproducible baselines:
  1. SABRE-best20: A standard heuristic minimizing a cost function of two-qubit count and depth.
  2. Target-aware SABRE: A calibration-aware heuristic using Qiskit's target information and ESP for selection.

Key Results
The evaluation was conducted across three calibration snapshots and nine circuit families, totaling 1,500 paired episodes.

• Fidelity Gains: The learned policy achieved a pooled mean exact fidelity of 0.727, significantly outperforming SABRE-best20 (0.440) and target-aware SABRE (0.481). The improvement was statistically significant ($p < 1.5 \times 10^{-6}$).
• Overhead Trade-off: The fidelity gains came at the cost of increased overhead. The learned routes added an average of +8.63 two-qubit gates and +4.61 depth compared to SABRE-best20.
• Circuit Size Dependency: The performance gains were highly dependent on circuit size and the flexibility of the action graph:
  • 5q and 8q Families: The router successfully used additional gates to steer the circuit away from unreliable couplers, resulting in substantial fidelity improvements.
  • 10q Families: On the fixed tree action graph used in the study, the 10q families showed no fidelity gain; in fact, SABRE-best20 performed better. The authors attribute this to the fixed tree topology offering too few alternative paths for the RL agent to exploit calibration data effectively.

Significance and Claims
The paper claims that calibration-aware learned routing can improve exact state fidelity beyond what is achievable by gate-count-driven compilation, provided the hardware graph offers sufficient alternative paths. The study demonstrates that:

1. Calibration Data is Critical: Same-day calibration data allows a learned policy to make routing decisions that prioritize fidelity over minimal gate counts.
2. Action Space Constraints Matter: The utility of calibration-aware routing is contingent on the action graph providing useful alternatives. In constrained topologies (like the fixed tree used for 10q circuits), the ability to choose better couplers is limited, and traditional heuristics may remain superior.
3. Metrics Limitations: Gate count and depth are incomplete proxies for fidelity on calibrated hardware; routes with higher overhead can yield higher fidelity.

The authors conclude that while their specific implementation shows promise, future work requires evaluating cyclic subgraphs, held-out circuits, and matched learned-router baselines to fully validate the approach. They emphasize that routing comparisons should report fidelity and calibration context alongside traditional overhead metrics.