Diagnosing Simulation and Hardware Barriers to Cross-Size Transfer in Equivariant Quantum Reinforcement Learning

Problem Statement

Combinatorial optimization (CO) problems, such as the Traveling Salesman Problem (TSP), are central to logistics and network design but are NP-hard, making exact solutions intractable for large inputs. While Reinforcement Learning (RL) and Quantum Reinforcement Learning (QRL) offer heuristic strategies, a major scalability bottleneck remains: most QRL methods require retraining from scratch for every new problem instance or size.

This paper investigates whether Equivariant Quantum Circuits (EQCs)—which encode permutation symmetry to ensure parameter counts remain independent of problem size—can facilitate zero-shot cross-size transfer. Specifically, the authors ask if parameters $\theta_n$ trained on small $n$-city TSP instances can be directly transferred to larger $m$-city instances ($m > n$) without retraining, and whether this transfer survives realistic execution on quantum hardware.

Methodology

The authors employ a five-stage, protocol-matched evaluation pipeline to isolate the effects of simulation approximations, noise, and finite-shot sampling from genuine transfer behavior. The study uses the Skolik et al. [11] EQC architecture (depth $L=1$, two trainable scalars $\beta, \gamma$) applied to Euclidean TSP instances.

The pipeline progresses through:

1. Base Stage: Selection of a stable training recipe (frozen observation rescaling, best-checkpoint reporting) on a 12-node validation lane.
2. N1 (Source Policies): Training from-scratch on $n \in \{5, 10, 15, 20\}$ cities using exact statevector simulation.
3. N2 (Exact Transfer): Evaluating zero-shot transfer and fine-tuning across sizes (e.g., $5 \to 10$, $10 \to 15$) using exact statevector simulation.
4. N3 (Scaling Frontier): Extending to larger sizes ($n$ up to 100) using Matrix Product State (MPS) simulation, noisy MPS (IBM noise model), and high-bond-dimension sweeps to characterize simulation fidelity limits.
5. N4 (Hardware Execution): Executing the same transferred policies on Quantinuum trapped-ion hardware (H2-2, Helios-1) and emulators, followed by a cross-platform campaign across five devices (Quantinuum, IBM, Rigetti, IQM) spanning two qubit technologies and four vendors.

Theoretical Framework

The paper derives a conditional diagnostic transfer bound for the expected performance $P_m(\theta^*_n)$ on a target size $m$ given source training on size $n$. The bound decomposes the performance drop into:

• Source Generalization: A term $G^{roll}_n(\delta)$ controlling the gap between empirical and true source performance.
• Transfer Penalty ($D_{n \to m}$): A sum of:
  • Parametric Mismatch: Scaling linearly with the parameter $\ell_1$-norm and the relative size jump $(m-n)/m$.
  • Structural Smoothness: A term $\Psi(n, m)$ (modeled as $\sqrt{m} - \sqrt{n}$ based on Beardwood–Halton–Hammersley scaling) representing the smoothness of the policy across graph sizes.

The authors explicitly note that this bound is diagnostic and worst-case, intended to identify scaling structures rather than predict numerical gaps with high precision. They also clarify that the specific EQC architecture studied (Model B) is classically simulable via Lie-algebraic methods (Model A), and they use this simulability as a "measurement instrument" (ground truth) rather than claiming quantum advantage.

Key Results

1. Validated In-Regime Transfer

Within a validated regime (small size jumps, e.g., $5 \to 10$), zero-shot transfer beats target-size training in all six completed evaluations. The $5 \to 10$ transfer achieved a mean optimality gap of 5.07%, outperforming the scratch-trained baseline. This suggests that EQCs encode structural insights that generalize across scales when the size jump is moderate.

2. Three Independent Barriers to Scalability

Beyond the validated regime, three distinct barriers prevent the dense, all-to-all EQC architecture from scaling:

• Barrier B1: Backend-Induced Signal Loss (Simulation/Truncation)

  • Observation: In MPS simulation, a bond dimension of $\chi=64$ (standard for many studies) causes a catastrophic failure for $n=20$ (mean gap 85.22%), even for equal-size transfer.
  • Cause: The all-to-all entanglement structure generates entanglement growth that invalidates low-bond-dimension approximations. The truncation error exceeds the signal magnitude ($\langle Z_i Z_j \rangle$), corrupting the greedy action ranking.
  • Threshold: Reliable policy-level accuracy requires $\chi \ge 256$, which is computationally prohibitive for large $n$.

• Barrier B2: Cross-Size Transfer Degradation (Large Jumps)

  • Observation: Performance degrades smoothly but substantially as the size jump $|m-n|$ increases (e.g., $5 \to 20$ gap rises to ~12–18%).
  • Cause: This aligns with the theoretical transfer bound, driven by parametric mismatch and structural shifts. While fine-tuning can recover some performance (e.g., $15 \to 20$ gap drops from 13.9% to 10.6% with fine-tuning), zero-shot transfer alone is insufficient for large jumps.

• Barrier B3: Finite-Shot Execution Penalty (Hardware)

  • Observation: On hardware, the gap inflates from ~5% (statevector) to 31.3% (noiseless emulator with 4096 shots) and 45.3% (actual hardware).
  • Cause: The action margins (differences between candidate Q-values) lie below the shot-noise floor ($1/\sqrt{N_s}$). At 4096 shots, the noise floor is ~0.016, while the mean action margin is ~0.006. Consequently, greedy decisions become statistically unresolved (37 of 40 decisions had $z < 2$).
  • Cross-Platform Confirmation: A campaign across five devices (Quantinuum, IBM, Rigetti, IQM) confirmed that the penalty is set by the native two-qubit gate count and error mitigation, not the shot budget.
    • Trapped-ion (all-to-all, 45 gates): 45.3% gap.
    • Superconducting (routed, 153–172 gates, unmitigated): 108–125% gap (effectively random tours).
    • Superconducting (mitigated, 172 gates): 67.8% gap.
  • Conclusion: Simply increasing shots does not resolve the issue because the signal-to-noise ratio is fundamentally limited by the gate-induced bias and the low per-edge contrast of the dense observable family.

Significance and Claims

The authors explicitly make no claim of computational quantum advantage. The EQC architecture studied is classically simulable. Instead, the paper's significance lies in:

1. Establishing a Diagnostic Standard: Providing a reproducible, protocol-matched methodology for evaluating cross-size transfer in QRL, disentangling transfer behavior from simulator or hardware artifacts.
2. Identifying Architectural Obstructions: Demonstrating that the canonical dense, all-to-all EQC ansatz faces fundamental barriers (B1, B2, B3) rooted in its connectivity. The all-to-all topology invalidates efficient tensor-network simulation (B1), exacerbates transfer penalties (B2), and creates action margins too small to resolve on current hardware (B3).
3. Proposing a Path Forward: The results motivate sparse equivariant circuit designs as the necessary architectural fix to relax all three barriers simultaneously, rather than relying on increased shot budgets or error mitigation alone.

The paper concludes that while EQCs offer a promising inductive bias for transfer, their current dense implementation is not scalable to industrially relevant sizes on existing hardware without architectural modification.