Entanglement as a Structural Complexity Axis: A PAC-Bayesian View of Generalization in Quantum Policies and Value Functions

Technical Summary: Entanglement as a Structural Complexity Axis in Quantum Policies

Problem Statement
Parameterized quantum circuits (PQCs) are increasingly deployed as policies and value functions in Quantum Reinforcement Learning (QRL). However, the field remains predominantly empirical, with evaluations typically reporting only average return. This leaves a critical theoretical gap: there is no rigorous understanding of when and why a quantum policy generalizes. Existing generalization bounds for quantum models often rely on parameter counts or gate counts, which fail to capture the structural nuances of entanglement. Furthermore, no prior PAC-Bayesian analysis has isolated entanglement as a governing complexity factor for quantum policies.

Methodology
The authors propose a PAC-Bayesian framework to derive a generalization bound for stochastic quantum policies. The core methodological innovation is shifting the complexity metric from the raw number of parameters ($d$) to the effective dimension ($d_{\text{eff}}$) of the Fisher geometry induced by the circuit.

1. PAC-Bayes Bound Derivation:

   • The authors define a stochastic policy $\pi_Q$ via a Gaussian posterior $Q = \mathcal{N}(\theta, \Sigma)$ over circuit parameters, centered at the initialization $\theta_0$.
   • Unlike standard isotropic bounds, the posterior covariance $\Sigma$ is shaped by the Fisher Information Matrix (FIM), $\hat{F}(\theta)$, which captures the local curvature of the loss landscape.
   • The resulting bound (Theorem 2) includes a complexity term composed of the distance to the prior and the Fisher effective dimension:
     $$d_{\text{eff}}(\gamma) = \log \det(I + \gamma \hat{F}) = \sum_{i=1}^d \log(1 + \gamma \lambda_i)$$
     where $\lambda_i$ are the eigenvalues of the FIM.

2. The Entanglement Mechanism:

   • The paper posits that for single-observable readouts (standard in QRL), the rank of the FIM is determined by the size of the backward light-cone of the readout observable.
   • Proposition 1 establishes that adding entangling gates expands this light-cone, thereby increasing the rank of the FIM and inflating $d_{\text{eff}}$, even when the total number of trainable parameters ($d$) remains fixed.

3. Experimental Design:

   • To isolate entanglement from parameter count, the authors use a fixed ansatz layout (layers of $R_Y$ and $R_Z$ rotations) and vary only the entangling connectivity: None (no CNOTs), Linear (CNOT chain), and Full (all-to-all CNOTs).
   • Experiments span supervised classification (4–16 qubits), a reward-only contextual bandit (one-step RL), and multi-step value-function generalization.
   • Controls include matched training accuracy, alternative readout observables, different optimizers, and execution on real IBM Heron hardware.

Key Contributions

1. A PAC-Bayes Bound for Quantum Policies: The paper derives a generalization bound where the complexity term is the Fisher effective dimension ($d_{\text{eff}}$) rather than parameter count. This bound is valid for stochastic policies under Markovian dynamics.
2. Entanglement as an Independent Complexity Axis: The authors demonstrate that entanglement is a distinct structural factor controlling generalization. At a fixed parameter count, increasing entanglement connectivity monotonically increases $d_{\text{eff}}$ and the generalization gap.
3. Empirical Validation Across Domains: The study confirms that entangled circuits generalize worse than non-entangled circuits of identical parameter count across:
   • Supervised classification on synthetic and real-world datasets (Iris, Breast Cancer, Wine).
   • Contextual bandits (reward-only RL).
   • Multi-step value-function regression.
   • Real quantum hardware (IBM Heron).

Results

• Predictive Power: In a grid of 300 configurations, the Fisher effective dimension ($d_{\text{eff}}$) showed the strongest Spearman correlation with the generalization gap ($\rho = 0.82$), outperforming raw parameter count ($\rho = 0.45$) and learned parameter norms ($\rho = 0.73$).
• Generalization Gap: For fixed parameter counts (e.g., $d=16$), the generalization gap followed the ordering: None < Linear < Full. For instance, in supervised classification with $N=64$, the "Full" entangled circuit had a gap of 0.104, while the "None" circuit had a gap of -0.01 (effectively zero).
• Ranking Certificate: While the absolute numerical bound is loose (within $\approx 2.3\text{--}4.5\times$ of the true gap), it serves as a robust ranking certificate. It correctly orders circuits of identical size, a capability parameter-counting bounds lack entirely.
• Readout Dependence: The mechanism is "readout-gated." Replacing a local readout ($\langle Z_0 \rangle$) with a global readout ($\frac{1}{n}\sum \langle Z_i \rangle$) activates parameters in non-entangled circuits, raising their $d_{\text{eff}}$ and gap, thereby absorbing the entanglement contrast at small qubit counts. This confirms that the gap is driven by the number of parameters causally coupled to the readout.
• Real Hardware: The entanglement-generalization trade-off persists on the IBM Heron processor under real noise, with entangled circuits showing larger gaps than non-entangled ones.
• Limitations: The effect is depth-gated (a single entangling layer is inert for single-qubit readouts). In deep circuits approaching barren plateaus, the Fisher spectrum flattens, suppressing $d_{\text{eff}}$ and the mechanism. In end-to-end multi-step policy learning (REINFORCE), high variance in return estimates makes the full ordering difficult to resolve, though the trend remains significant at small sample sizes.

Significance and Claims
The paper reframes the design of quantum policies around an entanglement–generalization trade-off rather than expressivity alone. The authors claim that entanglement is a "double-edged sword": it expands the representable function space but incurs a generalization cost that parameter-counting analyses miss.

The primary contribution is not a solved account of generalization in all end-to-end QRL, but a controlled study demonstrating that:

1. Entanglement acts as an independent axis of complexity.
2. The Fisher effective dimension is the governing quantity for generalization in quantum decision models (policies and value functions).
3. Designers should "budget" entanglement against its $d_{\text{eff}}$ cost, selecting the sparsest entangling pattern that achieves necessary training fit without excessive complexity inflation.

The authors explicitly state that their strongest evidence comes from low-variance settings (classifiers, value heads, and one-step policies), treating the paper as a foundational step toward understanding generalization in quantum RL rather than a complete solution for high-variance, multi-step policy learning.