Automated generation of photonic circuits for Bell tests with homodyne measurements

1. Problem Statement

The violation of Bell inequalities (specifically the Clauser-Horne-Shimony-Holt or CHSH inequality) is essential for device-independent (DI) quantum information processing, such as DI quantum key distribution (DIQKD) and self-testing. While proof-of-principle experiments exist, identifying physical platforms that are realistic, scalable, and close to commercial viability remains a challenge.

• The Gap: Existing proposals for photonic Bell tests using homodyne measurements (which are efficient, low-noise, and operate at room temperature) often require highly complex, non-standard states that are experimentally infeasible to generate.
• Current Limitations: The best known realistic optical setups using homodyne detection achieve a CHSH score ($B$) of approximately 2.048. This is below the threshold required for certain advanced DI protocols (e.g., self-testing the singlet state requires $B > 2.051$, and advanced protocols require $B \gtrsim 2.106$).
• The Challenge: The search space for photonic circuits grows exponentially with the number of optical elements, making manual design of circuits that maximize CHSH violations while remaining experimentally feasible extremely difficult.

2. Methodology

The authors propose an automated framework combining Deep Reinforcement Learning (RL) with efficient numerical simulations of quantum optical processes to discover photonic circuits.

A. The Learning Task

• Agent-Environment Framework: The environment is an $N$-mode optical circuit. The agent's goal is to construct a circuit that generates a nonlocal state between Alice (mode 1) and Bob (mode 2), while the remaining $N-2$ modes are used for heralding.
• Action Space: The agent appends Gaussian gates to the circuit. The available gates include:
  • Passive: Beam splitters ($\hat{B}(\theta)$) and Phase shifters ($\hat{R}(\theta)$).
  • Active: Single-mode squeezers ($\hat{S}_1(r)$) and Two-mode squeezers ($\hat{S}_2(r)$).
• State Representation: The state shared by Alice and Bob is a heralded state obtained by detecting photons in the auxiliary modes. Since Gaussian states and homodyne measurements alone cannot violate Bell inequalities, the authors use threshold detectors to herald non-Gaussian states. The state is represented as a linear combination of Gaussian states (Eq. 10 in the paper), parameterized by covariance matrices and coefficients.
• Reward Function: The agent receives a reward only at the end of an episode (when the circuit reaches a predefined depth $n_{circuit}$). The reward is an increasing function of the CHSH score $B$:
  • If $B < 2$: Linear, weak reward.
  • If $B \ge 2$: Exponential reward to strongly bias the agent toward violations.
• Optimization: After the circuit is constructed, the parameters of the gates are numerically optimized (using the Nelder-Mead algorithm) to maximize the CHSH score for specific measurement settings.

B. Learning Strategies

To improve efficiency and experimental realism, the authors defined specific exploration strategies:

1. Initialization: Instead of starting from a vacuum and allowing the agent to choose any gate (which leads to unrealistic sequential squeezing), the circuit is initialized with fixed active gates (squeezers) to generate photons.
2. Action Restriction: The agent is restricted to choosing only passive gates (beam splitters and phase shifters) for the remainder of the circuit. This reduces the action space and prevents unrealistic chains of squeezers.
3. Heralding Schemes: Two schemes were tested:
   • Click detection: Standard threshold detection on auxiliary modes.
   • Single-photon projection approximation: Using an unbalanced beam splitter and threshold detectors to approximate the projection onto a single-photon Fock state, which is known to induce necessary non-Gaussianity.

C. Algorithm

The authors utilize Proximal Policy Optimization (PPO), a state-of-the-art RL algorithm, implemented via the Julia Reinforcement Learning package. They compare the PPO agent's performance against a random search over the same circuit space.

3. Key Contributions

1. Automated Design Framework: A novel pipeline integrating RL with Gaussian state simulation to automatically discover photonic circuits for Bell tests.
2. Experimental Feasibility: The method specifically targets setups using standard optical components (beam splitters, phase shifters, squeezers) and homodyne detection, avoiding exotic or currently impossible state preparations.
3. Superior CHSH Scores: The framework discovers circuits that significantly outperform previous realistic proposals, achieving scores well above the 2.051 threshold required for singlet self-testing.
4. Robustness Analysis: Comprehensive analysis of the proposed circuits against photon loss (fiber attenuation) and detector inefficiency.

4. Key Results

• Optimal Circuit Discovery:
  • The most successful setup found involves 4 bosonic modes and 4 optical components (two two-mode squeezers and two beam splitters).
  • CHSH Score: $B \approx 2.068$.
  • Parameters: Requires squeezing levels of 3.9 dB and 0.008 dB.
  • Heralding: The state is heralded by clicks in threshold detectors on modes 3 and 4.
• Performance Comparison:
  • In the 4-mode scenario, the PPO agent found a circuit with $B=2.069$, slightly outperforming the random search ($B=2.068$).
  • In the 6-mode scenario, random search found a more complex circuit (42 gates) with $B=2.076$, while the PPO agent found a simpler circuit (8 gates) with $B=2.072$. The authors note that the RL agent was limited by computational resources and network depth, suggesting potential for further improvement with larger models.
• Robustness to Loss:
  • The proposed 4-mode circuit maintains a CHSH violation ($B > 2$) for distances up to 8.1 km between Alice and Bob (assuming 0.2 dB/km fiber loss).
  • A "central station" configuration (state preparation in the middle) extends this range to 9.8 km.
• Robustness to Detector Inefficiency:
  • The CHSH score is remarkably insensitive to the efficiency ($\eta$) of the heralding threshold detectors. Even as $\eta \to 0$, the score remains near $B=2.067$.
  • This implies that cryogenic single-photon detectors are not required; basic threshold detectors suffice, provided the heralding probability is accounted for.

5. Significance

• First Loophole-Free Candidate: This work proposes the first realistic photonic setup using homodyne measurements capable of achieving a loophole-free Bell test (or DIQKD) with current or near-future technology.
• Scalability: The use of integrated photonic circuits and homodyne detection (which operates at telecom wavelengths and room temperature) makes this approach highly scalable compared to discrete-variable approaches requiring single-photon detectors.
• Methodological Impact: The study demonstrates that Reinforcement Learning is a powerful tool for navigating the vast, complex space of quantum optical circuit design, potentially accelerating the discovery of protocols for other quantum information tasks.
• Practical Viability: By showing that high CHSH violations can be achieved with modest squeezing (3.9 dB) and standard components, the paper bridges the gap between theoretical nonlocality proofs and practical, device-independent quantum networks.