Deep reinforcement learning for near-deterministic preparation of cubic- and quartic-phase gates in photonic quantum computing

This paper demonstrates that deep reinforcement learning can control a quantum optical circuit using only photon-number-resolving measurements to achieve a 96% success rate in preparing cubic-phase states and directly generating quartic-phase gates for universal continuous-variable quantum computing.

The Gist

Imagine you are trying to bake a very specific, complex cake (a "cubic-phase state") that is essential for building a super-advanced quantum computer. In the world of light-based (photonic) computing, making this cake is notoriously difficult. Usually, you have to rely on a "lucky guess" method: you mix ingredients, check the result, and if it's not perfect, you throw it away and start over. This is slow and inefficient.

This paper presents a new way to bake that cake using a "smart robot chef" powered by Deep Reinforcement Learning (DRL). Here is how the authors did it, explained simply:

1. The Goal: The "Magic" Ingredient

To make a universal quantum computer that can solve any problem, you need a special ingredient called a cubic-phase state. Think of this as the "magic spice" that turns a simple, predictable machine into a powerful, complex one. Without it, the computer is limited.

2. The Old Way vs. The New Way

• The Old Way (Classical/Probabilistic): Imagine trying to bake the cake by randomly shaking a box of ingredients and hoping you get the right mix. If you get it wrong, you discard the batch. This is what previous methods did using "photon-number-resolving" (PNR) measurements. It worked, but it was like trying to win the lottery every time you wanted to bake.
• The New Way (The AI Chef): The authors trained a deep neural network (a type of AI) to act as a chef. This chef doesn't guess; it learns by doing.
  • The Setup: The "kitchen" is a loop of mirrors, beam splitters, and lasers (a quantum optical circuit).
  • The Process: The AI chef looks at the current state of the mixture (the light). It decides whether to add a pinch of "squeezing" (compressing the light), a dash of "displacement" (shifting the light), or to let the mixture pass through a beam splitter.
  • The Feedback: After each step, the chef checks the result. If the cake is getting closer to the perfect recipe, the AI gets a "reward." If it goes off-track, it gets a "penalty."
  • The Learning: Over millions of tries, the AI learns the perfect sequence of moves to create the cubic-phase state almost every time.

3. The Results: Near-Deterministic Success

The paper reports that this AI chef achieved a 96% success rate.

• What this means: Instead of throwing away 90% of your batches (as in older methods), the AI successfully bakes the cake in 96 out of 100 attempts.
• The "Reset" Trick: The AI learned a clever strategy. If it realizes a batch is ruined and cannot be fixed, it immediately hits a "reset" button (turning a mirror to start fresh) rather than wasting time trying to fix a broken cake. It also learned to stop adding ingredients once the cake is perfect, rather than over-mixing it.

4. The "Quartic" Bonus

The authors also showed that this same "kitchen" and "chef" could be used to make an even more complex cake called a quartic-phase gate.

• The Challenge: Usually, making this complex cake requires building it out of 29 smaller cubic cakes (a very long assembly line).
• The Discovery: The authors found a simpler, direct recipe using the same ingredients. While this specific version still relies on a bit of luck (post-selection), it proves that you can skip the long assembly line and make the complex cake directly. They suggest that with more training, an AI could eventually make this one reliably too.

5. Why This Matters (According to the Paper)

• Efficiency: This method requires less "squeezing" (energy) and less complex photon counting than previous proposals.
• Feasibility: The equipment needed (mirrors, lasers, and photon detectors) already exists in current labs. The only "non-standard" thing needed is the ability to count photons precisely, which is now possible.
• Robustness: The AI learned to handle "noise" (imperfections in the equipment). Even when the detector was only 99% efficient (slightly "noisy"), the AI still managed to produce high-quality results, though it had to adjust its strategy (oscillating its moves) to compensate.

In summary: The paper demonstrates that by teaching a computer to "play" with a quantum light circuit using trial-and-error learning, we can generate the most difficult and necessary ingredients for quantum computing with near-perfect reliability, turning a game of chance into a reliable manufacturing process.

Technical Summary

Technical Summary: Deep Reinforcement Learning for Near-Deterministic Preparation of Cubic- and Quartic-Phase Gates

Problem Statement
Continuous-variable quantum computing (CVQC) offers exceptional scalability and potential for fault tolerance, but achieving universality requires access to non-Gaussian resources, specifically cubic Hamiltonian evolution. While cubic-phase gates ($\exp(i\gamma Q^3)$) are sufficient for universal CVQC, their deterministic generation is challenging. Traditional approaches relying on third-order optical nonlinearities are inefficient due to weak optical nonlinearities. Probabilistic methods using photon-number-resolving (PNR) measurements, such as the Gottesman-Kitaev-Preskill (GKP) protocol, require extreme resources (e.g., ~17 dB squeezing and detection of ~50 photons) to reach useful gate parameters. Furthermore, existing optimization methods for quantum state preparation often rely on post-selection, which leads to low success rates and computationally expensive optimization over all possible detection patterns.

Methodology
The authors propose a control framework utilizing Deep Reinforcement Learning (DRL) to manage a quantum optical circuit for generating cubic-phase states.

• Quantum Circuit: The system employs a looped optical circuit containing a variable beamsplitter, a squeezing operation, and a displacement operation. The loop is terminated by a switchable mirror. A PNR detector measures the photon number in the loop, and the result conditions the density matrix input to the neural network.
• Reinforcement Learning Framework: The interaction is modeled as a Markov Decision Process (MDP).
  • State ($S$): The flattened density matrix of the circuit state at each timestep.
  • Action ($A$): A vector controlling the beamsplitter transmittivity ($\tau_j$), the squeezing parameter ($r_j$), and the displacement magnitude ($\alpha_j$).
  • Reward ($R$): A function of the fidelity between the current state and the target cubic-phase state, penalizing low fidelities and unphysical results caused by Hilbert space truncation.
• Algorithm: The authors use Proximal Policy Optimization (PPO) with an actor-critic architecture (two deep neural networks). The agent is trained to maximize the fidelity of the final state without relying on post-selection, learning to adapt to the inherent randomness of PNR measurements.
• Training Parameters: Simulations were run using the StrawberryFields and StableBaselines3 libraries. The agent was trained over millions of timesteps with a Hilbert space truncation of 31 photons. The target state was a displaced cubic-phase state with $\gamma = 0.2$.

Key Results

1. Near-Deterministic Cubic-Phase Generation:

   • The trained agent achieved an average success rate of 96% in generating cubic-phase states with $\gamma = 0.2$.
   • This was accomplished using modest resources: squeezing no higher than 10 dB, low displacements, and PNR measurements significantly lower than those required by probabilistic GKP proposals.
   • Emergent Behaviors: The agent learned to:
     • Set the beamsplitter transmittivity to zero ($\tau_j=0$) once high fidelity was reached, effectively locking the state.
     • Apply corrective displacements after locking the loop.
     • "Reset" the circuit ($\tau_j=1$) if an input state was deemed unlikely to converge, restarting the process efficiently.
   • The method proved robust even with a PNR detector efficiency of 99%, though the agent exhibited oscillatory displacement behaviors in the lossy case. At 90% efficiency, the agent failed to learn a successful policy.

2. Direct Quartic-Phase Generation:

   • The authors identified a quantum optical algorithm for the direct generation of quartic-phase states ($\exp(i\delta Q^4)$), bypassing the need for decomposing the gate into 29 cubic-phase gates.
   • This algorithm involves "stamping" the Wigner function with displaced Fock states at specific phases in phase space using a two-step PNR detection process on a cluster state.
   • Preliminary Results: Post-selected simulations (Hilbert space truncation of 60 photons) demonstrated that this method can generate quartic-phase states with high fidelity (up to 95% in specific post-selected cases), validating the intuition that quantum interference can transform circular Fock contours into the characteristic ripples of a quartic state.

Significance and Claims
The paper claims that this DRL-driven approach provides a near-deterministic pathway to generating cubic-phase states, a critical resource for universal CVQC. Key advantages highlighted include:

• Resource Efficiency: The method requires significantly less squeezing and photon-number resolution capability compared to previous proposals.
• Experimental Feasibility: The required components (squeezed light, displacements, and PNR measurements) are available in current experimental setups, unlike the strong nonlinearities required by alternative deterministic methods.
• Scalability: By avoiding post-selection, the method avoids the low success rates and optimization bottlenecks associated with searching over all possible detection patterns.
• Direct Quartic Gates: The paper establishes a foundational algorithm for generating quartic-phase gates directly, suggesting that a similar ML extension could eventually render this process near-deterministic, though this remains a work in progress requiring larger computational resources.

The authors conclude that while the quartic-phase extension is currently probabilistic and computationally intensive, the demonstrated success with cubic-phase states validates the potential of deep reinforcement learning for controlling complex quantum optical circuits to produce non-Gaussian resources efficiently.