⚛️ quantum physics

Reinforcement Learning to Disentangle Multiqubit Quantum States from Partial Observations

This paper presents a deep reinforcement learning approach using a permutation-equivariant transformer architecture that, relying solely on local two-qubit reduced density matrices, autonomously constructs efficient and noise-resilient disentangling circuits for multiqubit states on NISQ devices, achieving optimal gate counts for arbitrary 4-qubit state preparation.

Original authors: Pavel Tashev, Stefan Petrov, Matthew T. Diaz, Friederike Metz, Alaina M. Green, Norbert M. Linke, Marin Bukov

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Pavel Tashev, Stefan Petrov, Matthew T. Diaz, Friederike Metz, Alaina M. Green, Norbert M. Linke, Marin Bukov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Untangling a Quantum Knot with a "Smart Assistant"

Imagine you have a giant, complex knot made of 16 strings (representing quantum particles called qubits). Your goal is to untangle this knot until every string is lying perfectly straight and separate from the others. In the quantum world, this is called disentangling.

The problem is that you can't see the whole knot at once. You can only peek at small sections where two strings cross each other. This is like trying to untangle a ball of yarn while wearing blinders that only let you see two strands at a time.

The authors of this paper created a smart computer assistant (using a type of Artificial Intelligence called Reinforcement Learning) that can look at these small, local views and figure out the best way to untangle the whole mess, step by step.

How the "Smart Assistant" Works

Think of the AI agent as a master puzzle solver playing a game.

The View (Partial Information): The AI cannot see the entire quantum state (the whole knot). It only gets to look at two-qubit reduced density matrices. In plain English, this means it can only measure how any pair of strings is interacting with each other. It's like looking at a map where you only see the traffic between two specific intersections, not the whole city.
The Move (The Action): Based on what it sees, the AI decides which two strings to grab and twist (apply a quantum gate). It doesn't just guess; it uses a "brain" (a neural network) that has been trained to recognize patterns.
The Reward: Every time the AI makes a move that makes the knot slightly looser (reduces entanglement), it gets a "point." If it makes a move that doesn't help, it gets no points. The goal is to get the knot completely straight using the fewest moves possible.

The "Magic" Brain: The Transformer

The paper mentions a specific type of AI architecture called a permutation-equivariant transformer. Here is a simple way to understand that:

Imagine you have a team of workers untangling a knot. If you swap the names of the workers (Worker A becomes Worker B, and vice versa), the team should still know exactly what to do because the structure of the knot hasn't changed, only the labels.

The AI in this paper is built so that it doesn't care about the "names" or "labels" of the qubits. If you shuffle the qubits around, the AI instantly recognizes the new arrangement and adjusts its plan accordingly. It learns the shape of the entanglement, not just the specific order of the strings.

What They Discovered

The researchers tested this AI on knots of different sizes (from 4 strings up to 16 strings). Here is what they found:

Beating the "Random" and "Greedy" Players:
- A Random Player just picks two strings to twist without thinking. It takes forever to untangle the knot.
- A Greedy Player looks at all possible pairs, picks the one that helps right now, and moves on. This is better, but it often gets stuck in a local trap.
- The AI Agent is smarter. It sometimes makes a move that doesn't look helpful immediately, but sets up a better position for later. It learns to see the "big picture" of the knot's structure.
- Result: The AI used significantly fewer moves (gates) than the other methods. For example, on a 6-string knot, it used about 56 moves, while the greedy player needed 72.
The 4-String Secret:
For a 4-string knot, the AI discovered a universal recipe. It found a specific sequence of moves (using at most 5 twists) that can untangle any 4-string knot, no matter how messy it starts. This is a major discovery because it proves you can prepare any 4-qubit state with very few resources.
Working in the Real World (Noisy Hardware):
Real quantum computers are "noisy"—they are like a room with a loud fan blowing, making it hard to hear instructions. The AI was trained in a perfect, quiet simulation. When the researchers put this AI on a real quantum computer (a trapped-ion machine), it still worked!
- Even though the measurements were "fuzzy" due to noise, the AI was robust enough to figure out the right moves and successfully untangle the state.

Why This Matters (According to the Paper)

The paper claims this is a breakthrough for NISQ devices (current, imperfect quantum computers). Because the AI only needs to look at pairs of qubits (local information) rather than the whole system, it can run on real hardware without needing impossible amounts of data.

It essentially acts as a compression tool. Just as you can zip a large file into a smaller one, this AI can take a complex, messy quantum state and "zip" it down into a simple, clean state using the minimum number of steps. Conversely, if you run the process backward, it can "unzip" a simple state into a complex, useful one.

Summary

The paper presents a "smart assistant" that learns to untangle complex quantum knots by only looking at small pieces of the puzzle. It is smarter than random guessing or simple step-by-step logic, it adapts to the shape of the knot regardless of how the pieces are labeled, and it works surprisingly well even on real, noisy quantum machines.

Technical Summary: Reinforcement Learning to Disentangle Multiqubit Quantum States from Partial Observations

Problem Statement
The paper addresses the challenge of disentangling arbitrary multiqubit quantum states into product states using a minimal number of two-qubit gates. This is the inverse problem of quantum state preparation and is crucial for quantum state compression, initialization, and complexity analysis. A primary difficulty lies in the exponential size of the Hilbert space, which renders exhaustive search algorithms infeasible for systems larger than a few qubits. Furthermore, practical constraints on Noisy Intermediate-Scale Quantum (NISQ) devices limit access to global state information; typically, only local measurements are feasible. The authors investigate whether partial knowledge of a quantum state—specifically, access only to two-qubit reduced density matrices (RDMs)—is sufficient to construct optimal disentangling protocols.

Methodology
The authors propose a Deep Reinforcement Learning (RL) framework utilizing an Actor-Critic (AC) algorithm, specifically a variant of Proximal Policy Optimization (PPO) with entropy regularization.

Observation Space: To comply with NISQ constraints, the agent does not observe the full quantum state. Instead, at each time step $t$ , the agent receives an observation $o_t$ consisting of all symmetrized two-qubit reduced density matrices of the current state. This reduces the information requirement from exponential to quadratic in the number of qubits ( $L$ ).
Action Space: The agent selects a pair of qubits $(i, j)$ on which to apply a two-qubit unitary gate.
Gate Construction: Once a qubit pair is selected, the specific unitary gate $U^{(i,j)}$ is determined analytically by diagonalizing the corresponding two-qubit RDM. This ensures the gate is locally optimal for disentangling that specific pair given the current partial information.
Network Architecture: The policy network employs a permutation-equivariant Transformer architecture. This design allows the agent to learn policies that are invariant to qubit labeling, enabling it to recognize entanglement structures regardless of how qubits are permuted in the input state.
Reward Function: The reward signal is designed to maximize the reduction in average single-qubit von Neumann entropy ( $S_{avg}$ ) while penalizing the number of gates taken. The goal is to drive $S_{avg}$ to zero (indicating a product state) with the fewest steps.
Training Regime: For larger systems (12 and 16 qubits), the authors employ a two-stage training process. First, the agent is trained on product states of small subsystems to learn basic disentangling. Second, it is fine-tuned on weakly entangled subsystems of Haar-random states.

Key Results

Efficiency on Small Systems: The trained agent successfully disentangles 4-, 5-, and 6-qubit Haar-random states. For arbitrary 4-qubit states, the agent identifies a protocol using at most five two-qubit gates (equivalent to 10 CNOT gates). For 5- and 6-qubit Haar-random states, the agent requires an average of 20 and 56 two-qubit gates, respectively.
Comparison to Baselines: The RL agent significantly outperforms both a "random" agent (selecting qubit pairs uniformly) and a "greedy" agent (selecting the pair that minimizes $S_{avg}$ locally at each step). For 6-qubit states, the RL agent reduces the gate count by nearly a factor of three compared to the random agent and by approximately 22% compared to the greedy agent.
Circuit Transpilation: When compared to the deterministic algorithm by Shende et al. (implemented in Qiskit), the RL agent, acting as a pre-transpilation routine, drastically reduces the number of CNOT gates required. For example, for a 4-qubit Haar-random state, the CNOT count was reduced from 22 to 12.
Pattern Recognition: Analysis of the learned protocols reveals strong temporal and spatial correlations. The agent often identifies a specific qubit and applies a sequence of gates involving it before switching to another, effectively learning to exploit the specific entanglement structure of the input state rather than applying a generic sequence.
Scalability and Noise Resilience:
- Scaling: While the underlying quantum simulation scales exponentially, the RL agent's computational cost scales quadratically with the number of qubits.
- Noise: The agent demonstrates robustness to shot noise (statistical errors from finite sampling in state tomography) and depolarizing noise. Even when trained in a noise-free environment, the agent successfully disentangles states on noisy simulators and real hardware, provided the state is not fully mixed.
Experimental Validation: The approach was experimentally validated on a trapped-ion quantum computer. The agent successfully disentangled Bell-Bell, GHZ, W, and shallow circuit states. While residual entropy remained due to hardware decoherence, the entanglement of formation between system qubits was successfully reduced to near zero.

Significance and Claims
The paper claims that using partial information (two-qubit RDMs) via a state-aware RL agent offers a practical advantage over deterministic, state-independent algorithms for disentangling quantum states. The authors emphasize that:

State-Dependence: Unlike universal disentangling machines (which do not exist), the RL agent constructs state-dependent protocols that adapt to the specific entanglement distribution, thereby minimizing resource usage.
NISQ Applicability: The reliance on local measurements makes the method directly applicable to current quantum hardware without requiring full state tomography.
Generalization: The agent exhibits generalization capabilities, successfully handling 12- and 16-qubit systems with weakly entangled subsystems, and transferring learned policies to noisy environments without retraining.
Optimal 4-Qubit Protocol: The analysis of optimal 4-qubit sequences leads to a general circuit capable of preparing any arbitrary 4-qubit state from a product state using at most five two-qubit unitaries (10 CNOTs).

The authors conclude that while the exponential scaling of resources for Haar-random states is an inherent feature of quantum mechanics, RL provides a mechanism to optimize the constant prefactor and discover efficient protocols for physically relevant classes of states, such as those with area-law entanglement or weak inter-subsystem entanglement.