⚛️ quantum physics

Trainable Quantum-Native Language Models: Dissipative Memory, Capacity Limits, and Hardware Validation

This paper presents trainable quantum-native autoregressive language models that utilize dissipative dynamics to achieve memory horizons beyond a single token and demonstrate successful hardware execution on IBM devices, though they currently lack computational advantage over classical baselines and exhibit capacity limits tied to register width.

Original authors: Deniz Yunus Göğüş

Published 2026-10-12

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Deniz Yunus Göğüş

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the quest to build machines that understand language, scientists have long wondered if the strange rules of the quantum world could offer a new kind of thinking machine. Classical computers, the ones we use every day, store information as bits that are either zero or one. Quantum computers, by contrast, use quantum bits, or qubits, which can exist in a delicate blend of states simultaneously. This property allows them to hold complex information in a way that is fundamentally different from classical memory. However, for a machine to learn and adapt, it must be able to forget. Just as a human brain discards irrelevant details to focus on the present, a learning system needs to let go of old data to make room for new inputs. In the quantum realm, a strict rule of physics often prevents this: if a system evolves without losing energy or interacting with its environment, the information it holds cannot be erased or compressed; it can only be rearranged. This creates a paradox for building a quantum brain: the very laws that give quantum computers their power also seem to lock them into a state where they cannot forget, and without forgetting, they cannot learn effectively.

A researcher recently set out to test whether a language-learning machine could be built entirely from these quantum parts, and if so, what rules it would need to follow to succeed. They did not just simulate this on a standard computer; they built the model, trained it, and ran it on a real quantum processor to see if the physics held up. Their work reveals a clear path forward for this technology, but also a hard ceiling on how much it can currently remember. The researcher found that a purely quantum memory, which tries to keep all information perfectly intact, fails to learn language. It gets stuck carrying old, useless data because it cannot throw it away. To fix this, they had to introduce a mechanism that allows the system to deliberately lose information, a process known as dissipation. By teaching the quantum system to forget in a controlled way, they created a model that could remember useful information for a few steps. However, this memory is incredibly fragile and limited. Even with the best techniques, the system could only hold onto information for about two steps before the noise of the quantum world and the limits of its size caused it to fail.

The experiment began with a simple question: could a machine that stores its thoughts as a quantum state learn to predict the next letter in a sentence? The researcher designed a system where the "hidden state"—the part of the brain that holds context—was a small register of quantum bits. As the machine read each letter, it would update this quantum state. In their first attempt, they used a method that kept the quantum state perfectly isolated, ensuring no information was lost. This is like trying to carry a full glass of water without spilling a drop; you can move it around, but you cannot pour any out. The results were disappointing. The machine performed worse than a much simpler version that only looked at the very last letter. The reason was that the machine was forced to carry every single letter it had ever seen, cluttering its memory with irrelevant history. It could not learn to ignore the past, and without that ability, it could not predict the future.

To solve this, the researcher changed the rules. They introduced a "dissipation" channel, a way for the quantum system to interact with its environment and deliberately discard old information. They made this forgetting process dependent on the current letter being read, allowing the machine to decide what to keep and what to drop based on the context. This approach worked. The new model, which could now forget, significantly outperformed the version that tried to remember everything. It learned to hold onto information for more than just one step, showing that the ability to forget is not a bug but a necessary feature for a learning system. However, the improvement had a sharp limit. Even with this new ability to forget, the model could never learn to look back further than two letters. When the researcher tested it on longer sentences, the model's performance plateaued, matching a simple system that only looked at the previous two letters. It could not go deeper.

The researcher then investigated why this limit existed. They ran a series of tests where they asked the machine to remember pairs of keys and values, similar to looking up a word in a dictionary. They increased the size of the quantum memory from four to seven qubits. With four qubits, the machine could reliably remember one pair. With six or seven qubits, it could remember two pairs. But when they asked it to remember three pairs, it failed every time, regardless of how many qubits they used. This suggests that the amount of information a small quantum memory can hold is strictly limited by the number of qubits available. The researcher noted that this limit aligns with a fundamental law of physics called the Holevo bound, which dictates how much classical information can be extracted from a quantum system. While they did not prove that this law was the sole cause of the failure, the evidence strongly suggested that the machine was running out of space to store the necessary details.

To ensure their findings were not just a result of computer simulations, the researcher took their model to a real quantum computer, specifically a 156-qubit processor from IBM. They translated their mathematical model into a physical circuit that could run on the machine's hardware. They tested the system step by step, checking if the quantum state evolved as predicted and if the probabilities of the next letter matched the simulation. The results were a success in terms of physics: the machine ran exactly as the theory predicted, and the measured probabilities of the next letter were close to the expected values. The system generated text byte by byte, with every decision based on a physical measurement of the quantum state. However, the text it produced was not fluent or meaningful. It was weak and lacked the quality of even the simplest classical language models. The researcher was clear that they were not claiming a breakthrough in language quality or a speed advantage over classical computers. Instead, they were proving that a specific type of quantum architecture—one that learns to forget—can be built and run on real hardware.

The study concludes that while the dream of a quantum-native language model is physically possible, the current technology is far from ready to compete with classical systems. The researcher demonstrated that a quantum memory must be able to lose information to learn, but that this memory is severely constrained by its size. They found that with the small number of qubits available today, the system hits a wall after just two steps of memory. To build a machine that can understand longer sentences, they suggest that we will need much larger quantum memories, likely requiring error-corrected systems that do not yet exist. The work serves as a crucial reality check for the field, showing exactly what is needed to move forward: not just more power, but a deeper understanding of how quantum systems can be taught to forget. The path to a quantum language model is open, but the road is long, and the first steps have shown us exactly where the obstacles lie.

Technical Summary: Trainable Quantum-Native Language Models

Problem Statement
The paper investigates whether a quantum system can serve as the native substrate for a recurrent language model, where the hidden state, state transitions, and output sampling are entirely quantum, with classical computation restricted to optimization and control. The central challenge addressed is the "forgetting" mechanism: language modeling requires the model to discard irrelevant past information (many-to-one contraction) to maintain a useful memory state. The author hypothesizes that purely unitary (closed) quantum dynamics, which preserve information and trace distance, may be structurally incapable of learning this necessary forgetting, thereby limiting the utility of quantum memory in autoregressive tasks.

Methodology
The study employs a fixed-width recurrent architecture with a hidden state $\rho_t$ defined as a density matrix on $n$ memory qubits. The model processes byte-level inputs ( $x_t \in \{0, \dots, 255\}$ ) and outputs the next-byte distribution via a 256-outcome Positive Operator-Valued Measure (POVM) based on the Born rule ( $p(y) = \text{Tr}(M_y \rho_t)$ ), without any classical vocabulary projection or softmax layer.

Three architectural variants were compared:

UMW (Unitary Memory): State transitions are unitary conjugations ( $W_x \rho W_x^\dagger$ ).
UMW-D (Unitary + Fixed Dissipation): A unitary write followed by a token-independent, learned dissipation channel (coupling to a work qubit, resetting, and discarding).
UMW-G (Gated Channel): A token-conditioned, learned Completely Positive Trace-Preserving (CPTP) map where writing and forgetting are jointly learned for each input token.

Experimental Protocol

Controls: To ensure fair comparison, the author used retrained, parameter-count-matched controls: a "one-token control" ( $h_1$ ) that resets memory at every step, and a "two-token control" ( $h_2$ ) that resets based on the previous two tokens. All models were trained from scratch on the byte-level corpus of Alice's Adventures in Wonderland.
Pre-registration: The study followed a falsification-first protocol with pre-registered hypotheses and pass/fail bars to ensure negative results were reported with the same rigor as positive ones.
Hardware Validation: Models compiled to hardware-shaped circuits were validated on an IBM Heron processor (156 qubits) using a three-level validation process: memory-register distribution, next-bit probabilities, and end-to-end autoregressive generation.

Key Results

Failure of Unitary Recurrence: Purely unitary models (UMW) failed to learn useful forgetting. They consistently underperformed their retrained one-token controls (e.g., 2.5167 vs. 2.3705 nats/byte at 4 qubits). The unitary dynamics preserved distinctions between states that should have been forgotten, leading to a "structural inability" to implement the adaptive forgetting required for language modeling.
Success of Dissipative Gating: Introducing learned dissipation (UMW-D) restored memory utility, outperforming the one-token control by $+0.10 \pm 0.008$ nats. Making the dissipation token-conditioned (UMW-G) further improved performance, doubling the beyond-one-token margin to $+0.20 \pm 0.03$ nats at context length 12.
Capacity Limits: Despite improvements, the models hit a hard ceiling. The UMW-G architecture never surpassed the retrained two-token control ( $h_2$ ) across contexts of 12, 24, or 48 tokens. On a key–value recall benchmark, the maximum solvable task shifted from one stored pair (at 4–5 qubits) to two pairs (at 6–7 qubits), with three pairs remaining unsolved. This "two-token horizon" is consistent with information-theoretic bounds (Holevo and Nayak bounds) but not proven to be caused by them.
Secondary Entropy Result: On a synthetic "perturbed coin" process, one-qubit UMW-G models achieved lower stationary memory entropy than classical predictors with matched prediction error (median margin 0.04 nats), demonstrating that gradient training can reach the regime of quantum statistical complexity predicted by analytic constructions.
Hardware Validation: The compiled model successfully reproduced its predicted computational-basis memory distribution on the IBM Heron processor across a 12-token context. Next-bit probabilities passed accuracy bars at 4-token prefixes, and the system generated bytes autoregressively using physically measured probabilities, validating the full stack from simulation to hardware execution.

Significance and Claims
The paper explicitly does not claim computational quantum advantage. All models studied are classically simulable at the tested widths (4–8 qubits), and none approach the quality of classical 4-gram models.

Instead, the work establishes three architectural and physical findings:

Dynamics Requirement: Trainable quantum-native recurrent models require controlled non-unitary (dissipative) memory dynamics to learn forgetting; purely unitary recurrence is insufficient for this task.
Capacity Constraints: Small quantum memories are sharply capacity-limited, exhibiting a persistent "two-token horizon" that aligns with accessible-information bounds.
Feasibility: A complete, trainable quantum-native recurrent model can be compiled and run end-to-end on present-day NISQ hardware, with its statistical outputs validated against simulation.

The author concludes that while current hardware and qubit counts are insufficient for competitive language modeling, the study defines the necessary dynamics for such models and provides a methodology for measuring their memory capacity and validating them on physical devices.

Technical Summary: Trainable Quantum-Native Language Models

More like this