🧬 biology

Reading Without a Reader: Large Language Models Collapse Reading and Writing into a Single Entangled Code

This paper demonstrates that unlike the human brain's dissociable reading and writing systems, large language models rely on a single, entangled autoregressive mechanism where input and output codes remain behaviorally coupled despite partial representational differentiation, positioning LLMs as a distinct type of mind shaped by cultural rather than evolutionary constraints.

Original authors: Diego Saldaña Ulloa

Published 2026-07-29

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Diego Saldaña Ulloa

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to understand how a human brain learns to read and write. For a long time, scientists thought these were two separate superpowers. Reading is like a decoder ring that turns squiggly letters into meaning, while writing is a different machine that takes those meanings and turns them back into letters. In the human brain, these two systems live in different neighborhoods. If you get a bump on one part of your head, you might lose the ability to read but still be able to write perfectly. If you get a bump on another part, you might forget how to write but still read like a book. They are distinct tools that happen to share a little bit of the same toolbox.

Now, imagine building a robot to do the same job. This robot doesn't have a brain that evolved over millions of years; it was built from scratch to learn from a massive library of books. The big question researchers have been asking is: Does this robot learn to read and write the same way a human does? Or does it take a shortcut, merging those two separate superpowers into one giant, tangled mess? This paper dives into that mystery by looking at the "math brains" of large language models (the kind of AI that writes stories and answers questions) to see if they have separate reading and writing parts, or if they are just doing everything with one single, entangled code.

The Paper: Reading Without a Reader

The authors of this paper decided to play detective with some of the smartest AI models in existence, like GPT-2, OPT, and a family of models called Pythia. They wanted to see if these machines treat reading and writing as two different jobs, or if they mash them together into one big, confused task.

To do this, they created a clever analogy. In a human brain, reading happens in the back (visual processing) and writing happens in the front (motor planning). In an AI model, there isn't a "back" or a "front" in the same way. Instead, the model has an Input Embedding (let's call it the "Reading Code") which turns a word into a number, and an Output Unembedding (the "Writing Code") which turns a number back into a word. The researchers asked: Are these two codes two different friends who hang out together, or are they actually the same person wearing two different hats?

They invented a score called the Entanglement Index (ranging from 0 to 1) to measure how similar these two codes are.

0 would mean they are totally different (like a human brain with separate reading and writing).
1 would mean they are identical (like a model where the input and output are forced to be the same).
Anything in between means they are mixed up, or "entangled."

The Big Discovery: One Tangled Code

The results were surprising. The human brain keeps reading and writing separate, but these AI models do not.

In the models where the researchers let the "Reading Code" and "Writing Code" learn separately (the "untied" models), the two codes didn't stay apart. Instead, they drifted toward each other and became coupled. The Entanglement Index settled between 0.23 and 0.35. This is far from zero (totally separate) and far from one (forced identical), but it proves they are definitely sharing a single, entangled statistical code. They aren't two different systems; they are one system doing double duty.

The researchers also watched how this happened over time. It wasn't instant. The codes started out totally different (like two strangers), then they got very close to each other (almost identical) around step 4,000 of training, and then they slowly pulled apart a little bit, but never fully separated. It's like two dancers who start on opposite sides of the room, dance right in the middle holding hands, and then slowly drift apart but still keep holding onto the same invisible string.

The Behavior Test: Reading Without a Reader

To make sure this wasn't just a math trick, the researchers tested how the models actually behaved. They checked if a model could understand a word (comprehension) without being able to write it (production), or vice versa. In humans, you can have "pure alexia" (can't read, can write) or "pure agraphia" (can read, can't write).

The AI models showed the opposite of this. In every single model they tested, if the model could write a word, it could also understand it. If it couldn't write it, it couldn't understand it either. They were perfectly locked together. There was no "reading without a writer" or "writing without a reader." The machine is doing both at the exact same time with the same internal gears.

What This Means for AI

The paper suggests that these AI models are not trying to copy the human brain. The human brain evolved to read and write using two separate, specialized systems. The AI, however, was built to optimize a single path: predicting the next word. Because of this, it collapsed reading and writing into one giant, efficient, but entangled package.

The authors are careful to say this isn't a "failure" of the AI. It's just a different way of being smart. It's like comparing a Swiss Army knife to a set of specialized tools. The human brain is the set of specialized tools (one for reading, one for writing). The AI is the Swiss Army knife: it does both jobs with the same blade, just in a different order.

The "Null" Results: What They Didn't Find

The paper is also very honest about what it didn't find, which is just as important.

No Bridge: They tried to see if the "math closeness" (the Entanglement Index) predicted how well the model behaved. It didn't. The two measurements didn't talk to each other.
No "Compression vs. Reasoning" Split: They wondered if the models got better at compressing text (making it smaller) while getting worse at understanding meaning. They found no evidence for this. In fact, as the models got better at compression, they seemed to get better at meaning too, though the data was too small to be 100% sure.
Not Just Decoder-Only: They checked other types of AI models (like T5 and BERT) that have separate "reading" and "writing" parts in their architecture. Even though those models have separate parts, they still behaved as if reading and writing were coupled. This suggests that the "tangled code" behavior is a general feature of how these language models work, not just a quirk of one specific design.

The Takeaway

This paper tells us that large language models are not miniature human brains. They are a distinct kind of mind. While humans evolved to keep reading and writing in separate rooms, these AI models built a single, open-concept studio where reading and writing happen in the same space, using the same tools. They achieve the same result—functional literacy—but they get there by taking a completely different route. The authors call this a "distinct point in the space of possible minds," suggesting that to understand AI, we shouldn't try to force it to look like us, but rather appreciate the unique, entangled way it thinks.

Technical Summary: Reading Without a Reader

Problem Statement

The paper addresses a fundamental structural difference between human literacy and the functional literacy of Large Language Models (LLMs). In the human brain, reading (decoding visual symbols) and writing (encoding meaning into orthographic output) are doubly dissociable systems: they rely on partially distinct neural pathways (ventral decoding vs. fronto-parietal encoding) that can be selectively impaired (pure alexia vs. pure agraphia) while sharing a partial orthographic core.

In contrast, decoder-only LLMs realize literacy through a single autoregressive path optimized on written text, a cultural invention rather than an evolved instinct. The central empirical question is: To what extent does this single mechanism entangle the "reading" and "writing" functions? Does the model collapse these functions into one statistical code, or does it develop dissociable pathways analogous to the human brain?

Methodology

Models

The study utilizes two families of models to isolate the architectural variable of "tied" vs. "untied" embeddings:

Tied Models (Entangled by construction): GPT-2 family and OPT. Here, the input embedding ( $W_E$ ) and output unembedding ( $W_U$ ) are forced to be identical ( $W_U = W_E^\top$ ). These serve as a "ceiling" reference.
Untied Models (Learned separately): The Pythia suite (14M–1.4B parameters). Here, $W_E$ and $W_U$ are independent parameter blocks, allowing the model to learn whether to couple or dissociate them.

Geometric Analysis: The Entanglement Index ( $E$ )

The authors define a composite Entanglement Index ( $E \in [0, 1]$ ) to quantify the similarity between the input "reading code" ( $W_E$ ) and output "writing code" ( $W_U$ ). $E$ is the unweighted mean of four normalized components:

Linear CKA: Centered Kernel Alignment between the row-space geometries.
Procrustes Residual: The residual after finding the optimal orthogonal map between $W_E$ and $W_U$ , converted to a similarity score.
Mean Per-Token Cosine: Average cosine similarity between corresponding token vectors.
Mutual k-NN Overlap: The fraction of nearest neighbors in $W_E$ that are also neighbors in $W_U$ .

Calibration:

Floor: Independent initialization (random matrices).
Null: Shuffled token correspondence (destroys alignment).
Ceiling: Tied models ( $E=1$ ).

Behavioral Analysis

The study contrasts comprehension (log-probability discrimination of correct forms) with production (greedy generation of forms).

Metric: A $2 \times 2$ contingency table is constructed. The "dissociation mass" (off-diagonal probability) is compared against a statistical independence baseline.
Hypothesis: If the model behaves like the human brain, comprehension and production should be dissociable (high off-diagonal mass). If coupled, the off-diagonal mass will be low.

Experimental Design

The paper reports on nine probes:

Exps 1–6: Replication of established findings (induction heads, orthographic acquisition, data efficiency, form-before-meaning, compression scaling, memorization).
Exp 7: The primary read/write analysis (Entanglement Index $E$ across architecture, scale, and training trajectory).
Exp 8: Behavioral read/write dissociation and the "bridge" (correlating $E$ with behavioral coupling $\phi$ ).
Exp 9: Architecture contrast (Decoder-only vs. Encoder-Decoder vs. Masked Encoder) to test if coupling is specific to the decoder-only forward path.

Key Results

1. Geometric Entanglement (Weights)

Coupling: Untied Pythia models exhibit a significant entanglement index ( $E \approx 0.23–0.35$ ), far above the independent initialization floor ( $E \approx 0.003–0.04$ ) but well below the tied ceiling ( $E=1$ ).
Trajectory: The training trajectory is non-monotonic ("couple-then-differentiate"). Codes couple early (peaking at CKA $\approx 0.49$ at step 4000) and then partially differentiate, settling at $E \approx 0.21$ by the final step.
Asymmetric Drift: The writing code ( $W_U$ ) drifts approximately 3.2 times farther from initialization than the reading code ( $W_E$ ) across all frequency deciles. This replicates and extends findings by Lopardo et al. (2026), ruling out gradient exposure as the sole cause.
Scale Trend: No systematic trend in $E$ across model sizes (flat, underpowered).

2. Behavioral Coupling

Positive Coupling: In all 12 non-degenerate models, comprehension and production are positively coupled. The off-diagonal "dissociation mass" is significantly below the statistical independence baseline (exact sign test $p < 0.001$ ).
Contrast with Brain: This is the opposite of the human brain's double dissociation (where lesions can abolish one function while sparing the other).
Comprehension > Production: Comprehension accuracy consistently exceeds production accuracy, though the authors note this may partly reflect task difficulty asymmetry.

3. Architecture Generality

Not Decoder-Only Specific: Encoder-decoder models (T5) and masked encoders (RoBERTa) also exhibit behavioral coupling, despite T5 models showing high representational separation (up to $1 - \text{CKA} = 0.96$ ) between their encoder and decoder stacks.
Conclusion: Behavioral coupling is a general property of current language models, not a unique artifact of the decoder-only forward path.

4. Null Results (Honest Accounting)

The paper explicitly reports several null findings:

Geometry $\to$ Behavior Bridge: There is no correlation between the weight-level entanglement ( $E$ ) and behavioral coupling ( $\phi$ ) across models ( $\rho = 0.00$ ) or within training ( $\rho = 0.08$ ). The bridge is untestable at early stages because production is undefined before the geometry plateaus.
Compression vs. Reasoning: No evidence that text compression decouples from functional competence; if anything, they co-vary positively.
Induction Circuit: Ablating the top induction head does not collapse the orthographic word-form gap (fraction remaining $\approx 1.009$ ), indicating orthographic decoding is not carried by the induction circuit.

Significance and Claims