💬 NLP

Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe

This paper introduces an Encoding Probe that reconstructs language model representations from interpretable features, offering a complementary perspective to traditional decoding probes by enabling direct feature comparison and revealing how different linguistic and acoustic attributes contribute to internal model states.

Original authors: Gaofei Shen, Martijn Bentum, Tom Lentz, Afra Alishahi, Grzegorz Chrupała

Published 2026-05-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gaofei Shen, Martijn Bentum, Tom Lentz, Afra Alishahi, Grzegorz Chrupała

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot that reads books and listens to stories. You want to know what's happening inside its brain. For a long time, scientists used a method called a "Decoding Probe."

Think of the Decoding Probe like a detective interrogating a suspect. The detective looks at the robot's brain activity and asks, "Can you tell me who the speaker is?" or "Can you tell me what part of speech this word is?" If the detective gets the right answer, they say, "Aha! The robot knows this!"

But the authors of this paper say this detective method has two big flaws:

The "Score Comparison" Problem: Imagine the detective asks two questions.
- Question A: "Who is speaking?" (The robot gets this right 95% of the time).
- Question B: "What is the sound of this letter?" (The robot gets this right 58% of the time).
- The old method says, "The robot knows the speaker better!" But wait! Maybe Question A was just an easier test to begin with. Maybe there are only 10 speakers but 100 different sounds. You can't compare the scores directly because the tests weren't fair. It's like comparing a runner's time in a 100-meter sprint to a marathon; the numbers don't tell you who is "more important" to the runner's overall fitness.
The "Copycat" Problem: Imagine the detective asks, "What is the grammar of this sentence?" The robot answers correctly 85% of the time. But what if the robot isn't actually thinking about grammar? What if it's just recognizing the words? Since certain words almost always appear with certain grammar (like "the" is usually a noun), the robot might be cheating by just looking at the word. The detective thinks the robot understands grammar, but it's just a lucky guess based on the vocabulary.

The New Solution: The "Encoding Probe"

To fix this, the authors invented a new tool called the Encoding Probe. Instead of a detective interrogating the robot, imagine you are a chef trying to recreate a secret soup.

The Old Way (Decoding): You taste the soup and guess, "Is there salt in here?" "Is there pepper?"
The New Way (Encoding): You take a list of ingredients (Speaker, Sound, Grammar, Words) and try to rebuild the soup from scratch using a recipe. You mix the ingredients together and see how close your new soup tastes to the original robot's "soup" (its brain activity).

Here is how the Encoding Probe solves the problems:

Fair Comparisons: Now, instead of guessing scores, you measure how much of the soup's flavor is missing if you leave out an ingredient.
- If you leave out "Speaker" and the soup tastes almost the same, the Speaker ingredient wasn't very important.
- If you leave out "Sound" and the soup tastes terrible, the Sound ingredient was crucial.
- This gives you a direct, fair comparison of how much each ingredient contributes to the final dish.
Stopping the Copycat: Because you are rebuilding the soup with all the ingredients at once, you can see what happens if you remove just one.
- If you remove "Grammar" but the soup still tastes perfect because "Words" are still there, you know the robot was just relying on the words, not the grammar.
- If you remove "Grammar" and the soup gets worse even though the words are still there, you know the robot actually does use grammar independently.

What They Found

The authors tested this new "soup recipe" method on robots that read text and robots that listen to speech.

The Speaker Test: They looked at whether the robots cared about who was speaking or what they were saying.
- The Result: In a robot trained to identify speakers, the "Speaker" ingredient became the main flavor of the soup. But in a robot trained to recognize speech sounds, the "Speaker" ingredient was barely noticeable, even though the robot could still guess the speaker if asked (the old detective method). The new method showed that for the speech robot, the sound of the words mattered way more than who said them.
The Grammar Test: They looked at whether the robots cared about sentence structure (grammar) or just vocabulary (words).
- The Result: Both text and speech robots used words and grammar to build their understanding. However, the "Words" ingredient was the biggest flavor contributor. But crucially, even after accounting for the words, the "Grammar" ingredient still added a unique flavor. This proved the robots do understand grammar, not just the words.

The Big Takeaway

The paper concludes that the old "Detective" method (Decoding) is good at telling you if a robot has a piece of information. But the new "Chef" method (Encoding) is much better at telling you how important that information is compared to everything else, and whether the robot is truly using it or just copying it from something else. It gives a clearer, fairer picture of what's really cooking inside the robot's brain.

1. Problem Statement

The paper addresses critical limitations in the standard Decoding Probe approach used to interpret Large Language Models (LLMs) and speech models. While decoding probes (mapping model representations $X$ to linguistic properties $Y$ ) determine if information is present, they suffer from two major flaws:

Incomparable Feature Contributions: Decoding accuracy scores (e.g., 95% for speaker ID vs. 58% for phonetics) cannot be directly compared because they depend on task-specific factors like the number of classes and label distribution. This prevents quantifying the relative importance of different features within the representation.
Feature Correlation Artifacts: High correlations between features (e.g., word identity and grammatical category) can lead to misleading conclusions. A probe might successfully decode a grammatical category simply because it correlates with specific words, rather than because the model explicitly encodes syntactic structure.

The authors propose a paradigm shift: moving from Decoding ( $f: X \to Y$ ) to Encoding ( $g: Y \to X$ ).

2. Methodology: The Encoding Probe

The core contribution is the Encoding Probe, which reverses the direction of analysis. Instead of predicting a label from a representation, it attempts to reconstruct the model's internal representations ( $X$ ) using interpretable linguistic features ( $Y$ ).

Formulation: A multivariate regression model (specifically Ridge Regression) is trained to predict the hidden state activations of a target model based on a set of interpretable features.
Feature Sets: The study utilizes diverse feature sets derived from audio waveforms and text, including:
- Acoustics: Pitch, formants, MFCCs.
- Phonetics: Phonetic posteriorgrams (PPGs).
- Speaker Identity: One-hot encoded speaker IDs.
- Syntax: POS tags, dependency labels, tree depth, and position.
- Lexicon: Static word embeddings (FastText).
Evaluation Metric: The performance is measured by Unexplained Variance (UV), defined as $1 - R^2$ . A lower UV indicates a better reconstruction.
Ablation Strategy: To quantify relative contributions and control for correlations, the authors perform feature ablation. They compare the UV of a "Full" model (all features) against models where specific features are removed (e.g., $Full \setminus Syntax$ $F u l l ∖ S y n t a x$ ).
- The increase in UV upon removing a feature directly measures that feature's unique contribution to the representation, independent of correlated features.
Target Models: The method is applied to:
- Speech: Wav2vec2 (Base, ASR-tuned, and Speaker-ID tuned).
- Text: BERT (Base).

3. Key Contributions

Methodological Innovation: Introduces the Encoding Probe as a complementary tool to decoding probes, leveraging standard regression analysis to solve the problems of feature comparability and correlation.
Quantitative Attribution: Provides a rigorous framework to measure the relative contribution of different features to model representations, rather than just their decodability.
Controlled Analysis: Demonstrates how to disentangle correlated features (e.g., syntax vs. lexicon, speaker vs. acoustics) by observing reconstruction error changes when features are removed simultaneously.
Empirical Insights: Offers new perspectives on how different training objectives (Self-supervised, ASR, Speaker-ID) shape the internal representations of transformer models.

4. Key Results

Experiment 1: Speaker Identity in Speech Models

Decoding vs. Encoding Divergence: While speaker identity is decodable from all Wav2vec2 variants, its contribution to reconstruction varies drastically based on the training objective.
- Base & ASR Models: Acoustic and phonetic features contribute significantly more to the representation than speaker identity. Removing speaker identity causes a minimal increase in reconstruction error.
- SID-Tuned Model: In the upper layers of the Speaker-ID tuned model, the representation is almost exclusively dedicated to encoding speaker identity. Removing speaker identity causes a massive spike in reconstruction error (UV $\approx$ 1), while removing phonetics has almost no effect.
Conclusion: Decoding probes show speaker info is present, but encoding probes reveal it is only dominant in representations when explicitly trained for that task.

Experiment 2: Syntax vs. Lexicon in Text and Speech

Independence of Features: Both lexical and syntactic features make independent contributions to the reconstruction of model representations.
- Ablating syntax increases reconstruction error even when lexical features are present, disproving the hypothesis that syntactic decodability is merely a byproduct of lexical correlations.
Relative Contribution: Lexical features consistently account for a larger fraction of the variance (higher contribution) than syntactic features across all models.
Modality Differences: The contribution of syntax and lexicon is more prominent in BERT (text) than in speech models.
Effect of ASR Tuning: Fine-tuning for ASR makes representations harder to reconstruct in absolute terms but increases the relative contribution of lexical and syntactic features compared to the base model.

5. Significance and Implications

Beyond "Is it there?": The paper argues that knowing a feature is decodable is insufficient. Researchers must understand how much of the model's internal state is actually dedicated to that feature.
Robust Interpretability: By controlling for correlations, the Encoding Probe prevents false positives where a feature appears encoded only because it correlates with another, dominant feature.
Training Objective Insights: The study highlights that the "nature" of a model's representation is highly dependent on its training objective. For instance, ASR tuning normalizes speaker information, while SID tuning isolates it, a nuance missed by standard decoding probes.
Statistical Rigor: The approach bridges the gap between NLP interpretability and established statistical methods (regression analysis), offering a more principled way to analyze neural representations.

Limitations Noted

The authors acknowledge that the Encoding Probe remains observational (not causal), relies on linear regression (potentially missing non-linear relationships), and is constrained by the specific feature sets chosen for operationalization. Additionally, results are based on read audiobooks and may differ for spontaneous speech.