💬 NLP

Rep2Text: Decoding Full Text from a Single LLM Token Representation

The paper introduces Rep2Text, a framework that decodes original input text from a single last-token representation of large language models using a trainable adapter, demonstrating that while token-level recovery decreases with sequence length, semantic coherence remains robust across various models and out-of-distribution data.

Original authors: Haiyan Zhao, Zirui He, Yiming Tang, Fan Yang, Ali Payani, Dianbo Liu, Mengnan Du

Published 2026-03-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haiyan Zhao, Zirui He, Yiming Tang, Fan Yang, Ali Payani, Dianbo Liu, Mengnan Du

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot (a Large Language Model, or LLM) that just finished reading a long story. Instead of keeping the whole story in its memory, the robot compresses the entire story into a single, tiny "thought bubble" at the very end. This thought bubble is the last-token representation.

For a long time, scientists thought this thought bubble was like a black box: once the story was compressed into it, the original words were gone forever, lost in the compression.

Rep2Text is a new invention that asks a bold question: "Is it possible to take that single, tiny thought bubble and magically expand it back into the original story?"

Here is how the paper explains this, using some fun analogies:

1. The Magic Translator (The Adapter)

Think of the robot that read the story as speaking "Robot Language," and the machine trying to write the story back as speaking "Human Language." They don't understand each other.

The authors built a translator (called an Adapter).

The Job: This translator looks at the robot's tiny thought bubble and says, "Ah, I see what this means! It's not just a number; it's a hint about a sentence."
The Process: The translator takes that single hint and passes it to a writer (a Decoding Model). The writer then tries to reconstruct the story, word by word, based only on that one hint.

2. The "Halfway" Miracle

The results were surprisingly good.

The Analogy: Imagine you are trying to describe a 16-page novel to a friend, but you are only allowed to whisper one single sentence to them.
The Result: The authors found that their translator could help the friend reconstruct about half of the original words correctly! Even better, the story the friend wrote still made sense. It wasn't gibberish; it kept the same topic and grammar, even if some specific names or details were slightly different.

3. The "Information Squeeze" (The Bottleneck)

The paper discovered a rule about how much information can fit in that tiny thought bubble.

Short Stories: If the original text was short (like a tweet), the translator could recover almost everything.
Long Novels: As the story got longer (32, 64, or more words), the "squeeze" got tighter. The translator started losing the specific details (like exact names or dates) but still remembered the main idea.
The Metaphor: It's like trying to fit a whole ocean into a teacup. You can't fit every drop of water (every specific word), but you can definitely tell that it's "water" and it's "wet" (the general meaning and topic).

4. Where is the Secret Hidden?

The researchers also looked at where in the robot's brain this information is stored.

Early Layers: Think of the robot's brain as having many floors. The lower floors are good at remembering the structure (like the skeleton of a sentence: "Subject, Verb, Object").
Middle Layers: The middle floors are best at remembering the specific words and names.
Top Layers: The top floors forget the specific words but hold onto the big picture (the theme or topic).
The Finding: To get the best reconstruction, you need to look at the middle floors, where the robot has a good mix of structure and specific details.

5. Why Does This Matter? (The Privacy Warning)

This is the most important part for regular people.

The Risk: If a robot can compress a secret message into a single thought bubble, and another machine can decode it back, then your privacy might not be as safe as you think.
The Test: The researchers tried this on medical notes (which are very private). Even though the robot had never seen those specific medical notes before, the translator could still guess the general patient condition and symptoms just from the compressed data.
The Conclusion: This proves that these "thought bubbles" leak information. If you think you are sending a secret message to a robot and it only stores the "last thought," you might be wrong. The secret is still there, just compressed.

Summary

Rep2Text is like a magic decoder ring for AI. It shows us that even when an AI compresses a whole conversation into a single number, it hasn't actually deleted the information. It's just hidden it in a way that, with the right key (the adapter), we can still read most of it. This is a wake-up call for privacy and a fascinating look at how these AI brains actually work.

1. Problem Statement

Large Language Models (LLMs) are often treated as "black boxes," and while their internal mechanisms are opaque, there is a growing concern regarding privacy and security. Specifically, the paper addresses the fundamental question: To what extent can the original input text be recovered from a single last-token representation in an LLM?

Unlike previous works that attempt to recover text from sequences of token embeddings or use iterative optimization with multiple embeddings, this work focuses on single-token inversion. The challenge is significant because the last-token representation is optimized for next-token prediction, acting as an information bottleneck that compresses the entire sequence history into a single vector. The goal is to determine if this compressed vector retains enough information to reconstruct the full original sequence without iterative searching or auxiliary hints.

2. Methodology: Rep2Text Framework

The authors propose Rep2Text, a framework designed to decode text from the last-token representation of a target model. The architecture is inspired by Large Vision-Language Models (LVLMs) like LLaVA and consists of two main components:

Trainable Adapter: A neural network module that bridges the representation space of the target model ( $M$ $M$ ) and the embedding space of a decoding model ( $M'$ $M^{'}$ ).
- Architecture: The adapter is a two-layer MLP with gated skip connections. It takes the residual stream representation of the last token ( $h_\ell$ ) from layer $\ell$ of the target model.
- Projection: It projects this single vector into the token embedding space of the decoding model. Crucially, it expands the single vector into $k$ token embeddings ( $X_e$ ), where $k$ is a hyperparameter (often set to match the original sequence length). This expansion allows the decoder to "spread" the compressed information across multiple positions.
- Mechanism: The projected embeddings are combined with system and user prompt embeddings and fed directly into the decoder's transformer layers (bypassing the decoder's own embedding layer).
Decoding Language Model: An autoregressive model (e.g., Llama, Qwen) that generates the inverted text sequence ( $\hat{S}$ ) based on the projected embeddings.

Training Strategy:
The system is trained using Teacher Forcing to maximize the log-likelihood of the ground-truth tokens. The training objective uses label smoothing to prevent overconfidence and improve generalization. Two training schemes are explored:

Adapter-only fine-tuning: Only the adapter parameters are updated; the decoder is frozen.
Joint fine-tuning: The adapter is fine-tuned, followed by joint fine-tuning of the adapter and the decoder (using LoRA).

3. Key Contributions

Novel Framework: Introduction of Rep2Text, the first framework to systematically decode full text from a single last-token representation without iterative search or in-context templates.
Quantitative Evidence of Recoverability: The study demonstrates that despite the information bottleneck, roughly 50% of the tokens in 16-token sequences can be recovered with strong semantic coherence.
Analysis of Information Bottlenecks: The paper provides a granular analysis of how information is preserved:
- Sequence Length: Recovery degrades as sequence length increases (from 8 to 64 tokens), but semantic/topic information remains robust even for longer sequences.
- Layer Depth: Structural and lexical information is best preserved in early-to-middle layers (around layer 10-15), while high-level semantic information becomes more prominent in later layers.
Generalization: The method shows strong generalization to Out-of-Distribution (OOD) data (specifically clinical notes), outperforming baseline methods significantly in this domain.

4. Experimental Results

The authors evaluated Rep2Text across various model combinations (Target: Llama-3.1-8B, Gemma-7B, Mistral-7B; Decoder: Llama-3.1-8B, Qwen-2.5-32B, etc.) using metrics like ROUGE, BERTScore, and LLM-as-a-Judge scores for structure, entities, and topics.

Token Recovery: On average, ROUGE-1 scores reached ~0.5 for 16-token sequences, indicating half the original tokens were recovered. ROUGE-2 (bi-grams) averaged around 0.24.
Semantic Coherence: Despite partial token recovery, Topic Scores and BERTScores remained high (often >0.75), indicating the reconstructed text maintained the original meaning and topic.
Model Sensitivity: Different target models exhibited varying invertibility. Mistral-7B-v0.1 showed the highest recoverability, suggesting its representations are richer but potentially more vulnerable to leakage.
Scaling Effects: Increasing the size of the decoding model (e.g., from 14B to 32B parameters) yielded only marginal improvements, suggesting that scaling effects are less pronounced in inversion tasks compared to generation tasks.
Comparison with Baselines: When compared to Vec2Text (a baseline using iterative correction):
- On in-distribution data (Wikipedia), Rep2Text performed comparably or better in token recovery.
- On OOD clinical data, Rep2Text significantly outperformed Vec2Text (Token F1: 0.26 vs. 0.06), demonstrating superior generalization and robustness.

5. Significance and Implications

Privacy and Security: The findings reveal a potential privacy vulnerability in LLMs. Even without access to intermediate layers or full sequences, a single last-token representation contains sufficient information for an adversary to reconstruct a significant portion of the input text, including sensitive entities and semantic context.
Interpretability: The work advances the understanding of how LLMs compress information. It confirms that while the last token acts as a bottleneck, it retains a "plastic" representation of the input that preserves both syntactic structure (early layers) and semantic intent (later layers).
Future Directions: The paper highlights that current models may not be sufficiently robust against representation inversion attacks, suggesting a need for new defense mechanisms or architectural changes to protect input privacy in LLM deployments.

In conclusion, Rep2Text proves that the "black box" nature of LLMs is not absolute; the compressed state of a single token can be effectively "unlocked" to reveal substantial details about the original input, posing significant implications for data privacy and model interpretability.