💬 NLP

Detecting Overflow in Compressed Token Representations for Retrieval-Augmented Generation

This paper addresses the challenge of information loss in compressed token representations for retrieval-augmented generation by defining "token overflow" and demonstrating that lightweight, query-aware probing classifiers can effectively detect this regime with high accuracy, enabling low-cost pre-processing to mitigate compression-induced errors.

Original authors: Julia Belikova, Danila Rozhevskii, Dennis Svirin, Konstantin Polev, Alexander Panchenko

Published 2026-02-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Julia Belikova, Danila Rozhevskii, Dennis Svirin, Konstantin Polev, Alexander Panchenko

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant but very busy librarian (the Large Language Model or LLM) who can answer any question if you give them the right books. However, the library is huge, and the librarian only has a tiny desk. If you try to shove 1,000 books onto that small desk, the librarian gets overwhelmed, drops some pages, and starts giving you wrong answers.

To fix this, researchers invented a "Magic Summarizer" (Soft Compression). Instead of bringing 1,000 books, you feed the books into the Magic Summarizer, which condenses them into just one tiny, magical note. The librarian then reads this single note and answers your question.

This works great most of the time. But sometimes, the note is too small. It's been crammed with so much information that it becomes a jumbled mess. The librarian reads the note, sees nothing but noise, and gives a wrong answer. The researchers call this "Token Overflow."

This paper is about building a security guard who can check that magical note before the librarian even reads it, to see if the note is too messy to be useful.

The Three Big Questions the Paper Asks

The researchers wanted to solve three mysteries:

What does a "broken" note look like? (Can we spot the mess just by looking at the note itself?)
Can we spot the mess quickly? (Do we need to ask the librarian to read it first, or can we tell just by looking at the note?)
Do we need to know the question? (Does the note look broken only for specific questions, or is it just broken in general?)

The Investigation: Trying Different Tools

The team tried three different ways to act as the security guard:

1. The "Physical Inspection" (Saturation Statistics)

The Analogy: Imagine holding the magical note up to the light. You check if the ink is too spread out, if the paper is too crumpled, or if the text is too dense.
The Result: This method was great at telling the difference between a "Magic Note" and a "Normal Book Page." It could easily say, "Hey, this is a compressed note!"
The Problem: It was terrible at telling if the note was useless. A note could look perfectly crumpled and dense but still contain the right answer. Or it could look clean but be empty. Verdict: Good for identifying the note type, bad for predicting if it will fail.

2. The "Librarian's Reaction" (Attention Patterns)

The Analogy: This method waits for the librarian to actually look at the note. It watches the librarian's eyes. If the librarian looks confused, squints, or looks away from the note, the guard knows something is wrong.
The Result: This worked okay, but it was slow because you had to let the librarian start reading first.
The Problem: By the time the librarian looks confused, you've already wasted time and energy. You wanted to stop the process before the librarian started.

3. The "Context Detective" (Learned Probing Classifiers)

The Analogy: This is the winning strategy. The guard doesn't just look at the note; they look at the Note AND the Question together.
- Example: If the note is about "Apples" and the question is "How to bake a cake," the guard immediately knows: "This note is useless for this question!"
- Even if the note looks perfect, if it doesn't match the question, the guard flags it as "Overflow."
The Result: This was the champion. By looking at the relationship between the question and the compressed note, the guard could predict with 72% accuracy whether the librarian would fail, all before the librarian even touched the note.

The Big Takeaways

You can't judge a book by its cover (or a note by its crumple): Just because a compressed token looks "dense" or "noisy" doesn't mean it's broken. You have to see if it matches the specific question being asked.
The damage happens before the reading: The moment the Magic Summarizer creates the note, the information is either saved or lost. Waiting for the librarian to read it is too late. You can detect the failure immediately after the note is created.
The "Security Guard" saves time: By using this new detector, we can stop bad notes from reaching the expensive, slow librarian. We can say, "This note is overflowed; let's fetch a fresh, uncompressed book instead," saving money and time.

Why This Matters

In the real world, AI models are getting bigger and smarter, but they are also getting more expensive to run. This paper gives us a cheap, fast way to check if our "compression" tricks are working or if they are about to make the AI hallucinate (make things up). It's like installing a smoke detector that goes off before the fire starts, rather than waiting for the house to burn down.

1. Problem Statement

Large Language Models (LLMs) face significant computational constraints when processing long contexts, particularly in Retrieval-Augmented Generation (RAG) systems where retrieved evidence must be compressed to fit within context windows. Soft compression architectures (e.g., xRAG) address this by mapping long token sequences into a small set of dense, learned "compressed tokens."

However, a critical failure mode exists: Token Overflow. This occurs when a compressed token is forced to encode more information than its fixed-dimensional capacity allows. In this state, the representation loses task-relevant signal and behaves like noise, silently degrading downstream performance (e.g., answering questions incorrectly).

The Gap: Current systems evaluate compression only via end-task metrics (e.g., accuracy), offering no insight into when or why a specific compressed token becomes "overflowed."
Research Questions:
1. How can we characterize overflow in compressed representations?
2. Can overflow be detected efficiently without full LLM inference?
3. Is overflow detectable from tokens alone, or does it require modeling query-context interactions?

2. Methodology

The authors propose a methodology to detect token overflow in the xRAG architecture (a projector-based soft compression system) using a spectrum of detection approaches ranging from query-agnostic to query-aware.

A. Problem Formalization

Overflow Definition: An instance $i$ is in an overflow state ( $O_i=1$ ) if the uncompressed baseline yields a correct answer, but the compressed version yields an incorrect one.
Goal: Predict $O_i$ using only the representations, without re-running the full generation task.

B. Detection Approaches

The study evaluates three tiers of detection complexity:

Query-Agnostic (Intrinsic Properties):
- Context Complexity: Measures like context length, perplexity, and statistical compressibility (gzip ratio).
- Token Saturation Statistics: Analyzing the geometric properties of compressed token vectors ( $c \in \mathbb{R}^d$ $c \in R^{d}$ ) and hidden states:
  - Hoyer's Sparsity: Measures energy concentration.
  - Spectral Entropy: Measures frequency distribution (white noise vs. structured signals).
  - Kurtosis: Measures tail heaviness (Gaussian vs. heavy-tailed).
Query-Conditioned (Behavioral Signals):
- Attention Features: Extracting attention weights during LLM generation to measure how much the model relies on compressed tokens versus other context. Includes mean attention, attention ratios, and attention entropy.
Query-Aware (Learned Probing):
- Joint Representations: Training lightweight classifiers on concatenated vectors of the Query and the Context (compressed tokens).
- Architectures: Linear Probes, MLP Probes, and MLPs with Supervised Contrastive Learning (SCL).
- Stages: Testing detection Pre-inference (immediately after projection, before LLM processing) vs. Post-inference (using hidden states from middle/last layers).

3. Key Contributions

Formalization of Token Overflow: Defined overflow as a regime where compressed representations lose sufficient task-relevant signal, distinct from general token compression.
Saturation Statistics as Identification Tools: Demonstrated that saturation statistics (sparsity, entropy, kurtosis) reliably distinguish compressed tokens from standard tokens (near-perfect linear separability, >0.95 AUC). However, these statistics fail to predict overflow (AUC ~0.55), proving that being "compressed" is not the same as being "overflowed."
Query-Aware Detection Superiority: Showed that overflow is not an intrinsic property of the token but emerges from the mismatch between the token's content and the specific query. Therefore, detection requires joint query-context modeling.
Efficient Pre-LLM Gating: Proved that overflow can be detected with high accuracy immediately after projection (Pre-inference), without requiring a full LLM forward pass.

4. Experimental Results

Experiments were conducted on SQuADv2, TriviaQA, and HotpotQA using the xRAG-7B model.

Detection Method	Average AUC-ROC	Key Finding
Saturation Statistics (Query-agnostic)	~0.53 - 0.58	Near-random performance. Good for identifying compressed tokens, useless for detecting overflow.
Context Complexity	~0.54 - 0.61	Minimal signal; length/perplexity alone cannot predict overflow.
Attention Features (Post-inference)	~0.62	Moderate signal but requires expensive LLM inference.
Learned Probing (Joint Reps)	0.72	Best Performance. Achieves strong detection pre-inference.

Pre-inference vs. Post-inference: Detection performance at the Pre-inference stage (using projected embeddings) is statistically identical to Post-inference (using LLM hidden states). This confirms that overflow is determined at the compression step, not during generation.
Query Importance: Joint models (Query + Context) outperform Context-only models by 5–8 percentage points, confirming that overflow is a relational property, not an intrinsic one.
Model Complexity: Simple Linear Probes were sufficient; complex MLPs offered no significant improvement, suggesting overflow manifests as a linearly separable structure in the joint representation space.

5. Significance and Implications

Low-Cost Safety Gating: The ability to detect overflow before the LLM processes the context allows for "pre-LLM gating." Systems can discard saturated representations or trigger adaptive chunking (retrieving more context) without wasting compute on a failed generation.
Theoretical Insight: The findings challenge the assumption that compression limits are purely capacity-based. Instead, they highlight that task-relevance is dynamic; a token may be sufficient for one query but overflowed for another.
Generalizability: While tested on xRAG, the methodology (probing joint representations) is architecture-agnostic and applicable to other soft compression paradigms.

Conclusion: The paper establishes that while compressed tokens have distinct geometric signatures, detecting when they fail (overflow) requires understanding the interaction between the compressed context and the specific query. This can be achieved efficiently using lightweight classifiers on pre-inference embeddings, enabling more robust and resource-efficient RAG systems.