💻 computer science

SEER: Long-Context Reasoning via Selective Visual-Text Compression

SEER is a novel framework that enhances long-context reasoning efficiency and precision by dynamically selecting query-relevant images for visual scanning and retrieving detailed text only when necessary, thereby outperforming existing visual-text and text-only baselines on benchmarks like LongBench.

Original authors: Jiawei Xu, Zhilin Zhai, Jinrui Fang, Ruohan Xu, Mingfei Lu, Yi Zhang, Guanchu Wang, Tianlong Chen, Ying Ding

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiawei Xu, Zhilin Zhai, Jinrui Fang, Ruohan Xu, Mingfei Lu, Yi Zhang, Guanchu Wang, Tianlong Chen, Ying Ding

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, there is a growing tension between how much information a computer can hold in its mind and how quickly it can think about that information. Large language models, the systems behind many modern chatbots and writing assistants, are designed to process vast amounts of text. However, when these systems try to read a very long document, such as a full book or a lengthy legal report, they face a significant hurdle. The more words they have to consider at once, the more computing power and time they require, often slowing down to a crawl or missing crucial details buried deep within the text. To solve this, researchers have recently tried a clever shortcut: turning pages of text into images. By rendering words as pictures, the computer can process the entire document much faster, similar to how a human might quickly scan a page to get the general layout. Yet, this speed comes with a cost. When text is converted into an image, the fine details—the exact numbers, specific names, or precise dates—can become blurry or difficult to read, leading to errors when the computer needs to answer a specific question about the content.

A team of researchers has developed a new approach called SEER that aims to get the best of both worlds: the speed of looking at images and the precision of reading words. Instead of treating every page of a document the same way, SEER learns to act like a skilled human reader who knows when to skim and when to stop and read closely. The system first looks at the document as a series of images to quickly identify which pages are likely to contain the answer to a specific question. Once it has located these relevant pages, it does not rely on the blurry image alone. Instead, it reaches back to the original text of those specific pages to retrieve the exact words. It then combines the broad context from the images with the precise details from the text to formulate an answer. This method allows the system to remain fast by ignoring irrelevant pages, while still maintaining high accuracy on the parts of the document that truly matter.

The researchers tested this new system on a standard set of challenging questions involving long documents. They found that SEER significantly outperformed previous methods that relied solely on visual compression. On a major benchmark used to measure long-document understanding, the new system achieved an average accuracy of 51.11 percent. This was a clear improvement over the previous visual-only approach, which scored 48.78 percent, and it also surpassed a leading text-based model that did not use any image compression at all. The gains were particularly noticeable in tasks that required extracting specific facts, such as counting how many novels a particular author wrote or finding a specific date in a report. In these cases, the ability to switch from a quick visual scan to a precise text lookup made a substantial difference, improving accuracy by nearly 19 points on some specific Chinese-language tasks where text is densely packed and harder to read in image form.

The study also revealed how the system behaves when it is working. On average, for every question asked, the system selected only about 1.59 pages of the document to examine in detail, even though the documents often contained many more pages. This suggests that the system successfully learned to ignore the vast majority of the text that was not needed for the answer. However, the researchers noted that this efficiency has limits. If the answer to a question depends on information scattered across many different pages, the system's strategy of picking just a few pages might not be enough, and its accuracy would drop. Furthermore, while the system saves a significant amount of computing power by not reading every single word of every page, it does not always run faster in terms of raw time. Because the system takes an extra step to decide which pages to read and then fetches the text, the total time to answer a single question can sometimes be longer than simply reading the whole document, even if the total amount of data processed is smaller.

Ultimately, this work demonstrates that the most efficient way to handle long documents is not to choose between speed and accuracy, but to use a hybrid strategy that adapts to the task at hand. By teaching the computer to know when to look at the big picture and when to zoom in for the details, the researchers have created a system that is both faster and more precise than previous methods for many types of questions. The findings suggest that future artificial intelligence systems will likely rely less on brute-force processing of every word and more on intelligent selection, mimicking the way humans naturally navigate complex information by focusing their attention only where it is needed.

Technical Summary: SEER – Long-Context Reasoning via Selective Visual-Text Compression

Problem Statement

Long-context reasoning remains computationally expensive for Large Language Models (LLMs) due to the quadratic complexity of self-attention mechanisms over text tokens. While visual-text compression—rendering text into images and processing them with Vision-Language Models (VLMs)—offers a promising alternative by reducing token counts (typically 3–4× compression), existing approaches suffer from a critical limitation: uniform compression.

Current methods (e.g., Glyph, Vist) apply compression uniformly across the entire context regardless of query relevance. This creates a trade-off: aggressive compression improves efficiency but sacrifices the fine-grained details necessary for precise reasoning (e.g., extracting specific numbers or names), while conservative compression preserves details but negates efficiency gains. The paper posits that many queries require detailed information from only a small subset of the context, while the remainder serves primarily for global understanding.

Methodology: The SEER Framework

SEER (Selective Visual-Text Compression) addresses this trade-off by learning to perform selective visual-text compression. It treats visual selection and text retrieval as tools within a unified reasoning process, combining the efficiency of visual scanning with the precision of text-based reasoning.

1. Three-Stage Inference Pipeline

SEER operates through a sequential tool-interaction trajectory:

Stage 1: Visual Selection. The model scans the rendered image sequence ( $I$ ) and the query ( $q$ ) to predict a set of relevant image indices ( $S$ ). This leverages the global accessibility of visual representations to rapidly identify query-relevant regions, analogous to human skimming.
Stage 2: Text Retrieval. For the selected images, the system deterministically retrieves the corresponding source text ( $y_R$ ). This step bypasses the need for an OCR component during inference, relying on the fact that source text is available from the rendering process. This provides fine-grained local reasoning capabilities.
Stage 3: Reasoning. The model generates the final answer by reasoning over the combined context of the full visual images ( $I$ ), the query ( $q$ ), the selected image indices ( $S$ ), and the retrieved text ( $y_R$ ). This follows a ReAct-style Thought-Action-Observation paradigm.

2. Training Strategy

SEER is trained via Supervised Fine-Tuning (SFT) on complete tool-interaction trajectories.

Data Construction: The training dataset consists of trajectories containing rendered images, queries, ground-truth relevant images (annotated using a stronger model, Qwen3.5-9B), reference Chain-of-Thought (CoT) reasoning, and final answers.
Objective: The model is trained to output structured tool calls (e.g., {"name": "select", "indices": [1, 3]}), receive the retrieved text as an observation, and proceed to reasoning. A per-token weight mask is used to supervise only model-generated content, masking out non-generated observation tokens.
Initialization: The model is initialized from Glyph-9B and fine-tuned on trajectories constructed from datasets like 2WikiMQA, HotpotQA, and Qasper.

Key Contributions

Framework Proposal: SEER is the first framework to combine visual-text compression with selective tool invocation, learning to retrieve text only from images most relevant to the inference task.
Pipeline Design: A novel three-stage inference pipeline (Select-Retrieve-Reason) that jointly optimizes selection and reasoning capabilities, allowing the model to balance efficiency and precision.
Empirical Performance: Demonstrated improvements on long-context benchmarks, specifically showing that selective retrieval recovers precision lost in uniform compression without fully reverting to the token costs of full-text processing.

Experimental Results

The authors evaluated SEER on the LongBench benchmark (21 tasks across 6 categories) against text-based LLMs and visual-text compression baselines.

Accuracy: SEER achieved an average accuracy of 51.11% on LongBench.
- Outperformed the visual-text baseline Glyph-9B by 2.33 points (48.78% vs. 51.11%).
- Outperformed the text-based Qwen3-8B by 3.49 points (47.62% vs. 51.11%).
Task-Specific Gains: The improvements were most pronounced in extraction-intensive tasks:
- +8.75 points on Qasper (Single-Doc QA).
- +19.17 points on MultiFieldQA-Zh (Chinese QA).
- +4.22 points on NarrativeQA.
Token Efficiency:
- Glyph-9B achieved ~3.14× prompt compression compared to full-text baselines.
- SEER achieved 1.28× compression. While less efficient than pure visual compression, it sacrifices some token savings to recover task-relevant textual precision.
- On average, SEER selects only 1.59 valid images per query (median of 1), keeping the retrieval budget sparse.

Ablation Studies

Without Retrieval: Removing the text retrieval stage (selection only) dropped accuracy to 43.00% (−8.11 points), indicating that selection alone is insufficient for precise extraction.
Without Selection + Retrieval: Pure visual reasoning (Glyph-9B behavior) scored 48.78% (−2.33 points).
Mechanism vs. Data: An iso-training control (same data, no select/retrieve mechanism) scored 45.20%, confirming that the performance gain stems from the select-retrieve mechanism rather than just the fine-tuning data.

Significance and Claims

The paper claims that SEER effectively mitigates the precision-efficiency trade-off inherent in uniform visual-text compression. By learning to selectively retrieve text only where needed, the framework maintains the global context awareness of visual processing while grounding specific reasoning steps in precise textual evidence.

The authors emphasize that the accuracy gains are concentrated on tasks requiring the extraction of exact details (names, numbers, dates), where visual decoding alone is prone to error, particularly with densely rendered text (e.g., Chinese characters). The work demonstrates that a sparse retrieval budget (selecting ~1–3 pages) is sufficient to recover most of the performance lost in uniform compression, suggesting that long-context reasoning does not require processing the entire context with high-fidelity text, but rather the right parts of it.

Limitations

The paper explicitly notes several limitations:

Robustness: The selection mechanism is not perfectly robust; the model occasionally predicts out-of-range indices, which are discarded, potentially leading to retrieval failures in production systems.
Efficiency Scope: Token savings are an average, not a guarantee. On some tasks, the overhead of retrieval makes the prompt larger than full-text processing. Furthermore, latency improvements are limited by the explicit reasoning trace emission; SEER is slower in wall-clock time than direct inference baselines.
Evidence Distribution: The method is optimized for questions where evidence resides on a few pages. Performance degrades when evidence is distributed across many pages, as the single-round selection is recall-limited.
Scope: SEER is specialized for rendered long documents (PDFs, code, etc.) and was not evaluated on natural images, charts, or standard OCR benchmarks.