💬 NLP

RAGOCR: Optical Compression of Retrieval-Augmented Text via Visual Representation

RAGOCR is a novel framework that compresses retrieved documents into query-conditioned visual representations with a dynamic resolution mechanism, achieving over 15% higher accuracy than standard RAG while reducing input tokens by 87.5% and outperforming existing hard and soft compression baselines.

Original authors: Jiayang Yu, Jialun Zhong, Lei Zou

Published 2026-08-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiayang Yu, Jialun Zhong, Lei Zou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a giant mystery, but instead of a single clue, you are handed a library's worth of books. You need to find the one sentence that holds the answer, but reading every word in every book takes forever and your brain gets tired. This is the daily struggle of modern "smart" computer programs called Large Language Models (LLMs). These programs are like brilliant detectives who can write stories, answer questions, and solve problems, but they have a short attention span when it comes to reading. If you give them too much text at once, they get overwhelmed and start missing the important details.

To help these detectives, scientists use a trick called "Retrieval-Augmented Generation" (RAG). Think of RAG as a super-efficient librarian. When you ask a question, the librarian quickly grabs the most relevant books from the shelves and hands them to the detective. But here's the catch: even with the librarian's help, the pile of books can still be too heavy to carry. The detective might drop the books, or worse, get so lost in the middle pages that they forget the beginning and the end. Scientists have tried to shrink these books by summarizing them or cutting out boring words, but it's like trying to fold a map that keeps tearing or losing the landmarks you actually need.

Now, imagine a different way to carry information. Instead of trying to shrink the words, what if you could turn the whole page of text into a single, tiny photograph? Recent discoveries suggest that computers can "see" a page of text as an image and understand it using far fewer mental "tokens" (the units computers use to think) than reading the words one by one. It's like the difference between reading a long letter and glancing at a photo of that letter; the photo packs the same information into a much smaller space. But there's a new problem: if you turn every book into a photo, you still have too many photos to look at, and some are just junk. The real challenge is figuring out which photos to keep big and clear, and which ones to squish down into tiny, blurry thumbnails without losing the secret clues.

This is exactly what the paper "RAGOCR" tackles. The authors, researchers from Peking University, propose a clever new system that turns retrieved text documents into images and then uses a smart "compressor" to resize them based on how important they are to your specific question.

Here is how their magic works: First, the system takes all the text documents the librarian found and turns them into images, like taking a picture of each page. Then, a special AI compressor looks at your question and all those images together. It acts like a wise editor who knows exactly what you need. If a document is super relevant to your question, the compressor keeps it at a high resolution, so every tiny detail is crystal clear. But if a document is only slightly related or totally irrelevant, the compressor aggressively shrinks it down, making it a tiny, low-resolution speck that takes up very little space.

The paper suggests that this "query-aware" approach is a game-changer. By using a training method called Group Relative Policy Optimization (GRPO), the system learns to balance the size of the images perfectly. In their tests, this method allowed the computer to answer medical questions with over 15% more accuracy than the standard way of doing things, while using only one-eighth of the usual amount of "thinking space" (tokens). It's like getting a full library's worth of knowledge into a single backpack.

Interestingly, the paper also found a surprising side effect: the way the compressor decides to shrink the images actually tells us which documents are the most important. The system naturally learns to give high scores to the best documents just by deciding how much to shrink them. This means the same tool that shrinks the text can also act as a filter to pick the best books, potentially replacing other complicated ranking tools.

The researchers tested this on five different sets of medical questions, including tricky exams that require deep reasoning. They found that RAGOCR consistently beat other methods, whether the documents were plain text, complex medical guidelines, or even slide decks with pictures and text mixed together. They showed that simply turning text into images helps, but turning them into smartly resized images helps even more. The paper argues that this approach bridges the gap between "hard" compression (cutting out words) and "soft" compression (summarizing), offering a way to keep the most critical details while throwing away the noise.

In short, RAGOCR suggests that we don't have to choose between reading everything and reading nothing. Instead, we can give our AI detectives a visual map where the important roads are wide and clear, and the dead ends are just tiny, blurry dots. This allows them to solve mysteries faster and more accurately, proving that sometimes, seeing is indeed believing—and seeing is also thinking.

Technical Summary: RAGOCR

Problem Statement

Retrieval-Augmented Generation (RAG) has become essential for knowledge-intensive question answering, yet scaling RAG pipelines faces a critical bottleneck: the prohibitive computational cost of processing lengthy retrieved contexts. As the volume of retrieved documents grows, Large Language Models (LLMs) struggle to attend to critical information buried within long inputs, leading to degraded generation quality.

Existing compression approaches face a fundamental trade-off:

Hard Compression: Methods that operate online in a query-aware fashion (e.g., token pruning, sentence extraction) achieve only modest compression rates and often require fine-tuning the generative model.
Soft Compression: Methods that attain higher compression ratios (e.g., embedding-based) rely on costly offline encoding that is entirely agnostic to the input query.

Furthermore, recent work suggests that visual tokens carry substantially higher information density than text tokens. However, applying visual rendering to RAG without a query-aware strategy introduces a new challenge: retrieved documents vary drastically in relevance. Applying a uniform resolution across all rendered images leads to either wasteful token expenditure on marginally relevant passages or catastrophic information loss on highly pertinent ones. There is a lack of a query-aware allocation strategy that dynamically assigns visual resolution based on the semantic relevance between each document and the query.

Methodology: RAGOCR

The authors propose RAGOCR (Retrieval-Augmented Generation via Optical Character Rendering), a framework that compresses retrieved documents into compact visual representations conditioned on the input query. The pipeline operates in three stages:

1. Rendering Stage

Retrieved text documents are converted into images through text rasterization. This transforms the one-dimensional token sequence into a two-dimensional visual representation with inherently higher information density. The rendering process is deterministic and parameterized by font size, maximum canvas width, and padding. These images can be precomputed offline, introducing negligible overhead at inference time.

2. Optical Compression Stage (The Core Innovation)

A query-aware compressor examines all rendered images jointly with the input query and predicts a per-image compression factor.

Mechanism: The compressor is a lightweight vision-language model (specifically Qwen3-VL-2B) trained using Group Relative Policy Optimization (GRPO).
Adaptive Allocation: The policy learns to assign lower compression (higher resolution) to images corresponding to highly relevant documents to preserve fine-grained details, while aggressively downscaling less relevant ones.
Training Objective: The GRPO reward function is a composite of three terms:
1. Task Reward ( $r_{task}$ ): Binary reward based on whether the downstream generator produces the correct answer using the compressed images.
2. Compression Reward ( $r_{comp}$ ): Encourages aggressive compression (maximizing the sum of log compression factors) but is only activated when the task reward is positive, preventing the model from sacrificing accuracy for compression.
3. Format Reward ( $r_{format}$ ): Ensures the model outputs a valid structured template for the compression factors.

3. Inference Stage

At inference, the compressor processes the query and all rendered images in a single forward pass to generate compression factors. Each image is resized by a factor of $1/\sqrt{c_i}$ (where $c_i$ is the compression factor), reducing visual token counts proportionally. The compressed image set is then fed to a vision-language generator to produce the final answer.

Emergent Reranking Capability

A key finding is that the compression ratios learned by the compressor serve as a surprisingly effective proxy for passage relevance. Since the compressor assigns lower compression ratios to more relevant passages, the inverse of the compression factor ( $1/c_i$ ) naturally encodes a relevance ranking. This allows the compressor to function as a reranker without any additional training objective or module.

Key Contributions

RAGOCR Framework: A novel system combining optical rendering with dynamic, query-aware resolution allocation to compress text-based RAG, balancing token efficiency and answer quality.
GRPO-Trained Compressor: Development of a lightweight compressor trained via GRPO that adaptively assigns compression ratios based on query relevance, avoiding the need for separate fine-tuning of the generative model.
Unified Compression-Reranking: The discovery that optical compression and relevance estimation are two sides of the same coin. The compressor inherently encodes passage relevance, enabling it to outperform dedicated reranking models (like Qwen3-Reranker-4B) and function as a unified compression-reranker system.
Generalizability: Demonstration that the approach works across diverse knowledge sources, including free-text passages, knowledge graphs, and visually rich image-based documents (e.g., slide decks).

Experimental Results

Experiments were conducted on five knowledge-intensive QA benchmarks (MedQA, MedMCQA, PubMedQA, BioASQ, MMLU-Med) using the MedOmniKB retrieval corpus.

Performance vs. Naive RAG: RAGOCR surpasses naive RAG by over 15% in accuracy (61.47% vs. 44.63% average) while requiring only one-eighth the number of input tokens (approx. 87% reduction in token count).
Comparison with Baselines: RAGOCR consistently outperforms both hard compression methods (LongLLMLingua, Provence, EXIT, FaviComp) and soft compression methods (OSCAR) across varying retrieval depths.
- It outperforms the best text-based compression baseline (Provence) by nearly 3 points in accuracy while using fewer tokens.
- It outperforms dedicated rerankers (Qwen3-Reranker-4B) when used as a standalone reranker (59.15% vs. 54.52%).
Multimodal Evaluation: On the SlideVQA benchmark (20-page slide decks), RAGOCR achieved the best performance across Exact Match, F1-score, and ANLS metrics, demonstrating its ability to handle documents with interleaved text and visual elements.
Ablation Studies:
- Uniform vs. Adaptive: Static OCR compression (uniform resolution) degraded performance compared to full-resolution Visual RAG, confirming that query-aware adaptation is essential.
- Pipeline Integration: Combining RAGOCR with a text-based reranker (R&R) yielded the highest accuracy (63.60%), suggesting that text-based reranking and optical compression capture complementary aspects of document utility.

Significance and Claims

The paper claims that RAGOCR bridges the gap between the high information density of visual tokens and the need for query-aware context management in RAG. By rendering text as images and adaptively compressing them, the framework enables models to absorb more knowledge within fixed context windows or transmit the same information with significantly fewer tokens.

Crucially, the authors highlight the unification of compression and ranking. The fact that a single model can simultaneously compress content and rank it by relevance suggests a new paradigm where "optical compression and relevance estimation are two sides of the same coin." This points toward a future where retrieval-augmented systems can utilize a single lightweight model to handle retrieval filtering, context compression, and relevance estimation, significantly reducing the computational overhead of current RAG pipelines.