💬 NLP

Unlimited OCR Works

This paper introduces Unlimited OCR, a model that replaces standard decoder attention with a novel Reference Sliding Window Attention (R-SWA) mechanism to maintain a constant KV cache, thereby enabling efficient, single-pass transcription of dozens of pages without the memory and speed degradation typically caused by long output sequences in LLM-based OCR systems.

Original authors: Youyang Yin, Huanhuan Liu, YY, Qunyi Xie, Chaorun Liu, Shiqi Yang, Shaohua Wang, Zhanlong Liu, Hao Zou, Jinyue Chen, Shu Wei, Jingjing Wu, Mingxin Huang, Zhen Wu, Guibin Wang, Tengyu Du, Lei Jia

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Youyang Yin, Huanhuan Liu, YY, Qunyi Xie, Chaorun Liu, Shiqi Yang, Shaohua Wang, Zhanlong Liu, Hao Zou, Jinyue Chen, Shu Wei, Jingjing Wu, Mingxin Huang, Zhen Wu, Guibin Wang, Tengyu Du, Lei Jia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Memory Overload" of Current AI

Imagine you are trying to copy a 100-page book by hand.

How Humans Do It: You look at the book, write a sentence, glance back at the last few words you just wrote to make sure you're on track, and keep going. You don't need to remember every single word you've written since page one; you just need to know where you are right now. Your brain naturally "fades out" the distant past while keeping the immediate context sharp.
How Current AI Does It: Current AI models act like a student who refuses to forget anything. Every time they write a new word, they try to re-read the entire book from page one to the current page to decide what to write next.
- The Result: As the book gets longer, the student gets slower and slower because they are carrying a massive mental load. Eventually, they run out of memory (RAM) and have to stop, reset, and start over on a new page. This is why current AI struggles to process a whole book in one go; it has to do it page-by-page, like a robot stuck in a loop.

The Solution: "Unlimited OCR" and the "Sliding Window"

The researchers at Baidu propose a new model called Unlimited OCR. They wanted to build an AI that thinks more like a human copier. To do this, they invented a new attention mechanism called Reference Sliding Window Attention (R-SWA).

Here is how it works, using a few analogies:

1. The "Reference Book" vs. The "Sliding Window"

Imagine the AI is sitting at a desk with two things:

The Reference Book (The Image): This is the document being scanned. The AI keeps this open on the desk the whole time. It never forgets the original image.
The Sliding Window (The Recent Past): Instead of remembering the whole history of what it wrote, the AI only keeps a small "sticky note" of the last 128 words it generated. As it writes a new word, the oldest word on the sticky note falls off, and the new one is added.

The Magic: The AI looks at the Reference Book (to know what to copy) and the Sliding Window (to know where it is in the process). It ignores the rest of the history. This keeps its "mental load" (memory usage) constant, no matter if it's copying 1 page or 100 pages.

2. The "Deep Squeeze" (High-Compression Encoder)

Before the AI even starts writing, it has to look at the image.

Old Way: If you have a high-resolution photo of a book, it's like trying to describe every single pixel. It takes up a huge amount of space.
Unlimited OCR Way: They use a "Deep Squeeze" (DeepEncoder). Imagine taking a high-resolution photo and compressing it into a tiny, efficient summary without losing the important details. This means the "Reference Book" takes up very little space on the desk, leaving plenty of room for the AI to work.

What Did They Achieve?

By combining the "Deep Squeeze" with the "Sliding Window," the researchers achieved three main things:

One-Shot Long-Horizon Parsing: The model can process dozens of pages of a document in a single go. It doesn't need to stop and reset. It can read a whole book from start to finish in one continuous flow.
Constant Speed: Because the AI isn't trying to remember the whole history, it doesn't get slower as the document gets longer. Whether it's on page 1 or page 50, it writes at the same speed.
Better Accuracy: Surprisingly, by focusing only on the relevant recent context and the original image, the AI actually made fewer mistakes than the previous best models. It didn't get confused by its own long history.

The "Real-World" Test

The researchers tested this on a benchmark called OmniDocBench.

The Result: Unlimited OCR scored higher than almost every other model, including the one it was built upon (DeepSeek OCR).
The Long-Form Test: They fed it books with 40+ pages. The model kept its accuracy high, proving it didn't get "lost" in the middle of the book.

What Does This Mean for the Future?

The paper suggests this isn't just for reading books. Because the "Sliding Window" logic is so efficient, it could be applied to other tasks where you need to listen or translate for a long time without getting tired or slow, such as:

ASR (Automatic Speech Recognition): Transcribing hours of audio without the system slowing down.
Translation: Translating long documents in one pass.

Summary

Think of Unlimited OCR as upgrading an AI from a "photographic memory" that gets overwhelmed by long tasks, to a "smart copier" that knows exactly what it needs to remember (the original image and the last few words) and what it can safely forget. This allows it to work faster, use less memory, and handle massive documents that used to be impossible for AI to process in one go.

Technical Summary: Unlimited OCR Works

1. Problem Statement

Current end-to-end Optical Character Recognition (OCR) models, even those leveraging Large Language Models (LLMs) as decoders, face significant limitations in long-horizon parsing tasks (e.g., transcribing dozens of pages in a single forward pass).

Memory and Efficiency Bottleneck: Standard attention mechanisms require the Key-Value (KV) cache to grow linearly with the output sequence length. As the sequence lengthens, this leads to unbounded memory consumption and progressively slower generation speeds due to increased attention computation costs.
The "For-Loop" Workaround: Existing models typically process documents page-by-page in a loop, resetting memory at each step. This fragments the task into isolated short-horizon problems managed by an external scheduler, failing to emulate the continuous cognitive state humans exhibit when transcribing long texts.
Visual Fidelity vs. State Transitions: While linear attention mechanisms reduce complexity, they often apply recurrent state updates to visual tokens. This causes "progressive blurring" of visual features, degrading recognition accuracy.

2. Methodology

The paper proposes Unlimited OCR, a model designed to emulate human working memory during long-horizon copying tasks. It builds upon the DeepSeek OCR baseline, retaining its high-compression encoder but fundamentally altering the decoder's attention mechanism.

Core Architecture

Baseline: Uses DeepSeek OCR's DeepEncoder, which employs a cascaded ViT architecture (SAM-ViT + CLIP-ViT) to achieve a 16× token compression rate. This ensures visual tokens are encoded once and remain static, avoiding the need for state transitions on the input side.
Decoder: A Mixture-of-Experts (MoE) LLM with 3B total parameters and 500M activated parameters.
Key Innovation: All standard Multi-Head Attention (MHA) layers in the decoder are replaced with Reference Sliding Window Attention (R-SWA).

Reference Sliding Window Attention (R-SWA)

R-SWA is a general-purpose parsing attention mechanism that constrains attention within a two-segment window of size $m + n$ :

Reference Segment ( $P$ ): Includes all prefix tokens (visual tokens and the prompt). This segment has a fixed size ( $L_m$ ) and remains globally visible to all subsequent tokens. Crucially, these tokens do not undergo state transitions, preserving visual fidelity.
Decode Segment ( $D_n$ ): A causal sliding window of fixed width $n$ (default 128) over the previously generated output tokens.

Mechanism:

Each generated token attends to all reference tokens (visuals + prompt) and only the preceding $n$ output tokens.
KV Cache Management: Unlike standard MHA where the cache size is $L_m + T$ (growing with sequence length $T$ ), R-SWA maintains a constant cache size of $L_m + n$ . The cache acts as a queue where the oldest generated token's KV pair is evicted as a new one is added.
Result: This design decouples memory usage and computational cost from the total output length, enabling constant latency and memory footprint regardless of how many pages are processed.

3. Key Contributions

R-SWA Algorithm: Introduction of a novel attention mechanism that combines full visibility of reference (visual) tokens with a bounded causal window for generated tokens. This achieves linear-complexity attention without blurring visual features.
Unlimited OCR Model: A practical implementation based on DeepSeek OCR that successfully parses dozens of document pages in a single forward pass under a standard 32K maximum length constraint.
Performance Gains: Demonstrated that replacing standard attention with R-SWA yields not only efficiency gains but also accuracy improvements on standard benchmarks, suggesting that the "soft forgetting" mechanism aligns well with OCR tasks.
Generalizability: The paper posits that R-SWA is applicable beyond OCR to other reference-based long-horizon tasks such as Automatic Speech Recognition (ASR) and translation.

4. Experimental Results

Benchmarks and Metrics

The model was evaluated on OmniDocBench (v1.5 and v1.6) and an in-house long-horizon test set. Metrics included Text Edit Distance, Formula CDM, Table TEDS, and Reading Order Edit Distance.

Main Findings

OmniDocBench v1.5: Unlimited OCR achieved an overall score of 93.23%, outperforming the DeepSeek OCR baseline (87.01%) by 6.22 points. It also surpassed the DeepSeek OCR 2 baseline.
OmniDocBench v1.6: Achieved a new SOTA of 93.92% overall, outperforming other end-to-end models like FireRed-OCR and Qianfan-OCR.
Subcategory Analysis: The model showed consistent improvements across all nine document types (PPT, Academic Paper, Book, etc.), with no metric showing a disadvantage compared to the baselines.
Long-Horizon Parsing:
- The model successfully processed inputs of 40+ pages in a single pass.
- At 40+ pages, the Edit Distance remained below 0.11, and Distinct-35 was 96.90%.
- Errors at extreme lengths were attributed primarily to resolution limitations of the "Base" encoder mode (1024×1024) rather than the attention mechanism losing direction.

Efficiency Analysis

Throughput (TPS): While both models perform similarly at short lengths (256 tokens), Unlimited OCR maintains a constant throughput (~7,800 TPS) as sequence length increases. In contrast, DeepSeek OCR's throughput drops significantly (by ~35% at 6,000 tokens) due to KV cache growth.
Latency: The per-call duration for the Flash Attention kernel remains constant for Unlimited OCR, whereas it grows linearly for the baseline.

5. Significance and Claims

The paper claims that Unlimited OCR represents a shift toward one-shot long-horizon parsing, moving away from engineering workarounds (page-by-page loops) toward a more AGI-aligned cognitive architecture.

Human-Like Efficiency: By mimicking human working memory—retaining the source (visuals) and recent context while "softly forgetting" distant outputs—the model achieves efficiency that does not degrade with task length.
Lossless Optimization: The authors assert that R-SWA is a "free lunch" for OCR, improving accuracy while drastically reducing memory and compute costs.
Future Potential: The work suggests that attention computation and memory footprint are no longer the primary bottlenecks for long-horizon parsing. The authors plan to extend R-SWA to ASR and translation and explore "prefill pools" to simulate human page-flipping for truly unlimited context lengths in the future.

Limitations Acknowledged:
The current implementation is still constrained by the prefill length (limited by the 32K context window and DeepEncoder's compression). Truly "unlimited" parsing (e.g., entire books without context limits) requires future work on longer context training (128K+) and dynamic prefill fetching mechanisms.