💬 NLP

InSight-doc: Agentic Visual Perception for Long-Document Understanding

InSight-doc is an agentic visual perception framework that adaptively zooms into high-resolution regions of long documents to significantly improve accuracy, reduce hallucinations, and lower inference latency without relying on external retrievers.

Original authors: Kaican Li, Weiyan Xie, Lewei Yao, Jiannan Wu, Lanqing Hong, Yongxiang Huang, Nevin L. Zhang

Published 2026-08-12

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Kaican Li, Weiyan Xie, Lewei Yao, Jiannan Wu, Lanqing Hong, Yongxiang Huang, Nevin L. Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to solve a massive jigsaw puzzle, but instead of a table, the pieces are scattered across a library the size of a city. Now, imagine you have a super-smart robot assistant to help you. In the old days, this robot would try to look at the entire library at once, squinting its eyes so hard it got a headache, missed the tiny details, and eventually gave up on the puzzle. This is the problem with current AI when it tries to read long documents: it gets overwhelmed by the sheer amount of information, loses its focus, and starts making things up (a glitch called "hallucination") just to fill in the blanks.

To fix this, scientists are teaching AI to be more like a human detective. Instead of staring blankly at a whole page, a human detective zooms in on a specific clue, reads the fine print, and then moves to the next spot. This paper introduces a new AI system called InSight-doc that does exactly that. It treats "zooming in" not as a fixed setting, but as a superpower it can use whenever it needs to. It starts by looking at the whole document from far away (low resolution), and only when it spots something interesting does it pull out a magnifying glass to get a crystal-clear, high-resolution view of just that tiny spot. This way, it saves energy, avoids getting confused, and finds the truth without guessing.

The Detective with a Magic Magnifying Glass

Meet InSight-doc, a new kind of AI detective designed to tackle the nightmare of reading long, complicated documents like research papers, financial reports, or legal contracts. Most AI models today are like students trying to read a 100-page book by squinting at a tiny photocopy of the whole thing at once. They try to process every single word and image simultaneously. This is a recipe for disaster: the computer gets slow, runs out of memory, and because it's trying to hold too much in its head at once, it starts to "rot" (a fancy term for getting confused and forgetting what it just read). Worse, when it gets stuck, it often just makes up an answer to look smart, a behavior we call "hallucination."

InSight-doc flips the script. Instead of trying to swallow the whole book in one bite, it acts like a curious teenager flipping through a magazine. It starts by skimming the pages quickly at a low resolution—just enough to see the big pictures and headings. Then, when it spots a paragraph or a chart that looks like it might hold the answer to a question, it doesn't just guess. It uses a "zoom-in" tool to grab a high-resolution, crystal-clear crop of just that specific area. It's like having a magic magnifying glass that you only use when you really need to read the fine print.

The paper shows that this "coarse-to-fine" approach is a game-changer. By starting small and zooming in only when necessary, InSight-doc doesn't just save time; it actually gets smarter. In tests on long documents, it reduced the number of "made-up" answers (hallucinations) by more than 40% compared to standard models. It also became much faster, cutting the time it takes to think and answer by 41% to 68%, all while getting the right answer more often.

How It Learned to Zoom

You might wonder, how does a robot learn to know when to zoom and where to look? The researchers didn't just tell the AI to "be smart." They built a massive training playground for it. They created a special dataset of 17,900 examples where the AI was taught exactly how to zoom in step-by-step to find the answer, like a teacher showing a student how to use a microscope. They also added another 19,200 tricky examples where the AI had to learn through trial and error (a method called Reinforcement Learning) to figure out the best way to hunt for clues.

Through this training, the AI learned a "multimodal chain of thought." This is a fancy way of saying it learned to talk to itself: "Hmm, the answer isn't on page 1. Let me zoom in on the chart on page 5. Oh, that's not it either. Let me check the table on page 12." It builds a trail of evidence, zooming in on different parts of the document until it has enough proof to give a confident answer. If the document doesn't have the answer, it learns to say, "I don't know," instead of making something up.

The Results: Faster, Smarter, and Less Gassy

The paper tested this new detective against some of the smartest AI models available, including big proprietary ones from major tech companies. The results were impressive. On standard document quizzes, InSight-doc improved accuracy by a huge margin—up to 16.4 percentage points better than the baseline model when starting with a low-resolution view.

But the real magic happened with the longest, most confusing documents. When the documents got really long (hundreds of pages), the old models started to stumble and make mistakes. InSight-doc, however, stayed cool. It managed to find the right answers while using 58% fewer "tokens" (the digital building blocks of text and images the computer has to process). This means it didn't just get the answer right; it did it with a fraction of the computing power and time.

The researchers also checked if this skill could be used on things other than documents, like looking at complex maps or charts. Even though it was trained mostly on documents, the AI showed it could generalize, improving its performance on these visual puzzles too.

What It's Not (And What It Doesn't Do)

It's important to know what InSight-doc isn't. It's not a magic wand that reads your mind, and it doesn't rely on an external search engine to find pages for it. Some other AI systems try to "search" for the right page first, like using Google to find a PDF before reading it. InSight-doc does this entirely on its own, inside its own brain, without needing to call an outside helper. This makes it faster and less likely to get lost if the search engine picks the wrong page.

The paper also makes it clear that this isn't a solved problem for every situation. The researchers tested it on specific types of documents and questions. They didn't claim it works perfectly on every single type of image or text in the universe, and they admit that there's still room to make it even better. They didn't use any fancy new math tricks or secret ingredients; they just showed that giving an AI the ability to "look closer" when it needs to is a powerful, simple idea that works surprisingly well.

In short, InSight-doc teaches us that sometimes, the best way to see the whole picture is to stop trying to look at everything at once. By learning to zoom in on the details only when they matter, AI can become a more reliable, efficient, and honest partner in solving the world's most complex document puzzles.

Technical Summary: InSight-doc: Agentic Visual Perception for Long-Document Understanding

Problem Statement

Long-document understanding, particularly for visually rich documents like research papers and financial reports, presents a dual challenge for Multimodal Large Language Models (MLLMs): computational inefficiency and "context rot." Traditional end-to-end approaches that feed high-resolution pages into MLLMs incur quadratic time and space costs ( $O(N^2)$ ) due to attention mechanisms, leading to degraded performance as context length increases. Conversely, existing retrieval-based or coarse-to-fine methods often rely on external retrievers, which introduce decoupled pipelines prone to retrieval errors and indexing overhead, or they operate at the page level, failing to isolate sub-page evidence efficiently. Furthermore, models often hallucinate when critical visual details are lost in low-resolution downsampling or obscured by excessive context.

Methodology

The authors propose InSight-doc, a fully end-to-end, retriever-free agentic framework that treats visual resolution as an adaptive, reasoning-time resource. Instead of processing documents at a fixed high resolution or relying on external page retrieval, InSight-doc employs a "coarse-to-fine" active perception strategy:

Adaptive Zooming: The model begins with a low-resolution overview of the entire document. During the reasoning process, it dynamically decides to "zoom in" on specific regions of interest by issuing tool calls to crop high-resolution patches from the original source images.
Interleaved Reasoning: The framework utilizes an interleaved multimodal chain-of-thought (CoT). The model emits a thought, followed by a zoom-in action (specifying image index, region description, and bounding box), receives the high-resolution crop as visual evidence, and appends it to the reasoning chain. This mimics human reading behavior, where one starts with an overview and drills down into details only when necessary.
Training Strategy: To train this agent, the authors constructed a specialized corpus:
- SFT Data: 17.9K high-quality examples with region-level zoom-in trajectories generated by a two-agent system (a vReasoner and a vSearcher) using InSight-o3.
- RL Data: 19.2K hard examples used for Reinforcement Learning (specifically GRPO) to refine the agent's ability to locate evidence and avoid hallucinations on unanswerable questions.
- Filtering: A three-stage pipeline filters out questions answerable without the document or without zooming, ensuring the model learns to actively seek visual evidence.

Key Contributions

Framework: Introduction of InSight-doc, an end-to-end agentic framework that adaptively acquires multi-scale visual evidence without external retrievers, enabling sub-page, region-level grounding within a unified reasoning loop.
Dataset: Construction of a diverse, high-quality training corpus comprising 17.9K SFT trajectories and 19.2K RL examples, specifically designed to teach active perception and multi-hop reasoning over long documents.
Efficiency-Accuracy Trade-off: Demonstration that the model significantly improves the accuracy-efficiency Pareto frontier, achieving higher accuracy with substantially reduced sequence lengths and inference latency compared to baselines.

Experimental Results

Evaluated on standard and long-document VQA benchmarks (DUDE, MP-DocVQA, MMLongBench-Doc, LongDocURL) and general high-resolution VQA (MME-RealWorld-Lite, O3-Bench), InSight-doc-8B (based on Qwen3-VL-8B) shows substantial improvements:

Accuracy: On long-document benchmarks, InSight-doc-8B (SFT+RL) improves over the baseline Qwen3-VL-8B by 4.3 to 16.4 accuracy points depending on the initial resolution setting. For instance, it achieves 66.9% average accuracy at 50 DPI (low res), surpassing the 100 DPI baseline by 16.4 points.
Efficiency: The model reduces inference latency by 41%–68% and sequence length by up to 69% compared to high-resolution baselines, while maintaining or improving accuracy.
Hallucination Reduction: On unanswerable questions, the model significantly improves the ability to recognize insufficient evidence, achieving F1 score gains of 24.6 points (on DUDE) and 25.9 points (on MMLongBench-Doc) at low resolution compared to the baseline without zoom. This corresponds to a reduction in the rate of non-abstaining answers (hallucination rate) by more than 40%.
Generalization: The learned policy generalizes to general high-resolution VQA tasks, outperforming the base model with zoom capabilities and competing with proprietary models.

Significance and Claims

The paper claims that InSight-doc addresses the central question of how to handle long-context tasks reliably and efficiently. By shifting from passive, fixed-resolution processing to active, adaptive visual perception, the framework mitigates context rot and reduces computational costs without sacrificing fidelity. The authors emphasize that their retriever-free, end-to-end approach allows for more robust multi-hop reasoning and self-correction compared to methods relying on external retrieval systems. The results suggest that "thinking with images" via selective zooming is a viable and efficient paradigm for long-document understanding, offering a favorable trade-off between token consumption, latency, and reasoning accuracy. The authors modestly note that their evaluation is currently limited to the Qwen3-VL-8B backbone and that further exploration of advanced RL methods and other model architectures remains an open direction.