🤖 AI

Reading is not Reasoning: Bridging the Agentic Policy Gap in Vision-Text Compression

This paper introduces CAPS, a cross-modal agentic policy self-distillation framework that bridges the performance gap between text-based and vision-compressed agent histories by using a model's stronger text-policy to supervise its visual-history counterpart, thereby significantly improving decision-making accuracy while drastically reducing memory context costs.

Original authors: Cheng Fan, Junyi Zhou, Tingzhang Luo, RongJian Xu, Qiyanhui Lu, Mingjian Zhu, Hanting Chen, Jianyuan Guo

Published 2026-08-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Cheng Fan, Junyi Zhou, Tingzhang Luo, RongJian Xu, Qiyanhui Lu, Mingjian Zhu, Hanting Chen, Jianyuan Guo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a super-smart robot detective trying to solve a mystery. To do its job, it has to talk to a computer, ask questions, read the answers, and then decide what to do next. But here's the catch: every time the robot takes a step, it has to remember everything that happened before. If the mystery is long, the robot's "memory" gets huge, like a backpack that keeps getting stuffed with more and more heavy rocks. Eventually, the backpack gets so heavy that the robot moves slower, costs more money to run, and might even trip over its own thoughts.

Scientists have tried to solve this by taking all those heavy text memories and squishing them into a single, dense picture. It's like turning a 100-page novel into a single, information-packed comic book page. The idea is that the robot can look at the picture and remember the story without carrying the heavy text. But there's a problem: robots are usually trained to read words, not to "think" through pictures. When they switch from reading a book to looking at a comic, they often get confused. They might be able to read the words in the picture perfectly, but they forget how to use those words to solve the puzzle. They become great at reading, but terrible at reasoning.

This is exactly the puzzle a team of researchers from City University of Hong Kong and Huawei Technologies set out to solve. They discovered that simply turning text into images isn't enough; the robot needs to learn how to "think" in pictures, not just read them. They found that the gap between a robot that reads text and one that reads images isn't because the robot can't see the words (it can read them just fine). Instead, the robot loses its "game plan." It starts making bad choices, asking the wrong questions, or giving up too soon, even though it has all the information it needs right in front of its eyes.

To fix this, the team invented a clever training method called CAPS (Cross-modal Agentic Policy Self-distillation). Think of it like a master chef (the text-reading robot) teaching an apprentice (the image-reading robot) how to cook. The master chef doesn't just show the apprentice the ingredients; the chef watches the apprentice cook and says, "Hey, when you see this picture of the ingredients, you should chop the onions this way, not that way." The apprentice learns to mimic the master's decision-making process, even though the master is looking at a recipe book and the apprentice is looking at a photo of the ingredients.

The researchers tested this on two very different types of games: a search engine quiz where the robot has to find answers online, and a virtual house where the robot has to move objects around. They found that their new method, CAPS, made the image-reading robot much smarter. It didn't just get better at reading the pictures; it got better at reasoning with them. The robot started making the right choices, asking the right questions, and stopping at the right time, just like the text-reading robot.

The results were impressive. On the search quiz, the new method improved the robot's success rate by about 5% compared to previous methods. On the virtual house game, the improvement was even bigger, jumping up by over 15%. But the best part? The robot was still carrying that super-light "comic book" backpack. It solved the problems faster and used way less computer memory—cutting the memory cost by up to 83% in some cases.

The team showed that you don't have to choose between a heavy, slow memory and a fast, light one. By teaching the robot to think like a pro, even when looking at pictures, you can get the best of both worlds: a robot that is both incredibly smart and super efficient. They proved that the problem wasn't that the robot couldn't "read" the compressed history; the problem was that it needed to learn how to "reason" with it. And with CAPS, they finally taught it how to do just that.

Technical Summary: Reading is not Reasoning: Bridging the Agentic Policy Gap in Vision–Text Compression

Problem Statement

Large language model (LLM) agents solving complex, multi-step tasks accumulate growing interaction histories. Repeatedly processing this expanding context incurs substantial token, memory, and latency costs. Vision–Text Compression (VTC) offers a solution by rendering text-based histories into information-dense images, significantly reducing token counts. However, while VTC improves context efficiency, it introduces a significant capability gap: agents operating on visual histories perform markedly worse than those operating on text histories, despite receiving the same interaction content.

The central question addressed by this work is whether this performance degradation stems from an inability to read the compressed history (a perceptual/OCR issue) or an inability to reason and act upon the recovered information (an agentic policy issue).

Methodology: CAPS Framework

The authors introduce CAPS (Cross-modal Agentic Policy Self-distillation), a two-stage framework designed to transfer reasoning and decision-making capabilities from a strong text-history policy to a visual-history policy. The core insight is that the text-history policy serves as a superior "teacher" for the visual-history "student," as they share the same base model and action space but differ only in the modality of the history representation.

1. Offline Trajectory Self-Distillation

In the first stage, the framework initializes the visual-history policy by learning from successful trajectories generated by the text-history policy.

Data Construction: Successful rollouts from the text-history policy are collected. For each decision step, the text history is rendered into its visual form (image), while the target response (reasoning and action) remains unchanged.
Objective: The visual-history policy is trained via next-token prediction to reproduce the complete responses (reasoning + action) of the teacher policy given the visual history input. This stage transfers the "successful behavior" but is limited to states visited by the teacher.

2. Online Policy Self-Distillation

The second stage addresses the distribution mismatch during reinforcement learning (RL), where the visual-history policy visits states induced by its own actions, which may differ from the teacher's trajectory.

Mechanism: During GRPO (Group Relative Policy Optimization) training, for every state visited by the student (visual) policy, the system constructs a paired text-history context containing the same observations and actions.
Supervision: A frozen text-history teacher policy processes this paired text context and provides full-vocabulary supervision on the student's generated response prefixes.
Loss Function: A forward KL divergence loss is computed between the teacher's and student's token distributions, clipped to prevent gradient dominance by outliers. This loss is combined with the standard RL reward objective, providing dense policy supervision on the student's own interaction distribution.

Key Contributions

Diagnosis of the Agentic Policy Gap: Through controlled evaluations (history recovery, matched-state decisions, and trajectory quality), the authors demonstrate that the performance gap is not primarily due to OCR quality. While history recovery is imperfect, the primary failure mode is a systematic drift in action selection, query formulation, stopping decisions, and evidence use. The visual-history agents "read" the content but fail to reason effectively upon it.
CAPS Framework: The proposal of a two-stage cross-modal self-distillation framework that successfully transfers text-history reasoning behaviors to visual-history agents, bridging the policy gap without sacrificing the token efficiency of VTC.
Empirical Validation: Extensive experiments showing that explicit cross-modal policy alignment is necessary to preserve agent capability under vision–text compression.

Experimental Results

The framework was evaluated on SearchQA (multi-hop search tasks) and ALFWorld (embodied household tasks) using 3B and 7B parameter backbones.

SearchQA: CAPS improved over the baseline AgentOCR by 5.0% (3B) and 3.4% (7B) in Exact Match (EM). Notably, CAPS with a 7B backbone outperformed its own text-history teacher by 0.5 points.
ALFWorld: On full-history tasks ( $H=50$ ), CAPS achieved success rates of 93.8% (3B) and 95.7% (7B), surpassing AgentOCR by 15.6% and 14.5%, respectively.
Efficiency: Despite the performance gains, CAPS maintained the efficiency benefits of VTC. Compared to matched text-history policies, CAPS reduced average memory-context costs by up to 63.3% and peak costs by up to 83.4%.
Ablation Studies: Both offline and online distillation stages were shown to be complementary. Offline distillation alone was insufficient to initialize the policy for visual reasoning, while online distillation was crucial for adapting to the student's own state distribution.

Significance and Claims

The paper argues that context efficiency does not inherently imply capability preservation. The primary bottleneck in vision–text compression for agents is not the ability to recover text from images (OCR), but the agentic policy gap—the failure of the model to reason and act effectively on the recovered information.

The authors claim that their results demonstrate that explicit cross-modal policy self-distillation is a viable and necessary mechanism to bridge this gap. By leveraging the stronger reasoning capabilities of a text-history policy to supervise a visual-history policy, CAPS allows agents to operate on compressed visual contexts while maintaining, and in some cases exceeding, the performance of their text-based counterparts. This suggests that future efficient agent systems must focus on policy alignment across modalities rather than solely on compression fidelity.

The work concludes that effective visual-history compression requires policy alignment, not merely the preservation of recoverable content.