🤖 AI

Agentic System as Compressor: Quantifying System Intelligence in Bits

This paper proposes a framework that quantifies agentic system intelligence as the reduction in bits required to reconstruct a target object, demonstrating through five controlled experiments that agentic components like tool use and search effectively compress information and lower residual uncertainty.

Original authors: Zihan Qin, Hongrui Zhang

Published 2026-06-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Zihan Qin, Hongrui Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a very complex message to a friend, but you have a strict limit on how many "bits" (tiny units of information) you can send. Usually, if you want to send a long story or a complex instruction, you have to write it all out word-for-word. That takes a lot of space.

This paper proposes a new way to measure how "smart" a system is. Instead of asking, "Did the system get the answer right?" (which is like a simple pass/fail test), it asks: "How much information did the system have to send to get the job done?"

The authors call this "Agentic Compression." Here is the core idea broken down with simple analogies:

1. The Core Concept: Intelligence as a Shortcut

Think of intelligence not as "knowing everything," but as knowing the best shortcuts.

The Old Way (Just the Model): Imagine you are trying to describe a specific chess move to a friend over a walkie-talkie. If you just say, "The knight goes to square B," you have to describe the whole board state every time. That's a lot of words (bits).
The Agentic Way (The System): Now, imagine you and your friend both have a rulebook (the environment) and a set of tools. You don't need to describe the whole board. You just say, "Knight to B." Because your friend has the same rulebook, they instantly know exactly which move that is because the rules of chess already eliminate all the illegal moves. You saved a huge amount of space.

The paper argues that the smarter the system, the fewer bits it needs to send. If a system uses tools, searches the web, or checks its own work, it can compress the message down to a tiny hint. If it has to send the whole message from scratch, it's "dumber" in this specific metric.

2. How They Measured It (The Three Tools)

To test this, the researchers built a "compression machine" that tries to shrink messages in three different ways:

The "Exact Copy" Method (Arithmetic Coding): This is like a super-precise typewriter. If the system has to send a specific string of text, it calculates the probability of every letter and sends the most efficient code possible. This is the baseline.
The "Magic Seed" Method (Seed Coding): This is like a game of "Guess the Number." Imagine the system has a list of 1,000 possible answers. Instead of writing down the answer, the sender just says, "It's the 42nd one on the list." The receiver has the same list and the same rules, so they just pick the 42nd one. This is incredibly efficient if the right answer is likely to be on the list.
The "Safety Net" (Fallback): If the "Magic Seed" method fails (the right answer wasn't in the top 1,000), the system switches back to the "Exact Copy" method to ensure the message gets through.

3. What They Tested (The Five Experiments)

They tested this idea on five different scenarios to see if adding "agentic" tools actually saved bits:

Reversed Text: They took a story and reversed it.
- Without tools: The system had to send the reversed letters one by one.
- With tools: The system and receiver agreed on a rule: "We will both reverse the text." The sender just sent the normal text. Result: Massive savings in bits.
Chess Moves:
- Without tools: Sending a move like "Knight to B" requires describing the whole board context.
- With tools: The system uses a chess engine (the environment) that knows the legal moves. It only sends the move, and the engine filters out the impossible ones. Result: Significant savings.
Protein Sequences (Science):
- They tried to generate protein structures.
- With tools: They used a "template" (a known similar protein) to guide the guess.
- Result: The system didn't have to guess the whole structure from scratch; it just filled in the blanks. This saved a massive amount of data.
Retrieval (RAG):
- They asked questions where the answer was hidden in a document.
- With tools: If the system retrieved the right document, it only had to send a tiny hint to get the answer. If it retrieved the wrong document, it had to send much more info.
- Result: Good information (relevant docs) drastically reduced the bits needed.
Story Compression:
- They tried to summarize a story.
- Result: They found that if you let the system "think" longer (use more computer power/search), it could find a shorter summary. This proved a trade-off: More computing power = Shorter message.

4. The Big Takeaways

The paper makes three main points using this "bit-counting" method:

Tools make systems smarter: Adding tools (like a chess engine or a search bar) doesn't just help the system get the right answer; it fundamentally changes how the system communicates, allowing it to send much shorter messages.
Strictness costs bits: If you demand a perfect, word-for-word reconstruction, you need more bits. If you only care about the "gist" or the "meaning" (semantic compression), you can send fewer bits. The "smartness" of the system depends on how strict the rules are.
Power vs. Efficiency: There is a balance between how much computer power you use and how short your message is. If you let the system search harder, it can find a shorter, more efficient way to describe the answer.

Summary

In everyday terms, this paper suggests that true intelligence in AI systems isn't just about the model's brain; it's about the whole team.

If you have a smart worker (the AI model) but no tools, they have to do everything from scratch, which is inefficient. But if you give them a library, a calculator, and a rulebook (the agentic system), they can do the same job with a tiny note. The authors say: "The fewer words you need to write to get the job done, the smarter your system is." They created a way to count those words (bits) to measure exactly how much value each tool adds to the team.

Technical Summary: Agentic System as Compressor: Quantifying System Intelligence in Bits

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed not as isolated predictors but as agentic systems that integrate tools, retrieval mechanisms, environment constraints, verifiers, and multi-turn search. Current evaluation metrics (e.g., accuracy, success rate) fail to capture the source of capability within these complex workflows. A high success rate does not distinguish whether the performance stems from the model's intrinsic prediction ability or from external components (e.g., a retriever supplying evidence, a rule-based environment filtering illegal moves, or a verifier correcting errors). Furthermore, success rates often lack resolution in difficult tasks where both systems "almost fail" (near-zero success) despite having vastly different levels of residual uncertainty over the target space.

The paper addresses the need for a unified, fine-grained analytical framework to quantify system intelligence by measuring how much residual uncertainty different components (models, tools, environments) remove from a target distribution.

2. Methodology: Agentic Codelength

The authors operationalize the principle that "compression is intelligence" by treating an agentic system as a conditional compressor. Under a fixed task distribution, observation standard, system interface, and compute budget, a more intelligent system requires fewer bits to reconstruct a target object.

2.1 Theoretical Framework

Agentic System Definition: Defined as a triple $A = (P, M, E)$ , where $P$ is the program, $M$ represents model interfaces, and $E$ represents environment interfaces (tools, APIs, retrievers).
Conditional Compression: The system compresses a target $x$ given a public condition $c$ (e.g., a prompt or task specification). The codelength $L(x|c)$ measures the information the encoder must transmit to allow a decoder (sharing the same system conditions) to reconstruct $x$ .
Observational Equivalence: Reconstruction does not require byte-exact recovery. Instead, the decoder output $\hat{x}$ must satisfy $y \approx_O x$ under a family of observations $O$ (e.g., functional equivalence in protein folding or semantic equivalence in storytelling).
Component Value: The contribution of a specific component $g$ is defined as the marginal reduction in average codelength: $\Delta_g = \bar{L}_{\Pi \setminus g} - \bar{L}_{\Pi_g}$ .

2.2 Coding Schemes

To estimate codelength in agentic settings, the authors propose an executable compress–decompress protocol combining three schemes:

Arithmetic Coding: Used for exact encoding of specific outputs. It accumulates negative log-likelihood (NLL) token-by-token, converting model predictions into codelength.
Seed Coding: Used for tasks with dense qualifying outputs (e.g., deterministic or replayable environments). The encoder transmits a latent description plus the rank of the first random seed that produces a qualifying output. The cost is $\lceil \log_2 N \rceil$ bits, where $N$ is the sampling budget.
Fallback: If seed coding fails within the budget, the system falls back to arithmetic coding. A 1-bit selector distinguishes the path.

3. Key Contributions

System-Level Perspective: The paper is the first to apply the "compression is intelligence" view to agentic systems rather than isolated models, treating tools, environments, and interaction processes as integral parts of the compression mechanism.
Marginal Bit Value Metric: Introduces a quantitative metric to analyze the contribution of specific components (e.g., a retriever or verifier) via average codelength reduction, offering higher resolution than success rates.
Executable Protocol: Combines arithmetic coding, seed replay, and fallback into a unified protocol capable of measuring codelength across diverse task types.
Empirical Validation: Validates the framework across five distinct settings, demonstrating its ability to analyze component value, observation granularity, and compute trade-offs.

4. Experimental Results

The framework was evaluated in five settings (A–E), isolating typical agentic components:

A. Reversed Text (Deterministic Tool): Using a shared reverse transform reduced codelength from 2.877 to 0.742 bits/byte, demonstrating that deterministic tools can drastically reduce residual uncertainty.
B. Chess Moves (Rule-Based Environment): Enforcing legal-move constraints via a rule-based environment reduced codelength from 9.828 to 6.545 bits/move (saving 3.283 bits/move), showing how environment constraints shrink the candidate space.
C. Protein Sequences (Scientific Templates):
- Calibrated homologous templates reduced codelength from 200.65 to 116.80 bits/seq (83.85 bits saved).
- C' (Functional Equivalence & Verifier): In a functional-equivalence task where success rates were low, adding verifier feedback and repair reduced mean codelength from 402.99 to 27.58 bits/seq (saving 375.41 bits/seq), even while success rates only rose from 68% to 98%. This highlights the metric's sensitivity where success rates plateau.
D. Retrieval-Augmented Generation (RAG): Relevant documents reduced answer codelength by 18.29 bits on average compared to retrieval failure. Distractor documents provided negligible help (2.95 bits), and mixed retrieval retained most of the gain, proving that retrieval value is priced continuously by relevance.
E. Semantic Story Compression:
- Observation Granularity: As the observer standard tightened (from character preservation to full plot/ending correctness), codelength increased monotonically, confirming that compression ability is relative to the observation standard.
- Compute Trade-off: Increasing the summary rollout budget ( $T$ ) improved compression (reducing bits from 292.1 to 127.5) but with diminishing returns, establishing a Pareto frontier between compute budget and codelength.

5. Significance and Claims

The paper claims that agentic codelength serves as a unified accounting tool for residual uncertainty across different information sources and success criteria.

Resolution of "Almost Fail": The metric can distinguish between systems that both have near-zero success rates but differ significantly in their reduction of residual uncertainty (e.g., the protein functional equivalence experiment).
Component Analysis: It allows for the quantification of "marginal bit value," enabling engineers to understand exactly how much a retriever, verifier, or environment constraint contributes to the system's capability.
Relativity of Intelligence: The paper argues that intelligence is not an absolute property of a model but is relative to the protocol (distribution, observer, interface, and budget). An agentic workflow effectively changes the "description language," moving information from the per-instance message into the shared protocol and decoding system.
Agentic Pseudo-Entropy: The authors propose a theoretical extension, "agentic pseudo-entropy," to define the incompressibility of data relative to a class of constrained agentic systems, characterizing the true complexity remaining after exploiting all available tools and search strategies.

The authors maintain a modest stance, noting that these experiments are small-scale and controlled to verify mechanisms rather than serve as a final benchmark for deployed systems. The results offer methodological guidance for evaluating real agent systems rather than definitive performance rankings.