💻 computer science

Latent Action Reparameterization for Efficient Agent Inference

This paper proposes Latent Action Reparameterization (LAR), a framework that learns a compact, multi-step latent action space from agent trajectories to significantly reduce the effective decision horizon and inference costs of LLM agents while maintaining or improving task success rates.

Original authors: Wenhao Huang, Qingwen Zeng, Qiyue Chen, Zijie Guo, Yu Sun, Cheng Yang, Siru Ouyang, Jiri Gesi, Fang Wu, Jiayi Zhang, Huaming Chen, Bang Liu, Xiangru Tang, Chenglin Wu

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wenhao Huang, Qingwen Zeng, Qiyue Chen, Zijie Guo, Yu Sun, Cheng Yang, Siru Ouyang, Jiri Gesi, Fang Wu, Jiayi Zhang, Huaming Chen, Bang Liu, Xiangru Tang, Chenglin Wu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are giving a robot a very long, detailed list of instructions to build a house.

The Old Way (The Problem):
Currently, when we ask a Large Language Model (LLM) agent to do something complex, we force it to think and speak in tiny, atomic steps. It's like telling the robot: "Pick up the hammer. Now move your hand 2 inches. Now swing. Now hit the nail. Now pick up the hammer again."

Even if the robot is smart, it has to generate a massive amount of text just to say "I am picking up the hammer." This creates a huge "decision horizon"—a long line of tiny decisions that the computer has to process one by one. This makes the robot slow, expensive to run, and prone to getting tired (or running out of memory) before it finishes the job.

The New Way (Latent Action Reparameterization - LAR):
The authors of this paper propose a clever shortcut called Latent Action Reparameterization (LAR).

Think of LAR as teaching the robot a new language of "Super-Commands." Instead of saying "Pick up hammer, move hand, swing," the robot learns to say a single magic word: "Hammer-Action."

Here is how it works, using simple analogies:

1. The "Macro-Button" Analogy

Imagine you are using a computer. Instead of clicking "File," then "New," then "Document," then "Name it," you press one button that says "Create New Doc."

The Paper's Claim: LAR automatically learns these "Super-Commands" by watching the robot work. It finds patterns where the robot always does the same sequence of steps (like opening a tool, typing a specific format, and closing it). It bundles those steps into one invisible "latent action."
The Result: The robot makes fewer decisions. Instead of 100 tiny steps, it takes 10 big steps. This makes it much faster and cheaper to run.

2. The "Zipper" vs. The "Suitcase" (What gets compressed?)

You might ask: "If we bundle everything together, won't the robot forget the important details?"
The paper is very careful here. They use a Zipper analogy:

The Zipper (The Structure): The boring, repetitive parts of the instructions (like "Please open the search tool" or "Here is the code format") are low-entropy. They are predictable. LAR zips these up into a single latent token.
The Suitcase (The Content): The unique, changing parts (like the specific search query "Who was the next British Prime Minister?" or a specific number) are high-entropy. These are the "stuff" inside the suitcase. LAR leaves these unzipped and visible.
Why this matters: If you zipped up the specific search query, the robot would try to search for "The Next British Prime Minister" every single time, even if the question was about a different country. That would break the robot. LAR only zips up the structure, not the content.

3. The "Goldilocks" Zone (The Abstraction Boundary)

The paper discovered a critical limit, which they call the Abstraction Boundary.

Too Little Compression: The robot is slow because it's still saying too many small words.
Just Right: The robot is fast because it uses "Super-Commands" for the boring parts but still speaks clearly for the important parts.
Too Much Compression: If you try to bundle the specific questions or unique details into a "Super-Command," the robot breaks. It's like trying to use a "Hammer-Action" button to search for a specific person; the robot gets confused because the button doesn't know who to search for. The paper shows that if you cross this line, the robot's performance crashes suddenly.

4. The Results (What actually happened?)

The researchers tested this on three different types of tasks:

Trivia (TriviaQA): Answering complex questions.
Coding (KodCode): Writing computer code.
Web Browsing (Mind2Web): Navigating websites and using tools.

The findings were:

Speed: The robots using LAR generated significantly fewer words (tokens) to do the same job. This means they finished tasks faster and used less computer power (GPU memory).
Smarts: Despite using fewer words, the robots were just as good (or sometimes better) at getting the right answer. They didn't lose their intelligence; they just stopped wasting time on repetitive chatter.
Generalization: They trained the robot on one set of tasks (like coding), and it could use those "Super-Commands" on new, unseen coding tasks without needing to be retrained.

Summary

The paper argues that the biggest bottleneck in making AI agents faster isn't just building bigger computers or smarter models; it's how we ask them to speak.

By teaching agents to group repetitive, predictable actions into single "Super-Commands" while keeping the unique, important details visible, we can make them faster, cheaper, and just as smart. It's like upgrading from a robot that counts every single step to a robot that knows how to "walk," "run," and "jump" as single, fluid movements.

Technical Summary: Latent Action Reparameterization for Efficient Agent Inference

Problem Statement

Large Language Model (LLM) agents often rely on long sequences of low-level textual actions to perform multi-step reasoning and tool interaction. This token-level granularity results in excessively large effective decision horizons, leading to high inference latency and prohibitive computational costs. While prior work has addressed efficiency through system-level optimizations, hardware acceleration, or prompt engineering, these approaches largely operate orthogonally to the agent's decision process. They may reduce the cost of individual inference steps but fail to address the fundamental bottleneck: the representation of the action space itself. The authors argue that inference efficiency is constrained by the granularity at which agent actions are represented, and that treating action representation as a first-class modeling choice is necessary to scale efficient LLM agent inference.

Methodology: Latent Action Reparameterization (LAR)

The authors propose Latent Action Reparameterization (LAR), a framework that learns a compact latent action space where each latent action corresponds to a multi-step semantic behavior. Unlike hand-crafted macros or hierarchical controllers, LAR learns these units directly from agent trajectories and integrates them into the model.

Core Concepts

Action Redefinition: Instead of operating over token-level primitives, the agent reasons over higher-level latent action units ( $z_t$ ). Each latent action subsumes a sequence of low-level actions, effectively shortening the decision horizon from $H_{eff} = \sum |a_t|$ to $H_{lat} = \sum |z_t|$ .
Executability Constraint: A critical challenge is balancing abstraction with executability. LAR addresses this by strictly preserving high-entropy, parameter-rich inputs (e.g., specific search queries, numerical entities) in the explicit output space. Only low-entropy, structurally recurring patterns (e.g., system prompts, tool invocation syntax, recurring configurations) are abstracted into latent units.
Transition Equivalence: A latent action is defined as an equivalence class of action segments that induce transition-equivalent behaviors. Formally, two segments are transition-equivalent if, for any preceding history, they lead to the same task-relevant outcomes.

Implementation Pipeline

LAR is realized through a four-stage pipeline:

Identification: The system identifies transition-equivalent action segments from agent trajectories using an entropy-based filter. Segments with low next-token entropy (predictable continuations) and high frequency are candidates. High-entropy segments are excluded to ensure executability.
Vocabulary Construction: Identified segments are assigned dedicated vocabulary symbols, forming a latent action vocabulary.
Dual-Format Training Data: Original trajectories are paired with reparameterized counterparts where matching segments are replaced by latent symbols.
Trajectory-Level Distillation: A student model (base LLM + LoRA adapter + new latent embeddings) is trained to mimic a frozen teacher model (the original LLM) using KL distillation. The loss is computed only over shared content positions, forcing the new latent embeddings to encode the full semantics of the compressed segments. Only the LoRA weights and new embeddings are trainable (~0.1% of parameters).

Key Contributions

The paper outlines four primary contributions:

Significant Efficiency Gains: LAR significantly reduces the effective action horizon, leading to substantial reductions in action tokens and corresponding system-level gains in token throughput and peak GPU memory across diverse benchmarks.
Preserved Task Performance: Despite operating over a compressed space, LAR maintains or improves task success rates compared to baselines using raw textual actions. It also demonstrates the ability to transfer to held-out benchmarks without retraining.
Analysis of Abstraction Limits: The authors empirically identify a distinct performance collapse threshold. This delineates the boundary between compressible structural redundancy and essential semantic content, showing that excessive abstraction leads to abrupt failure when parameter-binding content is incorrectly abstracted.
New Perspective on Scaling: The work positions action representation learning as a critical, underexplored factor in scaling efficient LLM agent inference, offering a path complementary to advances in model architecture and hardware.

Experimental Results

The authors evaluated LAR on two backbone models (Llama-3.1-8B-Instruct and Qwen3-8B) across three benchmarks: TriviaQA (reasoning), KodCode (code generation), and Mind2Web (web interaction).

Performance vs. Efficiency: LAR consistently achieved favorable accuracy-efficiency trade-offs. For instance, on Qwen3-8B with TriviaQA, LAR improved accuracy from 67.40% (Vanilla) to 80.09% while reducing action tokens by 27.1%. On KodCode, it matched or approached strong baselines while reducing tokens by ~9-10%.
Generalization: Latent actions learned on "held-in" datasets (TriviaQA, KodCode) generalized effectively to "held-out" benchmarks (Musique, HumanEval, MBPP) without retraining, suggesting the learning of domain-level structural abstractions rather than dataset-specific artifacts.
System-Level Efficiency: LAR improved token throughput and reduced peak GPU memory usage. Crucially, latent action decoding introduces zero additional computational overhead as latent symbols are processed as standard vocabulary tokens.
Ablation Studies: Progressive abstraction experiments revealed a three-phase behavior: performance improvement with moderate abstraction, a peak at the abstraction boundary, and a sharp collapse when high-entropy, parameterized components were abstracted. This validates the necessity of the entropy filter in the identification stage.
Industrial Transfer: A case study on the OpenClaw industrial agent runtime demonstrated that LAR could be deployed as a plug-in optimization layer, replacing static system prompt scaffolding with latent tokens, yielding up to a 27% relative improvement in Exact Match scores without modifying the framework's runtime logic.

Significance and Claims

The paper claims that action representation learning is a critical and underexplored factor in scaling efficient LLM agent inference. By reparameterizing the action space, LAR directly addresses the inefficiency caused by overly fine-grained action interfaces. The authors assert that this approach allows agents to operate over a shorter effective horizon while preserving the expressiveness of the original action space and, crucially, the executability of interactions with external environments.

The work suggests that efficiency should not be viewed merely as a byproduct of shorter text generation but as a consequence of operating over a more appropriate decision representation. By treating action representation as a first-class modeling choice, LAR offers a complementary path to hardware and architectural advances, validated across model scales up to 32B and in industrial deployment scenarios. The authors emphasize that their method is conservative by design, improving efficiency on structurally regular portions of the action space while introducing no risk on rare, irregular portions, thereby avoiding the performance degradation often seen in token-pruning or context-compression methods.