🤖 AI

Looped Language Models Improve Compositional Tool Calling

This paper demonstrates that looped language models, particularly when employing adaptive inference to allocate computation dynamically, significantly enhance performance in compositional tool-calling scenarios by effectively coordinating multiple API calls, maintaining state, and managing dependencies.

Original authors: Andrei Cristian Popescu, Haitz Sáez de Ocáriz Borde, Pietro Liò

Published 2026-08-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Andrei Cristian Popescu, Haitz Sáez de Ocáriz Borde, Pietro Liò

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the rapidly evolving landscape of artificial intelligence, a significant shift is occurring from models that simply answer questions to systems that take action. These new systems, often called agents, are designed to use external tools—like software programs or databases—to solve complex problems. Imagine a digital assistant that doesn't just tell you the weather but actually books a flight, checks your calendar, and sends an email to confirm the trip. For these agents to work, they must be able to chain together multiple steps, remembering the result of one action to inform the next. This requires a form of structured thinking where the system plans a sequence, executes it, and adapts if something goes wrong. The challenge for researchers has been figuring out how to make these large language models reliable enough to handle such multi-step workflows without getting lost or making errors in the middle of the process.

A team of researchers at the University of Cambridge has investigated a specific architectural approach to solve this problem, focusing on a design known as a "looped" language model. Unlike standard models that process information in a single, straight pass from input to output, these looped models allow the system to revisit and refine its internal thoughts multiple times before committing to an answer. Think of it as a person solving a difficult puzzle who pauses to double-check their work after every few moves, rather than rushing to the finish line in one go. The researchers wanted to know if this extra time spent thinking—this iterative refinement—would help the models get better at coordinating complex tool use, such as calling multiple APIs in the correct order or managing dependencies where one tool's output becomes another tool's input.

To test this, the team conducted a series of controlled experiments using three different benchmarks designed to measure tool-calling abilities. These tests ranged from simple tasks, like selecting a single function to answer a question, to highly complex scenarios requiring the model to generate multiple independent calls or manage nested sequences where the output of one call is strictly required for the next. They compared standard models against both "native" looped models, which were built with this iterative capability from the start, and "retrofitted" models, where the looped behavior was added to existing standard models. The researchers trained these models on the same datasets and with the same optimization settings to ensure a fair comparison, isolating the effect of the looping mechanism itself.

The results suggest that giving the model more time to think through its decisions significantly improves its ability to handle complex, multi-step tasks. On benchmarks requiring the model to coordinate multiple tool calls or manage dependencies between them, the looped models consistently outperformed their non-looped counterparts. The improvement was most pronounced in tasks where the model had to maintain a structured plan across several decisions, such as when the output of a first tool call needed to be passed as an argument to a second. In these scenarios, the ability to refine its internal representation of the task allowed the model to correct errors in function selection, argument grounding, and the ordering of calls. For instance, in one observed case, a model initially failed to recognize a required dependency, but after a few rounds of internal refinement, it successfully identified the missing link and generated the correct sequence of actions.

However, the benefits were not uniform across all types of tasks. When the task involved a single, isolated tool call, the advantage of the looped approach was much smaller and varied depending on the specific model. This indicates that the extra computational effort is most valuable when the problem requires planning and coordination rather than simple retrieval. The researchers also explored whether this extra thinking time could be used more efficiently. They found that by allowing the model to decide dynamically how many times it needed to loop before stopping, they could achieve nearly the same high performance as using a fixed, deep number of loops, but with significantly less computational cost. This adaptive approach means the model can spend more time on difficult, complex problems while moving quickly through simpler ones.

The study concludes that looped language models represent a promising foundation for building more reliable agentic systems. The findings suggest that the ability to iteratively refine latent representations helps models preserve the structure of complex workflows, making them better at planning, coordinating, and executing compositional tool use. While the retrofitted models showed improvement, the native looped models generally performed better on deeply nested tasks, hinting that the way a model is pre-trained shapes how well it can benefit from this type of recursive refinement. Ultimately, the research points toward a future where AI agents can dynamically allocate their computational resources, thinking deeper only when the complexity of the task demands it, thereby achieving a more efficient balance between speed and accuracy.

Technical Summary: Looped Language Models for Compositional Tool Calling

Problem Statement

Pretrained language models are increasingly deployed as decision-making components in agentic systems, tasked with selecting and invoking external tools to execute user-directed tasks. While simple requests may require only a single function call, complex tasks necessitate compositional tool use: the ability to coordinate multiple API calls, maintain intermediate state, and manage sequential or parallel dependencies. Current models often struggle to construct and preserve structured action sequences across multiple decisions, particularly when outputs from earlier calls must serve as inputs for later ones.

While looped language models (which repeatedly apply a shared Transformer block to refine latent representations before generating tokens) have shown promise on reasoning benchmarks, their potential for enhancing agentic tool use remains largely unexplored. This paper investigates whether iterative latent computation improves performance in compositional tool-calling settings where models must preserve dependencies and coordinate multi-step workflows.

Methodology

The authors evaluate looped language models across three distinct benchmarks: API-Bank, BFCL v3, and NESTful. The study employs two complementary experimental settings to isolate the effects of architecture and training:

Native Looped Models: The authors compare Ouro-1.4B and Ouro-2.6B (native looped architectures) against standard Transformer baselines from the Qwen3 and Llama families. These comparisons are controlled via matched supervised fine-tuning (SFT) recipes using the Hermes function-calling dataset.
Retrofitted Looped Models: To more directly isolate the architectural contribution of recurrence, the authors retrofit OLMo-2-1B and Llama-3.2-1B with looped computation. These models share backbones with their non-looped parents but differ in that a subset of layers is shared across recurrent iterations.

Training and Inference Protocols:

Training: All controlled models are fine-tuned on the Hermes dataset. Native Ouro models learn a per-token halting policy to enable adaptive computation. Retrofitted models are trained with randomly sampled recurrence depths (Poisson-lognormal distribution) and do not learn an exit policy.
Inference: The study evaluates both fixed-depth inference (varying the number of loop iterations $T$ ) and adaptive inference (using a learned exit gate to terminate loops based on confidence thresholds).
Benchmarks:
- BFCL v3: Evaluates function selection and independent multi-call generation (Simple, Multiple, Parallel, Parallel-Multiple categories).
- NESTful: Focuses on hierarchical, nested workflows where outputs of earlier calls are consumed as arguments for later calls (dependency-aware execution).
- API-Bank: Focuses on isolated API invocation, emphasizing correct tool selection and argument grounding for single calls.

Key Contributions

Assessment of Iterative Computation: The paper provides a controlled assessment of whether iterative latent computation improves compositional tool use, distinguishing gains from recurrence itself versus training artifacts.
Comprehensive Evaluation: The study covers both native and retrofitted looped models across three diverse benchmarks, utilizing matched fine-tuning and shared-backbone comparisons.
Depth and Adaptivity Analysis: The authors demonstrate that increasing recurrent depth generally improves performance on multi-step tasks, while adaptive inference achieves a superior compute-performance trade-off by allocating computation only when necessary.
Architectural Insights: The results suggest that recurrent computation is particularly beneficial when action selection must preserve structure across multiple decisions, specifically in dependency-aware workflows.

Results

Compositional vs. Isolated Tasks: Looped models consistently outperform non-looped baselines on compositional tasks (multi-call and dependency-aware). On BFCL, the largest gains appear in the Parallel and Parallel-Multiple categories. On NESTful, looped models show significant improvements in Win Rate and full-sequence accuracy. Conversely, gains on API-Bank (predominantly single-call tasks) are smaller and more model-dependent, suggesting that recurrence offers diminishing returns for isolated API invocations.
Effect of Recurrent Depth: Accuracy on multi-step tool use generally increases with recurrent depth. For smaller models (e.g., Ouro-1.4B, retrofitted Llama), performance on complex tasks continues to improve with additional iterations before plateauing. Larger models (e.g., Ouro-2.6B) tend to reach near-saturation with fewer iterations.
Adaptive Inference: Adaptive stopping mechanisms recover most of the performance gains of deeper fixed-depth inference while using fewer average loop iterations. This yields a more favorable compute-performance frontier, particularly on BFCL and NESTful.
Native vs. Retrofitted: While retrofitting recurrence improves several compositional tasks, native recurrent models (Ouro) generally outperform retrofitted models on deeply nested workflows. This suggests that the effectiveness of recurrent refinement may depend on how representations are shaped during pretraining.
Qualitative Improvements: Analysis of prediction trajectories reveals that iterative computation corrects semantic errors in call structure and dependency binding (e.g., recovering missing dependent calls or fixing invalid variable references) rather than merely refining output formatting.

Significance and Claims

The paper concludes that looped language models are a promising architecture for agentic systems requiring reliable planning, coordination, and execution of compositional tool use workflows. The authors claim that the primary benefit of looped computation lies in its ability to refine structured decision-making over multiple steps, specifically when tasks require maintaining dependencies across sequential or parallel API calls.

The study posits that while fixed-depth inference improves performance, adaptive inference is critical for practical deployment, allowing models to dynamically allocate computational resources based on task difficulty. The authors note that their conclusions are drawn from controlled, static, single-turn settings and acknowledge that extending this analysis to live, multi-turn environments where models must recover from execution failures remains future work.