Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents

Technical Summary: Think Short, Defer Smart (TSDS)

Problem Definition
Large Language Model (LLM) agents operating under the ReAct paradigm are increasingly deployed in resource-constrained edge environments (e.g., robotics, industrial control). These agents face a critical trade-off: they must manage a limited "reasoning budget" (compute and tokens) while maintaining reliability. Current approaches often suffer from two inefficiencies:

1. Overthinking: Agents frequently generate thought tokens well beyond the point where the intended action has stabilized, wasting compute. Furthermore, excessive reasoning can lead to "overconfidence," where the model suppresses uncertainty signals, causing it to commit to suboptimal actions rather than seeking help.
2. Inefficient Deferral: Existing deferral mechanisms (escalating to a larger cloud model) often rely on heuristics or single-objective calibration. They fail to account for the interplay between stopping reasoning early and the resulting uncertainty scores, leading to either excessive cloud calls or unsafe edge execution.

The goal is to minimize edge thinking costs while guaranteeing that the expected episode reward remains above a user-specified floor ($R_{min}$) and the cloud deferral rate stays within a budget ($C_{max}^D$).

Methodology: Think Short, Defer Smart (TSDS)
The authors propose TSDS, a framework that synergistically integrates two mechanisms, jointly calibrated via a multi-objective Learn-Then-Test (LTT) procedure:

1. Thought Calibration (Convergence Probe):

   • A lightweight probe $\phi$ monitors the hidden representations of the edge agent during reasoning.
   • The probe is trained offline to detect when the agent's reasoning trace has converged to a stable action (i.e., truncating the thought at this point yields the same action as the full trace).
   • Once the probe score exceeds a threshold $\lambda_L$, generation halts immediately, preventing "post-convergence waste" and mitigating the risk of overconfidence.

2. Uncertainty-Aware Deferral:

   • Upon extracting the action from the truncated thought, an uncertainty score (specifically Perplexity, PPL) is calculated.
   • If the uncertainty exceeds a calibrated threshold $\lambda_D$, the decision is deferred to a more capable cloud model ($\pi_{cloud}$).
   • Crucially, the paper observes that early truncation preserves meaningful uncertainty signals that might otherwise be suppressed by overthinking, making the deferral mechanism more effective.

3. Joint Calibration via Multi-Objective LTT:

   • The thresholds $\lambda = (\lambda_L, \lambda_D)$ are selected using a Learn-Then-Test (LTT) procedure.
   • This approach treats hyperparameter selection as a multiple hypothesis testing problem, providing distribution-free, finite-sample guarantees.
   • It ensures that the selected operating point satisfies both the reward constraint ($R \ge R_{min}$) and the deferral budget constraint ($C_D \le C_{max}^D$) with high probability ($1-\delta$).

Key Contributions

• Synergistic Integration: The paper demonstrates that thought calibration and cloud deferral are not independent; truncating reasoning early actually improves the efficacy of uncertainty-based deferral by preventing the model from becoming overconfident in suboptimal paths.
• Joint Optimization: Unlike prior work that calibrates stopping or deferral separately, TSDS jointly optimizes both thresholds to minimize compute while strictly adhering to safety and performance constraints.
• Theoretical Guarantees: The use of multi-objective LTT provides rigorous, finite-sample guarantees on both reward and cloud-call rates, a significant advancement over heuristic threshold selection.

Experimental Results
TSDS was evaluated on four benchmarks: GSM8K (arithmetic), HotpotQA (multi-hop QA), MBPP (code generation), and a simulated household robot planning task.

• Compute Reduction: TSDS reduced per-episode thinking compute by 43%–73% compared to deferral-only baselines (like ReDAct-CD) across HotpotQA, MBPP, and the household robot task.
• Constraint Satisfaction: Across all benchmarks, TSDS successfully maintained certified reward floors and cloud-call rate budgets, whereas baselines either failed to meet reward guarantees (when truncating without deferral) or exceeded cloud budgets (when deferring without truncation).
• Specific Findings:
  • In code generation (MBPP), TSDS used 64% fewer tokens than baselines while maintaining reward.
  • In multi-step tasks (HotpotQA, Household Robot), the synergy was most pronounced; truncation alone collapsed performance, and deferral alone was inefficient. TSDS balanced these to achieve high rewards with minimal tokens.
  • In single-step tasks (GSM8K), TSDS achieved higher accuracy than deferral-only baselines while using significantly fewer tokens.

Significance and Claims
The paper claims that TSDS offers a robust solution for deploying LLM agents at the edge by addressing the "overthinking" problem and the "overconfidence" trap. The authors emphasize that the joint calibration of stopping and deferral is essential: truncating reasoning leads to detecting uncertainty signals at their most meaningful state, while the deferral mechanism acts as a safety net for the genuinely difficult steps. The work provides a pathway to reliable, resource-efficient agentic systems that can operate within strict latency and bandwidth constraints without sacrificing task performance.