Agents Explore but Agents Ignore: LLMs Lack Environmental Curiosity

1. Problem Statement

Current Large Language Model (LLM)-based agents are assumed to integrate environmental observations into their reasoning processes. The prevailing hypothesis is that if an agent discovers highly relevant, unexpected information (such as a pre-existing solution), it should naturally adapt its strategy to exploit this discovery.

However, the authors argue this assumption is false. They posit that current agents lack Environmental Curiosity: the capability to recognize and investigate unexpected but relevant observations in response to environmental stimuli. Instead of adapting, agents tend to ignore discovered information, adhering to fixed patterns learned during training. This creates a "discovery-interaction gap" where agents find solutions but fail to use them.

2. Methodology: Solution Injection

To quantify this gap, the authors propose a novel evaluation method called Solution Injection.

• Concept: The gold-standard solution to a task is deliberately injected directly into the agent's environment (e.g., as an executable script solution.sh or a documented API endpoint).
• Goal: To measure two distinct capabilities separately:
  1. Discovery: Does the agent find the injected solution?
  2. Interaction: Does the agent read, execute, or query the solution?
• Benchmarks: The method was applied to three diverse benchmarks:
  • Terminal-Bench: Terminal-based tasks (file manipulation, system admin).
  • SWE-Bench Verified: Software engineering tasks (GitHub issue resolution).
  • AppWorld: Everyday digital tasks via API calls (email, notes, calendars).
• Metrics:
  • pass@k: Standard task success rate.
  • discovery@k: Probability the agent executes a command surfacing the solution.
  • interaction@k: Probability the agent interacts with the solution (e.g., reads the file or calls the API) after discovery.

3. Key Contributions

1. Definition of Environmental Curiosity: Formalizing the capability to investigate unexpected observations as a distinct metric from task success.
2. Solution Injection Framework: A method to adapt existing benchmarks to measure curiosity by injecting "cheat codes" into the environment.
3. New Metrics: Introduction of discovery@k and interaction@k to decouple finding information from acting on it.
4. Identification of Limiting Factors: Systematic analysis of test-time factors (tools, compute, prompting) and training factors (data distribution) influencing curiosity.

4. Key Results

A. The Discovery-Interaction Gap

Across all three benchmarks and multiple models (including gpt-oss-120b, GLM-4.7, and Command-a), agents consistently discover injected solutions but rarely interact with them.

• Terminal-Bench: Agents discover solutions in 79–81% of runs but interact in only 37–50%.
• SWE-Bench: Discovery rates range from 53–98%, but interaction drops to 6–17%.
• AppWorld (Starkest Case): Agents discover the solution API in 97.5% of attempts, yet interact with it in only 0.53% of cases. Even when documentation explicitly states a command "returns the complete solution," agents ignore it.

B. Factors Influencing Curiosity

The authors investigated three test-time factors and one training factor:

1. Tool Availability (Scaffold): Restricting agents to a bash-only environment increases interaction rates (roughly doubling them). Adding powerful tools like str_replace_editor improves overall task performance (pass@1) but reduces curiosity. Agents with rich toolsets default to learned tool-specific patterns rather than exploring the environment.
2. Reasoning Budget: Increasing the reasoning budget (e.g., "high reasoning" modes) significantly improves interaction rates (e.g., Terminal-Bench interaction triples from 11% to 37%). However, on AppWorld, increased compute alone could not overcome the gap.
3. Prompting: Explicit instructions to "explore," "reflect," or "investigate all discovered files" improve both curiosity and task performance. The best prompts for curiosity also yielded the best results on original, unmodified benchmarks.
4. Training Distribution: Fine-tuning on narrow, in-distribution data (e.g., training only on AppWorld tasks) reduces environmental curiosity and compresses the diversity of explored solution paths. This leads to worse pass@k scaling (diminishing returns on multiple attempts) compared to models trained on broader distributions.

C. The Root Cause

The authors ruled out alternative explanations:

• Adversarial Suspicion: Agents do not ignore solutions because they suspect them to be traps (LLM-as-a-judge analysis showed 0% deliberate rejection due to suspicion).
• Incapability: Agents can use the solutions if explicitly prompted to do so (Oracle interventions showed high success rates when the solution was forced into the prompt).
• Conclusion: The issue is a habitual failure to trigger investigation. Agents operate in an "open-loop" fashion: they fetch expected information but do not revise their strategy based on unexpected observations. The training process (Supervised Fine-Tuning on expert trajectories) reinforces a rigid Action → Observation → Reasoning loop, rather than a reflective Action → Observation → Reflection on Observation → Reasoning loop.

5. Significance and Implications

• Benchmarking Limitations: Current benchmarks measuring only pass@k fail to distinguish between agents that adapt to their environment and those that execute rigid, memorized patterns. This masks a fundamental brittleness in current agentic systems.
• Training Paradigm Shift: The findings suggest that current training methods (SFT and RL) suppress environmental curiosity. To build robust agents for open-ended, novel environments, training must incentivize the reflection on unexpected observations (Equation 5 in the paper) rather than just successful task completion.
• Tool Design: Providing agents with too many specialized tools may inadvertently reduce their curiosity by encouraging reliance on specific tool-use patterns rather than environmental exploration.
• Future Directions: The paper calls for new training paradigms that teach reflective behavior and the development of metrics (like interaction@k) that prioritize process-oriented evaluation over simple outcome success.

In summary, the paper demonstrates that while modern LLM agents are excellent at finding information, they are fundamentally poor at reacting to unexpected, high-value information, a deficit rooted in their training data and inference-time configurations.