Probing Embodied LLMs: When Higher Observation Fidelity Hurts Problem Solving

Technical Summary: Probing Embodied LLMs: When Higher Observation Fidelity Hurts Problem Solving

Problem Statement
Large Language Models (LLMs) are increasingly deployed as cognitive components for robotic systems operating in unstructured environments. However, their opaque decision-making processes make it difficult to distinguish between genuine reasoning capabilities and accidental success in closed-loop embodied tasks. Performance metrics, such as aggregate success rates, often fail to reveal whether an agent solved a task through coherent inference, accidental exploration, or an interaction between perceptual errors and reasoning failures. Specifically, it remains unclear how the fidelity of observational inputs (from raw sensory data to perfect symbolic ground truth) influences the behavioral mechanisms of embodied LLM agents.

Methodology
The authors employ an empirical AI methodology aligned with system identification, treating the LLM agent as an opaque system to be probed through controlled interventions.

• Task: The study utilizes the "Lockbox," a sequential mechanical puzzle consisting of four joints with hidden interdependencies. The objective is to unlock the box by manipulating joints in a specific order. This task requires integrating observations over time and inferring latent state dependencies.
• Experimental Setup:
  • Physical Setup: A Franka Emika Panda robotic arm with a soft end-effector manipulates a physical Lockbox. The system uses an RGB-D camera and force-torque sensors.
  • Simulation: A controlled simulation environment isolates the reasoning problem from physical actuation variability, allowing for systematic perturbation of the observation channel.
• Interventions: The study varies the observation fidelity provided to the LLM (OpenAI GPT-o1 in physical experiments; GPT-4o in simulation) across three modalities:
  1. RGB: Raw visual images.
  2. RGB-D: Visual images augmented with depth data.
  3. Ground-Truth Symbolic State: Explicit textual descriptions of joint states, bypassing the vision pipeline entirely.
• Probing Mechanism: In simulation, the authors introduce controlled observational noise by randomly flipping the perceived outcome of actions (state-flips) with varying probabilities (0% to 60%) to test the impact of stochasticity on performance.

Key Results
The study reveals a counterintuitive relationship between observation fidelity and task performance:

1. Fidelity vs. Performance: Contrary to the assumption that more accurate observations improve reasoning, agents performed best with raw RGB input and worst with perfect ground-truth symbolic states. The RGB-D modality yielded intermediate results.
   • GPT-o1 achieved an 80% success rate with RGB input in 11 steps, whereas the ground-truth condition required 15 steps to reach the same success rate.
   • A human-inspired heuristic achieved 100% success in 9 steps, highlighting the inefficiency of the LLMs regardless of input.
2. Misinterpretation Analysis:
   • Visual modalities (RGB/RGB-D) exhibited high state misinterpretation rates (approx. 21–27%), suggesting inconsistent perceptual grounding.
   • Surprisingly, the ground-truth condition also exhibited a ~5% misinterpretation rate, attributed to the model "hallucinating" state changes despite receiving explicit, correct data.
3. Non-Monotonic Noise Response: In simulation, introducing moderate noise (random state flips) improved performance relative to the noise-free baseline.
   • Success rates peaked at a 40% state-flip probability, achieving a 2.85-fold increase over the noise-free baseline.
   • Performance degraded again at higher noise levels (>40%).
4. Mechanism of Improvement: The performance gain under moderate noise is linked to a reduction in repetitive action loops.
   • Action loops (repeating subsequences of actions without progress) were negatively correlated with success.
   • Perfect observations allowed the agent to enter and remain in these loops when actions failed to produce expected changes.
   • Moderate perceptual noise disrupted these loops, forcing the agent to explore alternative actions, thereby breaking the cycle of failure.

Key Contributions

• Behavioral Probing Framework: The paper establishes a methodology for evaluating embodied LLMs by perturbing observation channels to reveal underlying behavioral mechanisms rather than relying solely on outcome-based metrics.
• Counterintuitive Fidelity Findings: It demonstrates that higher observation fidelity can actively hinder problem-solving in embodied LLMs, challenging the standard assumption that "better data" leads to "better reasoning."
• Identification of Failure Modes: The study identifies "repetitive action loops" as a critical failure mode in embodied LLMs and shows that perceptual errors can inadvertently mitigate this issue by disrupting recursive decision patterns.

Significance and Claims
The authors claim that success rates alone are insufficient for evaluating embodied LLMs. Measured performance may reflect an "accidental interplay" between perceptual errors and reasoning failures rather than robust problem-solving capabilities.

The paper argues that in closed-loop interactions, observations are not merely passive inputs but actively shape decision-making dynamics. Consequently, what appears as superior performance under lower-fidelity inputs should not be interpreted as evidence of stronger reasoning. Instead, it may reflect a side effect where compounding perceptual errors happen to disrupt specific failure modes (like action loops) that are sustained by accurate feedback.

The authors conclude that rigorous evaluation of embodied LLMs requires real-world closed-loop benchmarks combined with controlled behavioral interventions to disentangle perceptual limitations from reasoning capabilities. They note that their findings are specific to the models and task evaluated and call for future work to determine if these behaviors generalize across different model architectures and providers.