Technical Summary: What Drives Interactive Improvement from Feedback?

Problem Statement

As Large Language Models (LMs) are increasingly deployed as interactive agents that act, observe, and revise, a critical question arises: When does natural-language feedback actually drive improvement beyond the gains obtainable from repeated attempts alone?

In multi-turn settings, higher final accuracy can be conflated with several mechanisms:

1. Useful Feedback: The agent correctly interprets and acts on diagnostic information.
2. Resampling/Computation: The agent simply generates more attempts, benefiting from increased test-time compute (e.g., Best-of-N sampling or Chain-of-Thought).
3. Format Correction: The agent fixes output formatting errors rather than solving the underlying logical problem.

Current literature often treats multi-turn improvement as evidence of feedback utility without disentangling these factors. Furthermore, it remains unclear whether the bottleneck for interactive improvement lies in the teacher's ability to generate useful feedback or the student's ability to utilize it.

Methodology

The authors introduce a controlled student-teacher protocol designed to isolate the specific contribution of feedback from repeated sampling.

Experimental Setup

• Protocol: A "didactic" interaction loop where a student model attempts a problem. If incorrect, a teacher model provides natural-language feedback (without revealing the final answer), and the student revises its attempt. This repeats up to $K=10$ turns.
• Environments: Four verifiable reasoning benchmarks were used:
  • Omni-MATH: Olympiad-level mathematics.
  • Codeforces: Competitive programming.
  • BBEH Linguini: A task requiring learning new rules.
  • ARC-AGI1: Abstract reasoning grid puzzles.
• Models: Thirteen open-weight models across four families (Gemma4, Llama3, Qwen3.5, GPT-OSS) ranging from 1B to 120B parameters.
• Dense Matrix Design: Every model was evaluated in both the student and teacher roles across all environments, creating a $13 \times 13$ interaction matrix. This allows for the marginalization of student and teacher effects.
• Baselines & Controls:
  • Self-refinement: The student attempts to solve the problem again without any external feedback (controlling for repeated attempts).
  • Self-feedback: The student generates its own feedback (controlling for the presence of a feedback message).
  • Best Feedback: The student is paired with the strongest available teacher.
• Ablations: The study varied interaction history length ($h \in \{1..5\}$) and teacher access to privileged information (e.g., ground-truth solutions, execution traces) to test their selective impact.

Metrics

• acc@K: Accuracy at turn $K$.
• Raw/Normalized Gain: $acc@K - acc@1$ and $\frac{acc@K - acc@1}{1 - acc@1}$, isolating improvement from the initial attempt.
• AUC: Area Under the Curve of cumulative accuracy over turns.

Key Contributions

1. Controlled Evaluation Framework: A zero-shot student-teacher evaluation protocol that separates feedback-specific gains from the benefits of repeated sampling and self-refinement.
2. Disentanglement of Mechanisms: Empirical evidence showing that multi-turn improvement is not synonymous with feedback use; self-feedback often adds little over unguided self-refinement.
3. Student-Centric Bottleneck: A demonstration that interactive performance is primarily driven by the student's ability to use feedback rather than the teacher's identity, although teacher choice remains significant for a fixed student.
4. Selective Value of Privileged Information: Findings that longer histories and privileged teacher information (e.g., ground-truth answers) are not universally beneficial but act as targeted interventions dependent on task difficulty and model capability.

Key Results

1. Feedback-Specific Gains are Limited Without Strong Guidance

• Self-feedback vs. Self-refinement: Adding a feedback message generated by the same model (self-feedback) produced inconsistent results compared to unguided self-refinement. In some environments (e.g., BBEH Linguini), self-feedback offered no advantage over simple retrying.
• External Teacher Impact: The strongest gains came from pairing students with high-quality external teachers. "Best feedback" conditions added 9.2 to 16.6 percentage points in $acc@10$ over self-refinement, suggesting that useful feedback must provide corrective information beyond generic retrying.
• Front-Loaded Gains: Marginal improvements were heavily front-loaded; the first feedback-conditioned attempt accounted for 30–58% of total gains, with diminishing returns in later turns.

2. Interactive Performance is Student-Dependent

• Student Identity Dominance: In the dense model matrix, the student's identity explained 77.1%–96.5% of the variation in pair-level gains. The teacher's identity explained only 1.3%–12.4%.
• Distinct Capabilities: A model's initial solving ability ($acc@1$) does not predict its ability to improve from feedback ($ngain$). For example, GPT-OSS-120B often started with higher $acc@1$ but showed lower feedback adherence than Gemma4-31B in certain environments.
• Teacher Competence: A teacher's single-turn task performance is only a partial predictor of its interactive teaching ability. In 28.9%–41.4% of comparable pairs, a weaker solver acted as a more effective teacher for a specific student.

3. Context and Privileged Information are Selective

• History Length: Increasing visible history from 1 to 5 turns provided modest gains (e.g., +0.094 AUC on Omni-MATH) but was highly dependent on model capacity. Stronger teachers leveraged history better to diagnose repeated failures.
• Privileged Information: Giving teachers access to ground-truth answers or solutions did not uniformly improve outcomes.
  • BBEH Linguini: Showed the largest benefit (AUC increased from 0.151 to 0.331 with answer access).
  • Codeforces/Omni-MATH: Benefits were marginal or non-existent, likely because the teacher already had sufficient context (e.g., execution traces) or the task structure made the answer less critical for diagnosis.
• Leakage: Even with privileged access, information leakage from teacher to student was low (<5%), indicating that teachers generally adhered to the constraint of not revealing the solution.

Significance and Claims

The paper argues that the field must shift how it evaluates feedback-based agents:

• Evaluation Rigor: Multi-turn gains should always be benchmarked against repeated-attempt baselines (self-refinement) to ensure improvements are truly feedback-driven.
• Bottleneck Identification: The primary constraint in interactive improvement is often the student's ability to act on feedback, not the mere availability of feedback or the strength of the teacher.
• Strategic Deployment: Longer interaction histories and privileged teacher information should be treated as targeted interventions for specific bottlenecks rather than default improvements, as they increase computational cost and context pressure without guaranteed returns.

The authors conclude that while feedback is a powerful tool, its efficacy is contingent on the student's capacity to translate diagnostic information into actionable corrections, a capability that varies significantly across model architectures and sizes.