Evaluating Answer Leakage Robustness of LLM Tutors against Adversarial Student Attacks

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed as conversational tutors. However, a critical pedagogical conflict exists: LLMs are naturally trained to be "helpful" by providing direct answers, whereas effective pedagogy requires scaffolding (guiding students to derive solutions independently).

• The Core Issue: "Answer Leakage"—the premature disclosure of the final solution by the tutor.
• The Gap: Prior evaluations of LLM tutors typically assume cooperative, well-intentioned students. They fail to account for adversarial student behaviors where learners actively attempt to "jailbreak" the tutor to obtain the answer through manipulation, emotional pressure, or deceptive reasoning.
• Objective: To systematically evaluate the robustness of LLM-based tutors against these adversarial attacks, identify the most effective attack vectors, and propose defense mechanisms.

2. Methodology

The authors propose a framework for simulating multi-turn dialogues between a Tutor Agent and an Adversarial Student Agent.

A. Attack Taxonomy

The study adapts six categories of adversarial and persuasive techniques (based on Zeng et al., 2024; Chao et al., 2025) to the educational domain:

1. Direct Request: Explicitly demanding the answer.
2. Emotional Threat: Using distress, urgency, or negative consequences to coerce the tutor.
3. Intentional Wrong Answer: Deliberately providing incorrect answers to provoke correction.
4. Contextual Manipulation: Fabricating contexts where revealing the answer is logically necessary.
5. Interpersonal Influence: Using rapport or emotional support to encourage disclosure.
6. Request Shaping: Reframing the request to make revealing the answer seem reasonable.

B. Adversarial Student Agents

The paper evaluates four types of student agents to determine which best simulates a robust attacker:

1. In-Context Agents: Prompt-based agents using few-shot examples.
2. Reasoning-Augmented Agents: Agents that generate a "reasoning" step before the attack.
3. Multi-Agent Systems: A pipeline where a student generates an attack, a judge checks for leakage, and a refiner modifies the attack if it leaks the answer prematurely.
4. Fine-Tuned Adversarial Agent (Key Innovation): An LLM (Qwen2.5-7B) fine-tuned via Supervised Fine-Tuning (SFT) on a synthetic dataset of adversarial tutor-student dialogues. This agent is specifically trained to maximize tutor leakage while avoiding self-leakage.

C. Tutor Configurations & Defenses

• Models Tested: Standard SOTA models (Qwen2.5, Llama-3.1), pedagogically aligned models (TutorRL, MathDial, SocraticLM), and larger models (Qwen-72B, GPT-5).
• Defense Strategies:
  1. Reasoning Tutor: Forces the tutor to generate an intermediate Chain-of-Thought (CoT) reasoning step before responding.
  2. Multi-Agent Tutor: A pipeline where a "Refiner" agent checks the tutor's response for answer leakage and modifies it if necessary.

D. Evaluation Metrics

• Tutor Leakage Rate: Percentage of conversations where the tutor reveals the final answer.
• Student Leakage Rate: Percentage of conversations where the student solves the problem themselves (a failure of the attack, as the goal is to make the tutor reveal the answer).
• Turns to Leakage: The average number of dialogue turns before leakage occurs.
• Datasets: GSM8K (Math), MMLU (MCQ in Law, Philosophy, etc.), and HumanEval (Coding).

3. Key Contributions

1. Systematic Framework: Introduction of a standardized benchmark for evaluating LLM tutor robustness against adversarial student behavior, moving beyond cooperative assumptions.
2. Adversarial Student Agent: Development of a fine-tuned adversarial student agent that outperforms prompting-based agents. It achieves attack success rates comparable to or exceeding manually crafted prompts, serving as a core component for future benchmarks.
3. Attack Taxonomy: Identification that persuasive techniques (specifically Contextual Manipulation) are more effective at inducing leakage than direct adversarial requests.
4. Defense Validation: Demonstration that simple architectural changes (Reasoning and Multi-Agent setups) significantly reduce answer leakage.

4. Key Results

A. Vulnerability to Attacks

• High Leakage Rates: Most models are highly vulnerable. Under Contextual Manipulation, the average leakage rate was 74% across models.
• Persuasion > Adversarial: Persuasive techniques (Contextual Manipulation, Interpersonal Influence) were more effective than direct adversarial techniques (Direct Request, Emotional Threat).
• Model Differences:
  • Qwen-7B and TutorRL-7B were the most vulnerable (avg. 75% leakage).
  • Llama-8B was the most robust (avg. 40% leakage).
  • GPT-5 showed high robustness (4.58% leakage) but occasionally provided overly informative hints.
• Fine-Tuned Agent Superiority: The fine-tuned adversarial agent induced significantly higher leakage (up to 83% on TutorRL-7B) compared to in-context agents, proving that simple prompting is insufficient for robust evaluation.

B. Student Behavior Limitations

• Cooperative Failure: Standard in-context agents often failed to act adversarially; instead, they cooperated with the tutor or solved the problem themselves (Student Leakage rates up to 75%), inflating the perceived safety of the tutor.
• Fine-Tuning Success: The fine-tuned agent reduced student leakage (by ~60-70%) and forced the tutor to leak the answer, providing a more accurate stress test.

C. Defense Effectiveness

• Reasoning & Multi-Agent Defenses: Both strategies significantly reduced leakage.
  • For Qwen-32B, leakage dropped from 46% to 4% (Multi-Agent) and 2% (Reasoning) under manual prompts.
  • Under the fine-tuned agent, leakage for Qwen-32B dropped from 70% to 4%.
• Generalization: These defenses remained effective across domains (Math, MCQ, Coding), though coding tasks were particularly vulnerable to procedural leakage (tutors providing code snippets that converged to a full solution).

5. Significance and Implications

• Safety in Education: The paper highlights that current LLM tutors are not robust against real-world misuse where students seek shortcuts. Relying on "helpful" alignment is insufficient; specific pedagogical constraints are required.
• Benchmarking Standard: The proposed fine-tuned adversarial student agent offers a rigorous, automated standard for evaluating tutor safety, replacing unreliable manual prompting or cooperative simulations.
• Architectural Solutions: The study proves that robustness can be achieved through relatively simple architectural interventions (CoT reasoning and multi-agent refinement) rather than requiring massive retraining of the base model.
• Domain Agnosticism: The vulnerability to persuasive attacks and the efficacy of defenses generalize across math, humanities, and coding, suggesting a universal challenge for LLM tutors.

Conclusion

The paper concludes that LLM-based tutors are currently fragile against adversarial student behavior, particularly persuasive manipulation. While pedagogically aligned models help, they are not immune. The authors advocate for the adoption of fine-tuned adversarial agents as a standard evaluation tool and recommend reasoning-based and multi-agent defense architectures to ensure tutors adhere to pedagogical principles in adversarial scenarios.