Error as a Lens: Probing LLM Reasoning through Synthetic Misconception Generation

Technical Summary: Error as a Lens: Probing LLM Reasoning through Synthetic Misconception Generation

Problem Statement
Educational technology applications—including personalized tutoring, teacher training, and education research—rely on access to targeted synthetic misconceptions. However, the acquisition of large-scale, labeled corpora of authentic student errors is severely constrained by privacy regulations, Institutional Review Board (IRB) limitations, and the ethical sensitivity of storing identifiable assessment data. While Large Language Models (LLMs) offer a scalable path to generate synthetic errors, a critical gap exists: modern LLMs can easily produce arbitrary wrong answers, but they struggle to generate errors that align with specific, specified cognitive failure modes. The central challenge is to create a framework that produces incorrect answers which are (i) genuinely wrong, (ii) consistent with a specific cognitive failure mode rather than being arbitrarily incorrect, and (iii) stable across different subjects.

Methodology: The GA/EA Framework
The authors propose a two-agent framework designed to generate taxonomy-targeted synthetic errors. The system decouples the drafting of an erroneous solution from the verification of its class consistency.

1. Error Taxonomy: The framework utilizes a five-class error taxonomy adapted from the revised Bloom's taxonomy (Anderson et al., 2001). The retained classes are:

   • E1 (Mental typo/Sloppy work): Arithmetic or transcription slips in an otherwise correct trajectory.
   • E2 (Knowledge gap): Missing definitions, terms, or formulas.
   • E3 (Misconception): Faulty mental models or absent concepts.
   • E4 (Wrong choice): Incorrect problem classification or selection of the wrong solution procedure.
   • E5 (Structural blindness): Failure to distinguish components and their interactions.
     (Note: Classes describing sub-optimal but correct answers were excluded as they do not represent incorrect final answers.)

2. Agent Architecture:

   • Generation Agent (GA): A pre-trained LLM tasked with drafting a candidate erroneous solution conditioned on a specific target error class. The authors employ a single multi-class GA rather than five specialist agents to prevent "drift" into sibling classes, as specialists lacked the cross-class context to distinguish adjacent failure modes.
   • Examination Agent (EA): A judge that evaluates the GA's draft on two axes: (a) whether the response is genuinely incorrect, and (b) whether it matches the target error class. The EA operates without seeing the GA's in-context examples to prevent surface-pattern matching.

3. Pipeline Configurations: The study evaluates eight pipeline configurations (P0–P8) varying three design axes:

   • Answer Grounding: Whether the GA is exposed to the correct solution.
   • Conditioning: Whether the GA receives expanded few-shot examples or external textbook excerpts.
   • Feedback Loop: Whether the EA provides rejection signals (including the draft and justification) back to the GA for iterative regeneration (up to five attempts).
   • EA Implementation: Comparing a prompted LLM judge against a fine-tuned BERT-base-uncased classifier (P8) trained on 1,600 human-annotated examples.

Experimental Setup
The framework was evaluated on the TheoremQA dataset, a theorem-driven QA dataset covering mathematics, physics, EE/CS, and finance.

• Tier-1 (Verified Benchmark): A sweep of 20 questions across all 9 pipelines and 3 backends (OpenAI o3/GPT-4o mix, GPT-5, GPT-5-mini) with unlimited retries and final human verification.
• Tier-2 (Scale Run): A 200-question run on GPT-5 with a 5-attempt retry cap to assess cost and latency at scale.
• Metric: The primary metric is the Targeted-Error Rate, defined as the fraction of generations that are both incorrect and consistent with the requested error class. Responses that are correct or admit ignorance (epistemic refusal) are treated as failures.

Key Results

1. Difficulty of Targeted Generation: Targeted error generation is substantially harder than free-form incorrect answer generation. The naive baseline (P0) fell 23–39 percentage points below the ceiling across all backends.
2. The "Structural Blindness" Bottleneck: Class E5 (Structural Blindness) proved the most difficult target across all configurations, particularly for weaker backends. This class requires the model to globally misframe a problem while maintaining surface-coherent reasoning, a task distinct from simple arithmetic slips (E1) or formula substitutions (E2).
3. Impact of Design Axes:
   • Answer Grounding: Providing the correct answer to the GA generally improved performance on stronger backends (GPT-5) but degraded performance on weaker ones (o3+GPT-4o), as the weaker models struggled to suppress the correct trajectory, leading to "partial-correct" reasoning that failed the class-consistency check.
   • Feedback Loop: The EA feedback loop (P3) provided the most significant and consistent improvement, especially for weaker backends. It allowed the GA to diverge from rejected drafts, a capability that simple temperature-based sampling could not reliably achieve.
   • Scaffolding: Expanded examples and external textbook excerpts (P4–P7) did not yield clear improvements over the leaner P1/P3 configurations on strong backends, and in some cases diluted the class signal.
4. Backend Disparities: The gap between backends was large and asymmetric. GPT-5 achieved an average targeted-error rate of ~0.82, while GPT-5-mini and o3+GPT-4o lagged by 12–14 points. This gap was driven by the ability to suppress the correct answer and diversify drafts under feedback, rather than raw problem-solving accuracy.
5. Fine-tuned vs. Prompted EA: The fine-tuned BERT classifier (P8) offered a deployment trade-off. It achieved near-single-pass acceptance (high efficiency) and outperformed the prompted EA on the weakest backend, but was less adaptable and did not close the E5 gap on GPT-5.

Significance and Claims
The paper makes one primary methodological contribution and one secondary empirical observation:

1. Methodological Contribution: The authors present a scalable, reusable recipe for building class-stratified synthetic error datasets in domains where authentic student corpora are unavailable. By decoupling drafting from judging and moving human supervision to the "judge" side (or using a fine-tuned classifier), the framework allows any pre-trained LLM to serve as a "substitute student" for generating controlled misconceptions.

2. Empirical Observation: Targeted error generation is fundamentally a controllability problem rather than a knowledge-acquisition problem. The results suggest that answer-grounding and iterative feedback loops are more critical for generating specific error types than providing expanded context or textbook content.

Limitations and Scope
The authors explicitly state that this framework produces taxonomy-targeted synthetic errors, not empirically validated student-like errors. The study does not measure whether the generated surface forms match what authentic students actually produce. The evaluation relies on a single taxonomy and closed-source models, and the "targeted-error rate" is presented as a complementary metric to correctness-based benchmarks, exposing behaviors (controlled failure modes) that standard accuracy metrics miss. The work positions itself as a practical instrument for building downstream educational applications, with the caveat that real-student validation remains a necessary future step.