Technical Summary: Evidence-Diagnosed Intervention Training (EDIT) for Rule-Faithful LLM Grading

1. Problem Statement

Reliable rubric grading for student exam responses is a complex rule-application task that requires more than accurate final score prediction. A grader must ground every judgment in the explicit mark scheme and specific evidence from the student's answer. While Large Language Models (LLMs) are increasingly deployed as automatic graders, training them to grade reliably remains challenging.

Existing approaches face significant limitations in this domain:

• Outcome-based Reinforcement Learning (RL): Methods like GRPO assign a single reward to the entire reasoning trajectory. They fail to identify which specific intermediate reasoning step caused an incorrect mark, making it difficult to correct the model's decision process.
• Self-contained Reasoning Methods: Techniques designed for tasks like mathematics (e.g., token-reallocation or intervention-style training) struggle here because rubric grading requires external grounding. The model must anchor judgments to an external policy (the rubric) and the student's text, rather than relying solely on internal logical consistency.
• Uncertainty Reduction: Rule-based reasoning is a process of reducing uncertainty. The model's belief about the final mark should progressively converge. Current methods do not control this trajectory; outcome-only RL ignores harmful belief shifts occurring mid-reasoning, while rigid token-level constraints may over-penalize benign exploratory phrasing.

2. Methodology: EDIT Framework

The authors propose Evidence-Diagnosed Intervention Training (EDIT), a two-phase framework designed to train rubric-faithful LLM graders. EDIT utilizes the model's internal decision signals to locate failures, revise them locally, and calibrate the evolution of the model's belief during reasoning.

Phase 1: EDIT-SFT (Supervised Fine-Tuning with Internal-State Diagnostics)

This phase constructs a supervised training pool by revising small, local parts of incorrect reasoning trajectories. It replaces unreliable prompt-based self-audits with Internal State Diagnostics.

1. Internal State Signal Extraction:
   • Posterior Belief Probe: At each reasoning step boundary, the model estimates its posterior belief over the final integer mark. The change in expected absolute error ($\Delta \epsilon_k$) between steps identifies whether a step moved the belief closer to or farther from the gold mark.
   • Mask-based Grounding Audit: The system computes support scores to measure how much the likelihood of a reasoning step depends on the marking rules, the student answer, and the reasoning prefix. Steps with low grounding scores (weakly anchored in evidence or rules) are flagged.
2. Candidate Selection: The system selects steps that most strongly moved the model's belief away from the gold mark (negative $\Delta \epsilon_k$) and are weakly grounded.
3. Atomic Revision: Selected steps are revised into 1–3 new sub-steps. This revision is guided by a rubric checklist (generated using the gold mark as privileged context during training) but is constrained by locality. The revision must fix the local error without destroying the coherence of the original reasoning chain or simply copying the gold answer.
4. Training: Successful revisions that lead to the correct mark are aggregated with naturally correct rollouts to train the policy via SFT.

Phase 2: EDIT-RL (Reinforcement Learning with Belief-Guided Reward Shaping)

This phase calibrates the SFT-trained policy using Group-Relative Policy Optimization (GRPO), augmented with a specific reward shaping mechanism.

• Distance-Aware Outcome Reward: A dense reward is assigned based on the distance between the predicted mark and the gold mark.
• Belief-Guided Reward Shaping: To manage the uncertainty reduction process, a penalty is added for large mid-trajectory belief drifts. The reward function penalizes the model if its posterior belief at any intermediate step deviates too far from the gold mark (beyond a tolerance threshold $\beta_{drift}$), while allowing small fluctuations for benign exploration.
• Implementation: The posterior probe used for shaping is run on a frozen checkpoint of the EDIT-SFT policy to maintain a stationary signal and prevent reward hacking.

3. Key Contributions

The paper outlines three primary contributions:

1. EDIT Framework: A novel two-phase training framework that adapts credit assignment for externally grounded rubric grading, moving beyond closed-system reasoning tasks.
2. Internal State Diagnostics: The replacement of prompt-based self-audits with robust posterior belief and grounding signals to locate flawed reasoning steps. This is coupled with rubric-checklist context to support local atomic revisions without imposing rigid output templates.
3. Belief-Guided Reward Shaping: A calibration RL mechanism that penalizes severe mid-trajectory belief drifts while tolerating benign exploration, thereby improving grading accuracy and out-of-domain generalization.

4. Experimental Results

The authors evaluated EDIT on two real-world, multi-subject benchmarks: SAS (a Chinese short-answer scoring benchmark with History, Geography, and Physics) and Private-Science (a proprietary GCSE-level dataset for Biology and Physics).

• Performance: EDIT consistently outperformed strong baselines, including standard SFT, GRPO, DGPO, and InT (Intervention Training), across both in-domain (ID) and out-of-domain (OOD) splits.
  • On the SAS benchmark (fully OOD), EDIT-RL achieved the best average macro-QWK (0.794), significantly outperforming the base model (0.727) and other RL baselines.
  • On Private-Science, EDIT-SFT showed steady gains across both ID and OOD splits, whereas other methods (like InT-RL) showed signs of overfitting to specific question types in the OOD setting.
• Ablation Studies:
  • Removing the internal-signal localizer caused the largest performance drop, confirming that selecting the correct steps for revision is critical.
  • Removing the rubric-checklist context increased variance in grading behavior.
  • Removing atomic locality constraints resulted in slight degradation, suggesting that preserving the original reasoning structure is beneficial.
  • Applying EDIT-RL directly to the base model did not yield the same OOD improvements as applying it after EDIT-SFT, indicating that the SFT phase is a necessary prerequisite for effective belief-guided calibration.

5. Significance and Claims

The paper claims that EDIT addresses a specific gap in automated grading: the need for models to not only predict a score but to do so through a reasoning process that is faithful to external rules and grounded in student evidence.

• Rule-Faithfulness: The authors demonstrate that EDIT improves "Rule-Sensitivity Ratio" (RSR), showing that the model's score changes more accurately reflect deterministic edits to the mark scheme compared to baseline models.
• Generalization: The belief-guided reward shaping is claimed to be particularly effective for out-of-domain generalization, as it prevents the model from developing unstable reasoning trajectories when encountering unseen questions.
• Methodological Shift: The work argues for a shift from outcome-only optimization to process-aware training that monitors and corrects the internal decision trajectory of the grader.

The authors maintain a modest stance, noting that while EDIT significantly outperforms baselines, performance varies by subject (e.g., Geography showed different trends than History or Physics), suggesting that different subjects may benefit from different forms of adaptation. The framework is presented as a robust solution for training rubric-faithful graders in real-world, multi-subject educational settings.