Confidence Estimation in Automatic Short Answer Grading with LLMs

1. Problem Statement

Automatic Short Answer Grading (ASAG) using Large Language Models (LLMs) has shown strong performance without task-specific fine-tuning and can generate synthetic feedback. However, LLM-based grading is not perfect due to linguistic variability, ambiguous scoring criteria, and inherent human grader disagreement.

In Human-in-the-Loop (HITL) educational assessment, systems must decide which responses to grade automatically and which to defer to human experts. This requires reliable confidence estimation. Current challenges include:

• Insufficient Study: Most existing confidence estimation research focuses on Pre-trained Language Models (PLMs), not generative LLMs.
• Model-Centric Bias: Existing methods rely solely on model-based signals (epistemic uncertainty), ignoring dataset-derived uncertainty (aleatoric uncertainty) arising from ambiguous or underspecified student responses.
• Lack of Reliability: Confidence scores are often used for simple thresholding without rigorous reliability analysis, leading to overconfidence or miscalibration in critical educational decisions.

2. Methodology

The authors propose a Hybrid Confidence Framework that integrates model-based confidence signals with an explicit estimate of dataset-derived aleatoric uncertainty.

A. Model-Based Confidence Signals (Epistemic Uncertainty)

The study evaluates three distinct strategies for extracting confidence from an LLM (specifically gpt-oss-20b):

1. Verbalizing: Prompting the model to explicitly output a confidence score (probability) alongside its grading decision.
2. Latent: Calculating the normalized log-likelihood of the predicted label token using the model's internal probability distribution (softmax over label tokens).
3. Consistency: Sampling multiple predictions at varying temperatures (0.2 to 1.0) and measuring the frequency with which the model returns the same label (stability).

Note: All scores are oriented so that higher values indicate higher confidence that a response is correct.

B. Dataset-Derived Uncertainty (Aleatoric Uncertainty)

To capture ambiguity inherent in the data (e.g., responses that are semantically similar but have conflicting human labels), the authors operationalize aleatoric uncertainty via semantic heterogeneity:

1. Embedding: Student responses are embedded using all-MiniLM-L6-v2.
2. Clustering: Agglomerative hierarchical clustering (Ward linkage) groups semantically similar responses.
3. Heterogeneity Quantification: For each cluster, the Shannon entropy of the label distribution is calculated. High entropy indicates that semantically similar responses receive different scores from human graders, signifying high ambiguity.
4. Assignment: Test responses are assigned the entropy value of their nearest cluster centroid.

C. Confidence Fusion

The model-based signals ($s_{verb}, s_{lat}, s_{cons}$) and the aleatoric uncertainty proxy ($s_{alea}$) are combined into a feature vector.

• Classifier: A Random Forest classifier is trained to map these features to the probability of a correct grade.
• Calibration: The output probabilities are post-hoc calibrated using Platt scaling (via 5-fold cross-validation) to ensure the predicted probabilities match empirical frequencies.
• Baseline: A "Hybrid (without aleatoric)" variant is also trained to isolate the contribution of dataset-derived uncertainty.

D. Evaluation Metrics

• Selective Prediction: Measured via ROC curves (AUROC) and Accuracy-Rejection Curves (AUARC). This assesses the ability to filter out low-confidence errors.
• Reliability: Measured via Reliability Diagrams, Brier Score, Expected Calibration Error (ECE), and Maximum Calibration Error (MCE). This assesses whether a predicted confidence of $p$ actually corresponds to $p$% accuracy.

3. Key Contributions

1. Systematic Comparison: The first systematic study comparing verbalizing, latent, and consistency-based confidence strategies specifically for ASAG with LLMs.
2. Hybrid Framework: Proposal of a novel method that fuses model-based confidence with semantic heterogeneity to estimate aleatoric uncertainty, addressing the limitation of model-only approaches.
3. Empirical Validation: Demonstration that hybrid confidence scores significantly outperform single-source methods in both selective grading accuracy and calibration reliability, providing a principled basis for HITL assessment.

4. Experimental Results

The experiments were conducted on SciEntsBank, a dataset of ~11,000 science short answers.

• Selective Prediction Performance:

  • Single-Source: Latent confidence performed the worst (lowest AUROC). Verbalizing and consistency performed moderately.
  • Hybrid: The hybrid model incorporating aleatoric uncertainty achieved the highest AUROC (0.885) and AUARC (0.876).
  • Impact: At a 40% rejection rate (filtering out the lowest confidence 40%), the hybrid model achieved 90.0% accuracy on the remaining responses, compared to 70.4% with no selection. The hybrid model without aleatoric uncertainty was competitive but consistently lower, proving the value of modeling data ambiguity.

• Reliability & Calibration:

  • Single-Source: Verbalizing showed overconfidence (high MCE). Latent was poorly calibrated. Consistency had low average error but high maximum error (localized failures).
  • Hybrid: The hybrid model with aleatoric uncertainty achieved the lowest Brier Score (0.138) and lowest MCE (0.100). It closely tracked the diagonal in reliability diagrams, indicating that its confidence scores are highly trustworthy.

5. Significance and Conclusion

• Beyond Model Signals: The paper establishes that model-based confidence alone is insufficient for ASAG. Ambiguity in student responses (aleatoric uncertainty) is a distinct source of error that must be explicitly modeled.
• Safe AI Deployment: By providing well-calibrated confidence scores, the proposed framework enables safer Human-AI collaboration. Educators can trust the system to automatically grade high-confidence responses while flagging ambiguous cases for human review.
• Practical Decision Support: The use of Accuracy-Rejection Curves (ARC) offers a practical tool for educators to balance grading accuracy against the cost of manual review, moving beyond arbitrary threshold selection.

In summary, this work advances the field of AI-assisted education by demonstrating that integrating semantic data heterogeneity with LLM confidence signals creates a more robust, reliable, and interpretable grading system suitable for real-world deployment.