Interpretability from the Ground Up: Stakeholder-Centric Design of Automated Scoring in Educational Assessments

1. Problem Statement

While AI-driven automated scoring systems offer scalable solutions for evaluating complex student responses, they face a critical adoption gap in large-scale, real-world educational assessments. Despite improvements in accuracy, these systems often lack transparency and interpretability, leading to concerns regarding bias, fairness, and the justifiability of scoring decisions.

• The Gap: Current "interpretable" solutions (e.g., LLMs generating chain-of-thought explanations) often fail to be faithful to the model's actual internal computation. They are frequently post-hoc rationalizations rather than reflections of the decision-making process.
• The Need: There is a lack of a widely accepted framework that balances high scoring accuracy with rigorous interpretability, specifically designed to meet the diverse needs of assessment stakeholders (test-takers, developers, and users).

2. Methodology: The FGTI Principles and ANALYTICSCORE

The authors propose a principled approach grounded in educational assessment literature, moving away from treating scoring as an isolated prediction task to a stakeholder-centric lifecycle.

A. The Four FGTI Principles

The authors define four foundational principles for interpretable automated scoring:

1. Faithful: Explanations must accurately reflect the model's actual computational mechanism, not just generate plausible text (ruling out standard LLM prompting for explanations).
2. Grounded: Features must be explicitly linked to identifiable elements of the student's work and the task (e.g., specific propositions), rather than abstract vector embeddings.
3. Traceable: The scoring logic must be decomposable into clear, sequential reasoning steps that a human can inspect and verify.
4. Interchangeable: Humans must be able to intervene in or replace the output of specific reasoning subroutines (e.g., correcting a feature extraction) without breaking the system.

B. The ANALYTICSCORE Framework

To demonstrate the feasibility of FGTI, the authors developed ANALYTICSCORE, a modular, three-phase framework for text-based constructed-response scoring:

• Phase 1: Extracting Analytic Components

  • The system extracts a set of explicitly identifiable elements (analytic components) from student responses.
  • In this study, components are operationalized as propositions (e.g., "Pandas and koalas only eat one food").
  • Implementation: An LLM is prompted to extract up to 15 components per item.

• Phase 2: Featurization (Grounded & Human-Understandable)

  • Student responses are converted into features based on the presence of the extracted components.
  • A labeling function $f(r; c)$ assigns values:
    • 2: Direct paraphrase of the component.
    • 1: Partial paraphrase.
    • 0: No paraphrase.
  • Optimization: To reduce costs, a small open-source model (Llama-3.1-8B) is fine-tuned via distillation using data generated by a proprietary LLM (o4-mini/GPT-4.1) to perform this featurization.

• Phase 3: Logically Traceable Scoring

  • The featurized data is fed into a scoring model that is inherently interpretable.
  • Implementation: The authors use Ordinal Logistic Regression (Immediate-Threshold variant).
  • The model calculates a weighted sum of evidence ($\eta$) based on the features and compares it against learned thresholds to determine the final score.
  • This structure allows for "what-if" analysis (e.g., "If feature X were different, the score would be Y").

3. Key Contributions

1. Stakeholder-Centric Framework: The paper shifts the focus from purely technical accuracy to a framework that addresses the specific interpretability needs of test-takers, developers, and policymakers.
2. FGTI Principles: The definition of Faithfulness, Groundedness, Traceability, and Interchangeability provides a rigorous standard for evaluating interpretability in high-stakes AI.
3. ANALYTICSCORE Implementation: The first framework to embody these principles in practice, demonstrating that interpretable systems can achieve competitive accuracy.
4. Distillation Strategy: A novel approach to making the framework scalable by distilling a proprietary LLM's featurization behavior into a small, open-source model (Llama-3.1-8B), ensuring cost-effectiveness without sacrificing the "grounded" nature of the features.

4. Experimental Results

The framework was evaluated on the ASAP-SAS dataset (10 open-ended items across Science and Reading).

• Scoring Accuracy:

  • ANALYTICSCORE (using distilled Llama or GPT-4.1-mini) outperformed several uninterpretable baselines (e.g., AutoSAS, ASRRN, NAM).
  • It achieved performance within 0.06 Quadratic Weighted Kappa (QWK) of the uninterpretable State-of-the-Art (SOTA) models (like fine-tuned DeBERTa and Llama-3.1-8B) on average.
  • Notably, the distilled Llama featurizer showed a significant performance boost (+0.13 QWK) compared to the non-distilled version, proving the efficacy of the distillation strategy.

• Featurization Alignment (Human vs. Model):

  • The study compared the model's feature labeling against human annotators.
  • Human-Human Agreement: Krippendorff's $\alpha$ ranged from 0.67 to 0.72, indicating a reasonable baseline for human consistency on this task.
  • Model-Human Alignment: The distilled Llama featurizer achieved high alignment with human judgments (QWK ~0.81–0.90 in Science and Informational Reading).
  • Weakness: Alignment was lower for the "partial paraphrase" (Label 1) category, attributed to the inherent ambiguity of this category and its rarity in human ratings.

5. Significance and Future Directions

• Paradigm Shift: The paper argues for moving automated scoring from a "black-box prediction" task to a "transparent evidentiary reasoning" process.
• Practical Viability: It proves that high accuracy does not require sacrificing interpretability. By constraining the architecture to be traceable and interchangeable, the system remains robust and auditable.
• Scalability: The distillation approach offers a path to deploy these systems in large-scale assessments without the prohibitive costs of running proprietary LLMs for every inference.
• Limitations & Future Work:
  • The current scoring module (Ordinal Logistic Regression) may be too simple for highly complex reasoning; future work could explore traceable LLM agents.
  • Validation is currently limited to the ASAP-SAS dataset; broader benchmarking across domains is needed.
  • Future research must address "Construct Alignment" (ensuring features map to the intended educational constructs) and the development of rigorous operational protocols for auditing these systems.

In conclusion, this work provides a foundational blueprint for building automated scoring systems that are not only accurate but also trustworthy, auditable, and aligned with the ethical and practical needs of the educational ecosystem.