💬 NLP

Knowing When to Defer: Selective Prediction for Responsible Knowledge Tracing

This paper introduces an intrinsic selective prediction framework for Knowledge Tracing models that leverages Monte Carlo Dropout to quantify epistemic uncertainty, demonstrating that deferring the most uncertain predictions significantly boosts accuracy and fairness while revealing that standard psychometric factors explain less than a quarter of the model's uncertainty signal.

Original authors: Joshua Mitton, Prarthana Bhattacharyya, Ralph Abboud, Simon Woodhead

Published 2026-05-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Joshua Mitton, Prarthana Bhattacharyya, Ralph Abboud, Simon Woodhead

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a math teacher grading a stack of quizzes. You are good at your job, but even the best teachers have moments of doubt. Sometimes a student's answer is so confusing or the question is so tricky that you aren't 100% sure if you're right. In those moments, a responsible teacher might say, "I'm not sure about this one; let me ask a colleague for a second opinion."

This paper is about teaching computer models (specifically, "Knowledge Tracing" models that predict how students learn math) to do the exact same thing.

Here is the breakdown of what the researchers did, using simple analogies:

1. The Problem: The "Overconfident Robot"

For years, researchers have built AI models to predict if a student will get a math question right or wrong. These models are like robots that have read every textbook in the library. They are very good at guessing, but they have a flaw: they don't know when they are guessing.

Even when they are wrong, they often sound very confident. In a real classroom, if a robot confidently tells a student the wrong answer, it's a disaster. The researchers wanted to give these robots a "gut feeling" so they know when to say, "I don't know, let a human teacher handle this."

2. The Solution: The "Confidence Meter" (MC-Dropout)

The researchers didn't build a brand new robot from scratch. Instead, they took three existing, popular robot designs (called DKT, SAKT, and AKT) and gave them a special trick called MC-Dropout.

Think of MC-Dropout like asking the same question to the same robot 100 times, but every time you ask, you slightly "fog up" its brain (by randomly turning off some of its internal connections).

If the robot gives you the same answer 100 times, it's confident.
If the robot gives you 50 different answers, it's confused.

By measuring how much the robot's answers wiggle around, the researchers created a "Confidence Meter." If the meter shows high wiggling (high uncertainty), the system automatically stops and says, "I'm going to pass this one to a human teacher."

3. The Results: "The Safety Net Works"

The team tested this on a real math dataset from a platform called Eedi. They asked the robots to predict student answers, but whenever the robot was "wiggly" (uncertain), they let the robot skip that prediction and defer it to a human.

Here is what happened:

Better Accuracy: By skipping the 20% of questions the robot was unsure about, the accuracy of the remaining predictions went up significantly (by about 2 to 3 percentage points). It's like a basketball player who stops shooting when they are tired and only shoots when they are fresh; their shooting percentage goes up.
Targeted Safety: The robot wasn't just skipping random questions. It was specifically skipping the ones it was most likely to get wrong. The questions it skipped were 1.5 times more likely to be errors than the ones it kept.
Fairness: The robot didn't just skip questions for "bad" students. It skipped questions for students of all skill levels and for questions of all difficulties. It treated everyone fairly.

4. The Big Surprise: "It's Not Just About Difficulty"

The researchers wanted to know why the robot was confused. They tried to explain the robot's confusion using simple, old-school math rules (like "this question is hard" or "this student is struggling").

They found that these simple rules could explain less than 4% of the robot's confusion. Even when they used a complex, non-linear calculator to try to find a pattern, they could only explain about 23% of it.

The Analogy: Imagine the robot is confused because it's trying to understand a student's unique, messy history of learning—how they forgot a concept last week, how they guessed correctly by luck, and how they connect two ideas in a weird way. Simple math rules (like "Question X is hard") can't see that messy history. The robot's own internal "brain fog" (MC-Dropout) is the only thing that can see it.

5. The Conclusion: "Know When to Step Back"

The main takeaway is that for AI to be responsible in a classroom, it needs to know when to step back.

Don't rely on simple rules: You can't just look at the difficulty of a question to know if the AI is safe.
Use the AI's own feelings: The AI needs to use its own internal uncertainty signal (the "wiggles") to decide when to ask for help.
It's a team effort: This system doesn't replace teachers. It's a tool that helps teachers by filtering out the tricky cases that need human attention, while letting the AI handle the straightforward ones.

In short, the paper proves that giving AI models a way to say "I'm not sure" makes them smarter, safer, and fairer, without needing to retrain them or build entirely new systems.

1. Problem Statement

Knowledge Tracing (KT) models are designed to predict a student's future performance on educational questions based on their interaction history. While recent deep learning architectures (e.g., DKT, SAKT, AKT) have improved predictive accuracy, they suffer from a critical limitation in real-world deployment: they lack the ability to quantify their own uncertainty.

The Gap: Traditional KT models produce point estimates (e.g., "80% chance of correct") without indicating confidence. This leads to low specificity, where models confidently make incorrect predictions that go undetected.
The Goal: The paper proposes treating responsible KT deployment as a Selective Prediction problem. Instead of forcing a prediction on every instance, the model should identify high-uncertainty cases and defer them to a human teacher, thereby improving overall system reliability and safety.

2. Methodology

The authors introduce an intrinsic selective prediction layer that integrates directly with existing KT models without requiring retraining.

A. Uncertainty Quantification via MC-Dropout

The core mechanism for estimating uncertainty is Monte Carlo Dropout (MC-Dropout).

Process: During inference, dropout layers remain active. The model performs $M$ stochastic forward passes for a single input.
Metrics:
- Mean Prediction ( $\bar{p}$ ): The average probability across $M$ passes.
- Uncertainty Signal: The variance (or entropy) of the predictions across the $M$ passes. High variance indicates high epistemic uncertainty (the model is unsure about its learned representation of the student's state).
Selection Strategy: Predictions are sorted by MC-Dropout variance. The system abstains (defers) on the top $X\%$ most uncertain predictions (e.g., the top 20%) and only outputs predictions for the remaining confident cases.

B. Experimental Setup

Datasets: The study uses the Eedi mathematics dataset, a large-scale real-world dataset containing question-response logs.
Architectures: The method is evaluated across three dominant KT families:
1. DKT (Deep Knowledge Tracing): LSTM-based.
2. SAKT (Self-Attentive Knowledge Tracing): Self-attention based.
3. AKT (Attentive Knowledge Tracing): Context-aware attention with forgetting mechanisms.
Baselines:
- Calibrated 2PL IRT: A standard Item Response Theory model (Two-Parameter Logistic) used as a psychometric baseline for uncertainty.
- Classical Psychometric Stack: A regression analysis testing if uncertainty can be explained by standard features: question difficulty, student ability, IRT outcome ambiguity, and curriculum coverage.

3. Key Contributions

Architecture-Agnostic Improvement: Demonstrated that selective prediction improves accuracy, AUC, and F1 scores across DKT, SAKT, and AKT without retraining.
Superiority over Psychometric Baselines: Proved that MC-Dropout variance is a significantly stronger signal for deferral than a calibrated 2PL IRT model.
Variance Decomposition (Epistemic Content): Showed that classical psychometric features (difficulty, ability, etc.) explain less than 4% (linear) to 23% (non-linear) of the model's uncertainty. The remaining 77–90% is architecture-specific epistemic content (uncertainty regarding the student's unique sequential trajectory) that simpler models cannot capture.
Fairness and Targeting: Verified that the deferral strategy is highly targeted (deferring actual errors) and fair (does not disproportionately defer low-ability students).

4. Key Results

A. Performance Gains (at 80% Coverage)

By deferring the 20% most uncertain predictions, the models achieved significant metric lifts without retraining:

Accuracy: +2.3 to +3.0 percentage points (pp).
AUC: +1.9 to +2.4 pp.
F1 Score: +1.4 to +4.3 pp.
Statistical Significance: All improvements were statistically significant with tight confidence intervals.

B. Targeting and Fairness

Error Rate: The set of deferred predictions had an error rate 1.45 to 1.60 times higher than the kept set. On the hardest questions, this ratio reached 2.3–2.5x.
Fairness: The abstention rate was consistent across student ability quartiles (ranging 16–23%, close to the 20% baseline). Crucially, the model did not default to deferring low-ability students; the weakest students were deferred less than others, avoiding a common failure mode in AI education.

C. Comparison to IRT Baseline

At 80% coverage, MC-Dropout provided an AUC lift of 2.0–2.4 pp.
The calibrated 2PL IRT baseline provided only 0.41–0.46 pp lift.
Conclusion: MC-Dropout offers roughly 5x the lift of the IRT baseline, proving it captures genuine epistemic uncertainty beyond simple item difficulty and student ability.

D. Variance Decomposition (The "Why")

The authors regressed the model's epistemic uncertainty (using BALD decomposition) against a stack of classical features:

Linear Model: Explained variance was < 4%.
Non-Linear Model (Random Forest): Explained variance rose to ~23% (max for DKT), leaving 77–90% unexplained.
Implication: The uncertainty signal is largely architecture-specific. It stems from the model's internal, non-linear representation of a student's unique learning trajectory, which cannot be reconstructed by external heuristics or simpler psychometric proxies.

5. Significance and Conclusion

This paper establishes that responsible Knowledge Tracing requires more than just high accuracy; it requires uncertainty awareness.

Necessity of Native Uncertainty: The study proves that relying on external proxies (like IRT or curriculum coverage) is insufficient for safe deployment. The model's own internal uncertainty signal (via MC-Dropout) is necessary to detect high-risk predictions.
Deployment Strategy: The proposed "Selective Prediction" layer acts as a gatekeeper. Low-variance predictions feed directly into the recommendation engine, while high-variance predictions trigger a fallback (e.g., teacher review, diagnostic questions, or mastery quizzes).
Complementarity: This approach is not a substitute for fairness audits or classroom evaluation but is a complementary component essential for building safe, confidence-aware educational AI systems.

In summary, the paper argues that for KT models to be deployed responsibly in real classrooms, they must be equipped with a mechanism to "know when they don't know," and MC-Dropout provides a robust, architecture-native solution to achieve this.