💻 computer science

Do Small Language Models Know When They're Wrong? Confidence-Based Cascade Scoring for Educational Assessment

This paper demonstrates that verbalized confidence in small language models can effectively route educational assessment tasks to larger models in a cost-efficient cascade system, provided the small models exhibit strong confidence discrimination to accurately identify uncertain cases.

Original authors: Tyler Burleigh

Published 2026-04-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tyler Burleigh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you run a massive school where thousands of students take math tests every day. You need to grade their answers instantly to give them feedback.

You have two types of graders:

The "Super-Grader" (Large AI): This is a genius professor. They are incredibly accurate, but they are slow, expensive to hire, and get tired easily.
The "Junior Grader" (Small AI): This is a smart intern. They are fast, cheap, and can handle most questions easily. But sometimes, they get confused or make mistakes.

The Problem:
If you use the Super-Grader for everything, your school goes broke and students wait forever. If you use the Junior Grader for everything, you save money, but you might give students bad grades because the intern isn't perfect.

The Solution: The "Confidence Cascade"
This paper asks a simple question: Can we teach the Junior Grader to say, "I'm not sure about this one, please ask the Professor"?

The researchers tried a new trick: they asked the Junior AI to give a score and a confidence number (like a percentage) with every answer.

If the Junior AI says, "I'm 95% sure," you accept the grade.
If the Junior AI says, "I'm only 40% sure," you immediately pass that specific question to the Super-Grader.

This is called a Cascade System. It's like a bouncer at a club: the Junior AI checks the ID. If it looks fake (low confidence), they call the Head Bouncer (Super-Grader) to double-check.

The Big Discovery: Not All Juniors Are Created Equal

The researchers tested three different "Junior" AIs. Here is what they found, using some fun analogies:

1. The "Honest Intern" (Claude Haiku)

How it works: This AI is great at knowing when it's confused. When it sees a tricky math problem, it honestly says, "I'm only 60% sure." When it sees an easy one, it says, "I'm 99% sure."
The Result: Because it knows its own limits, the school used this system and saved 76% of the money and 61% of the time, while still getting grades just as accurate as the Super-Grader.
Metaphor: It's like a reliable mechanic who knows exactly when a car problem is too complex for them and calls the master mechanic.

2. The "Overconfident Robot" (Gemini Lite)

How it works: This AI is fast, but it has a broken confidence dial. No matter how hard the question is, it always says, "I'm 99% sure!" It thinks it's a genius even when it's guessing.
The Result: Because it never admits uncertainty, the system never called the Super-Grader. The school saved money, but the grades were wrong. The "cascade" failed because the signal was broken.
Metaphor: It's like a GPS that always says "Turn Left" even when you are in a desert. You can't trust its "confidence" to tell you when to ask for help.

3. The "Nervous Intern" (GPT Nano)

How it works: This AI is smart but a bit paranoid. It says, "I'm only 60% sure" even on easy questions.
The Result: Because it was too nervous, it sent half of all questions to the Super-Grader. The school saved some money, but not as much as they could have, because they were paying the expensive professor to do the intern's easy work.

Why This Matters for You

This isn't just about math tests. This is about how we use AI in the real world.

The Bottleneck: The paper proves that the accuracy of the small AI doesn't matter as much as its self-awareness. A slightly less accurate AI that knows when it's wrong is more valuable than a smart AI that doesn't know it's guessing.
The Future: If we can build systems where AI knows its own limits, we can make smart, instant, and cheap educational tools (or medical checkups, or legal advice) that don't break the bank.

In a nutshell:
The paper teaches us that for AI to be truly useful at scale, it needs to learn the art of humility. It needs to know when to say, "I don't know," so the expensive experts can step in. If the AI is too arrogant (overconfident) or too shy (underconfident), the whole system falls apart. The "Honest Intern" is the key to the future of cheap, fast, and accurate AI.

1. Problem Statement

Automated scoring of student work at scale faces a fundamental tradeoff between accuracy, cost, and latency.

The Dilemma: Large Language Models (LLMs) generally offer higher scoring accuracy but are expensive and slow. Small Language Models (SLMs) are cost-effective and fast but less accurate.
The Challenge: "Cascade" systems attempt to solve this by using an SLM for most tasks and escalating only difficult cases to an LLM. However, the critical bottleneck is routing: determining which cases to escalate.
The Gap: Existing methods for estimating uncertainty (e.g., token probabilities, repeated sampling) are often impractical for commercial APIs due to access restrictions or cost. Verbalized confidence (asking the model to state a numerical confidence) is a promising alternative, but its efficacy in educational scoring and its ability to distinguish between accurate and inaccurate predictions had not been rigorously tested.

2. Methodology

Dataset & Context

Source: 2,100 scoring decisions derived from 1,200 "Explain Your Thinking" (EYT) conversations on Khan Academy's high school math platform.
Task: Binary judgment (Met/Not Met) on 7 specific rubric criteria across 4 math items (Algebra and Geometry).
Ground Truth: Each decision was rated by 3 expert human annotators. Ground truth was defined by majority vote (2/3 agreement).
Difficulty Proxies: Two human-derived metrics were used to validate AI confidence:
1. Inter-rater disagreement: Split decisions (2/1) vs. unanimous (3/3).
2. Annotator response time: Longer deliberation times indicate higher ambiguity.

Models Evaluated

The study tested three model families, each with a "Small" (SLM) and "Large" (LLM) variant:

GPT Family: gpt-5.4-nano (SLM) vs. gpt-5.4 (LLM).
Claude Family: claude-haiku-4.5 (SLM) vs. claude-opus-4.6 (LLM).
Gemini Family: gemini-3.1-flash-lite-preview (SLM) vs. gemini-3.1-pro-preview (LLM).

Experimental Design

Confidence Elicitation: SLMs were prompted to output a JSON object containing the score, reasoning, and a numerical confidence score (0–100) defined as "the probability that your scoring judgment is correct."
Cascade Simulation:
1. Both SLM and LLM scored all 2,100 decisions independently.
2. A threshold ( $\tau$ ) was applied: If the SLM's confidence $< \tau$ , the decision was escalated to the LLM; otherwise, the SLM's score was kept.
3. Thresholds were swept from 0.01 to 0.99 to generate Pareto-optimal curves (balancing cost/latency vs. accuracy).
Metrics:
- Discrimination: AUROC (Area Under the Receiver Operating Characteristic Curve) to measure if confidence separates correct from incorrect predictions.
- Agreement: Cohen's Kappa ( $\kappa$ ) to measure alignment with human ground truth.
- Efficiency: Cost per 1,000 decisions and median API latency.

3. Key Results

RQ1: Confidence Discrimination (Can SLMs distinguish right from wrong?)

Claude Haiku: Achieved excellent discrimination (AUROC 0.857). Its confidence distribution showed clear separation between accurate and inaccurate predictions.
GPT Nano: Achieved acceptable discrimination (AUROC 0.757) with high granularity (36 distinct values).
Gemini Lite: Failed significantly (AUROC 0.678). It exhibited a "near-degenerate" distribution, outputting nearly the same high confidence (near 1.0) regardless of whether the prediction was correct or incorrect.

RQ2: Correlation with Human Difficulty

All SLMs reported lower confidence on "split" (ambiguous) human decisions compared to "unanimous" ones.
Claude Haiku showed the strongest sensitivity (Cohen's $d = 1.16$ ), correlating strongly with both human disagreement and longer annotator response times.
Gemini Lite showed the weakest correlation, consistent with its lack of variance in confidence scores.

RQ3: Cascade Performance (Cost vs. Accuracy)

Claude Cascade:
- Achieved a Kappa of 0.802 (vs. Large LM's 0.819).
- Statistically indistinguishable from the Large LM alone (95% CI includes zero difference).
- Savings: 76% lower cost and 61% lower latency.
GPT Cascade:
- Achieved Kappa 0.763 (vs. Large LM's 0.770).
- Statistically indistinguishable from Large LM.
- Savings: 49% lower cost, 64% lower latency. However, it required a high escalation rate (47%), reducing efficiency gains.
Gemini Cascade:
- Achieved Kappa 0.783 (vs. Large LM's 0.810).
- Statistically worse than the Large LM (CI entirely below zero).
- Despite saving 81% in cost, it failed to close the accuracy gap because the routing signal was uninformative.

4. Key Contributions & Significance

Confidence Quality as the Bottleneck: The paper establishes that model accuracy alone is insufficient for cascades. The quality of the confidence signal (discrimination and variance) is the primary determinant of success. A highly accurate SLM with poor confidence discrimination (like Gemini Lite) cannot effectively route tasks, leading to accuracy degradation.
Verbalized Confidence Viability: The study validates verbalized confidence as a practical, API-agnostic routing signal that does not require log-prob access, fine-tuning, or repeated sampling.
Human-AI Alignment: It provides evidence that verbalized confidence tracks genuine task difficulty (human disagreement and deliberation time), suggesting SLMs are modeling ambiguity in ways consistent with human experts.
Practical Framework for EdTech: The research offers a concrete "Pareto frontier" for educational platforms. It demonstrates that with the right model pair (e.g., Claude Haiku + Opus), systems can achieve near-human accuracy at a fraction of the cost and latency, making synchronous, real-time AI assessment viable.
Diagnostic Value: The study highlights that poor confidence calibration (e.g., overconfidence in all cases) renders cascade systems useless, serving as a warning for practitioners to test discrimination before deployment.

5. Conclusion

The paper concludes that Small Language Models can know when they are wrong, but this capability varies drastically by model. When an SLM possesses strong confidence discrimination (like Claude Haiku), confidence-based cascading allows practitioners to trade cost for accuracy along a smooth frontier, achieving Large-LM performance at Small-LM costs. Conversely, if the confidence signal is degenerate, the cascade offers no advantage over simply using the Large Model for everything.