Condence, Calibration, and Abstention in Small Open-Weight Language Models on Psychiatric Knowledge Questions: A Cross-Domain Replication
This pilot study demonstrates that small, open-weight language models applied to psychiatric knowledge questions exhibit poor confidence calibration, domain-dependent accuracy rankings, and that prompting them to abstain from answering often degrades rather than improves performance when evaluated with rigorous paired statistics.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer
In the quiet corners of modern medicine, a new kind of tool is emerging: small, self-contained computer programs capable of reading and answering questions about human health. Unlike the massive, cloud-based systems that require powerful internet connections and raise privacy concerns, these smaller models can run on a single local computer, keeping sensitive patient data off the internet. This capability has sparked hope for their use in psychiatry, a field where doctors often rely on subjective reports rather than blood tests or scans to make decisions. However, a critical question remains for any tool that offers medical advice: does it know when it is wrong? If a computer program is confident but incorrect, the consequences can be severe. Researchers have long studied whether these systems can accurately gauge their own certainty, but most of that work has focused on huge, proprietary systems or general medical knowledge. The specific behavior of smaller, locally running models when faced with the nuanced, subjective questions of psychiatry has remained a mystery.
A recent pilot study set out to fill this gap by testing four different small language models on a specific set of 150 multiple-choice questions about psychiatric knowledge. The researchers wanted to see if these models could not only answer correctly but also recognize when they should say, "I don't know." They compared the models' performance on these psychiatric questions against their performance on general medical questions to see if the ranking of which model was "best" stayed the same. The study also investigated a common safety idea: that giving a model the option to abstain from answering when it feels unsure would automatically make it safer and more accurate. The team ran these tests entirely on a local workstation, ensuring no data left the machine, and used rigorous statistical methods to compare the results before and after the models were allowed to skip questions.
The results revealed that the landscape of these small models is far more complex than a simple list of "best" and "worst" performers. When the models were tested on general medical knowledge, one specific model ranked third out of four. Yet, when the same model faced the psychiatric questions, its performance dropped significantly, falling to last place. This suggests that a model's skill is not a fixed trait that transfers easily from one medical specialty to another; what works well for general medicine does not necessarily work well for psychiatry. Furthermore, the study found that the models were consistently overconfident. On average, they stated they were 90 to 95 percent sure of their answers, even though their actual accuracy ranged only from 57 to 75 percent. This gap between stated confidence and actual correctness was much wider than what has been observed in larger, commercial systems.
Perhaps the most surprising finding concerned the idea of letting the models skip questions they were unsure about. A simple look at the questions the models chose to answer suggested that skipping might improve their scores for some of them. However, when the researchers looked at the exact same questions under both conditions—forced to answer versus allowed to skip—they found a different story. For three of the four models, the ability to skip did not lead to a genuine improvement in accuracy. In fact, for the best-performing model, the decision to skip actually caused it to miss questions it would have answered correctly if forced to try. The model with the best ability to rank its own confidence still failed to use that ability to make smart choices about when to stop.
The only model that showed a clear change when allowed to skip questions was the smallest one, and the change was negative. When this tiny model was given the option to say "I don't know," its overall accuracy dropped significantly. This result was not a fluke; it mirrored a similar drop found in a separate test on general medical questions, suggesting a specific weakness in how this particular small model handles uncertainty. The study concludes that simply adding a "I don't know" option is not a magic safety switch. For these small, local models, the instruction to abstain does not reliably filter out errors and can sometimes make performance worse. The researchers emphasize that these findings are specific to this small group of models and this type of exam question, serving as a hypothesis for future, larger studies rather than a final rule for clinical use. The key takeaway is that in the realm of small, local AI, confidence is often an illusion, and the ability to know when to stop is not guaranteed.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.