When Adaptation Hurts: RAG and LoRA Fine-Tuning Can Degrade Risk-Tier Calibration in Small-Model Psychiatric Diagnosis
This paper demonstrates that while RAG and LoRA fine-tuning do not consistently improve diagnostic accuracy on small models for psychiatric tasks, they critically degrade safety-relevant high-risk recall, with fine-tuning failures traced to specific data labeling errors and RAG failures remaining an open mechanistic question.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of artificial intelligence, there is a growing belief that making a computer smarter at a specific job also makes it safer. If you teach a language model to understand the nuances of human mental health, the assumption is that it will not only get better at diagnosing conditions but also become more careful about identifying dangerous situations, like self-harm or suicide. This idea relies on two common ways to customize these models. The first is like giving the computer a library of reference books to consult while it answers a question, a method known as retrieval-augmented generation. The second is like a long apprenticeship, where the model studies a specific set of examples to learn the patterns of a new field, a process called fine-tuning. For doctors and researchers hoping to use these tools in psychiatry, the promise is that these adaptations will create a system that is both accurate and vigilant, catching every sign of risk while correctly identifying every disorder.
However, a recent pilot study conducted by researchers at the Indian Institute of Technology Kharagpur suggests that this promise may be fragile. The team set out to test whether these two popular methods actually improve how well small, local computer models handle psychiatric diagnosis, specifically looking at whether they get better at spotting high-risk cases. They used a set of twenty-four short, written scenarios describing patients, which included two cases that were clearly marked as high-risk due to self-harm behaviors. The researchers compared the models' performance before any changes against their performance after applying the two adaptation methods. What they found was a startling disconnect: while the models sometimes got better at naming the correct diagnosis, they consistently failed to recognize the danger. In fact, after the adaptations, the models stopped identifying the high-risk cases entirely, dropping their ability to spot the danger from a partial success rate to zero, even though their overall diagnostic accuracy remained roughly the same or only changed slightly.
The study focused on three different small computer models, running them on standard home computers rather than massive industrial servers. For two of the models, the researchers used the reference-book approach, feeding the computer summaries of medical criteria right before it made a diagnosis. For the third model, they used the apprenticeship approach, training it on a small set of twelve examples to learn the patterns of different mental health conditions. The results were mixed regarding accuracy. The reference-book method helped the smallest model get slightly more diagnoses right, but it actually made the medium-sized model worse. The training method left the third model's accuracy exactly the same as before. These small shifts in accuracy were not the main story. The real finding was that under every single adapted condition, the models failed to flag the two high-risk patients. In the original, untrained state, the models had correctly identified the risk in at least one of the two cases. After the changes, they missed both.
The researchers were able to trace exactly why the training method failed. It turned out to be a mistake in the very examples they used to teach the model. In the small set of twelve training cases, the single example of the high-risk condition was labeled as "moderate" risk instead of "high." The model learned this lesson perfectly, associating that specific diagnosis with a lower level of danger. When it encountered the high-risk patients in the test, it confidently downgraded them to low risk. This was a flaw in the teaching material, not necessarily a flaw in the training method itself. The researchers noted that if the training data had been labeled correctly, the outcome might have been different.
The reason for the failure in the reference-book method was less clear. The summaries the computer consulted actually contained the correct information about self-harm, and the system successfully found and read those summaries. Yet, the model still ignored the danger. The researchers suspect that when the computer read the extra text, it treated the information as a general description of the disease rather than a specific warning signal, or perhaps the extra words distracted it from the part of the instructions that asked it to assess risk. They did not find a definitive answer for this, leaving it as an open question for future study.
The author is careful to state that this is a small-scale experiment using synthetic scenarios written by the researchers, not real patient records. They also used very small computer models, and the results might look different with larger, more powerful systems. They do not claim that these methods are inherently dangerous or that they will always fail. Instead, they offer a specific warning based on their data: improving a model's ability to name a disease does not guarantee it will maintain its ability to spot a crisis. In their view, safety and accuracy are not automatically linked. When adapting these tools for medical use, the researchers argue that safety must be measured separately and explicitly every time, because making a model smarter at one thing can inadvertently make it blind to another.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.