Does a Safety-Priming Instruction Reduce Risk-Fact Omission in Psychiatric Note Summarization? A Synthetic-Vignette Study of Small Open-Weight Language Models

In the high-stakes world of mental healthcare, the written record is the lifeline. When a patient describes their fears, their history, and their current state, a clinician must capture every crucial detail to ensure safety. For years, doctors have relied on their own notes, but a new tool has entered the room: artificial intelligence. These computer programs, known as large language models, are being tested to help summarize these complex medical notes, turning pages of text into brief, readable bullet points. The hope is that they can save time and reduce errors. However, there is a specific fear that has received less attention than the fear of the machine making things up. While it is dangerous for a computer to invent a symptom that never happened, it is arguably more dangerous for it to silently delete a symptom that is already there. If a summary leaves out a mention of self-harm or suicidal thoughts, a doctor skimming the document might assume the patient is safe, with no warning that a critical piece of information is missing. This silence is the hidden danger researchers wanted to measure.

A team of researchers set out to test whether a simple change in instructions could fix this problem. They asked a straightforward question: if you tell an artificial intelligence to "always include safety-related findings," does it actually stop dropping those dangerous details? To find the answer without risking real patients, they built a controlled experiment using fifty completely made-up stories. These were not real medical records but carefully constructed scenarios, each written to contain exactly one specific safety risk—such as thoughts of suicide, self-harm, or dangerous hallucinations—and one routine detail, like a change in sleep patterns. They fed these stories to four different small artificial intelligence models, asking each one to summarize the story into three short bullet points. First, they let the models work with a standard instruction. Then, they ran the same stories through the models again, this time adding a specific command to always highlight safety issues.

The results revealed a complex picture that challenges the idea of a simple, universal fix. When the models were left to summarize without special instructions, they made mistakes, but the size of the mistake depended entirely on which model was used. The smallest, least powerful model missed the safety-critical detail in twenty-six percent of the cases, meaning it dropped the warning roughly one time out of four. The largest and most capable model in the group was much better, missing the detail only four percent of the time. When the researchers added the safety instruction, the outcome was not the same for everyone. For the smallest model that was struggling the most, the instruction worked like a miracle. The rate of missed safety details dropped dramatically from twenty-six percent down to eight percent. This was a clear, statistically significant improvement. The instruction helped the model that needed it the most.

However, the story did not end with a simple victory. The researchers discovered that for the other three models, which were already performing reasonably well, the safety instruction came with a hidden cost. While these models did not miss many safety details to begin with, the new instruction caused them to start missing the routine, non-dangerous details much more often. For two of the models, the rate of missing these routine facts jumped by twenty percentage points. It appeared that the models had a limited amount of space in their three-bullet summaries. When told to prioritize safety, they seemed to push out the other important medical information to make room for the safety warning. The instruction was not a free addition; it was a trade-off. The models that did not need the help were forced to sacrifice other parts of the clinical picture to follow the new rule.

The study also looked at which specific types of risks were most likely to be forgotten. The data suggested that for the weaker models, the most dangerous category—suicidal thoughts—was the most likely to be dropped, with omission rates reaching as high as fifty percent in some cases. This highlights a specific vulnerability where the most critical information is the most likely to vanish. The researchers emphasized that these findings come from synthetic, made-up stories, not real patient notes, and that the experiment was a pilot study rather than a final solution. They noted that the artificial intelligence models used were small and open-source, and that the results might differ with larger or different systems.

The ultimate lesson from this work is that there is no single "safety button" that works for every artificial intelligence system. Adding a safety instruction is not a uniform improvement that can be applied blindly to all tools. For the weakest model, it was a vital intervention that saved critical information without hurting the rest of the summary. For the stronger models, it was a disruption that traded safety gains for a loss of other important details. The researchers concluded that before any hospital or clinic adopts a safety instruction for their documentation tools, they must test it on their specific system. They need to verify that the instruction actually reduces the risk of missing dangerous facts and check whether it accidentally causes the system to drop other vital medical information. The path to safer documentation requires careful, model-by-model validation rather than a one-size-fits-all approach.