Context-Window-Mediated Corruption of Large Multimodal Models in Clinical Medicine: A Systematic Review

In the modern hospital, a new kind of assistant has arrived. These are large language and vision models, computer programs trained on vast amounts of text and images that can read patient charts, interpret X-rays, and suggest diagnoses. They work by taking a stream of information—what a doctor types, what a patient says, or what a picture shows—and processing it all as a single, continuous flow. The system does not inherently know which parts of that flow are trusted instructions and which parts are just background noise or accidental scribbles. To the computer, a command written by a doctor and a stray note written by a patient look like the same kind of data. This creates a unique vulnerability: if the information fed into the machine is altered, even slightly, the machine's output can change dramatically, regardless of how smart the machine itself is.

This is not a problem of the computer being broken or poorly built; it is a problem of how it listens. Imagine a highly skilled translator who is so eager to follow instructions that if someone whispers a new command into their ear while they are working, they immediately stop what they were doing and follow the whisper instead. In the world of artificial intelligence, this is known as prompt injection, where the input itself hijacks the system's behavior. While security experts have long worried about malicious hackers using this trick, a new systematic review asks a more unsettling question for the medical field: does the danger require a hacker at all? Could a simple, accidental mark on a medical image, or a patient's urgent but confused wording, trigger the same failure?

A team of researchers from Beth Israel Deaconess Medical Center and the University of Debrecen set out to answer this by gathering every available study that tested how these medical models react when their input is tampered with. They looked at ten specific studies published between 2025 and 2026, covering everything from cancer diagnosis and skin lesion analysis to patient advice and medication guidance. Their goal was to see if these powerful tools could be corrupted simply by changing what they read or see, without anyone needing to hack the computer's internal code or access its secret training data.

The researchers found that the models are indeed highly vulnerable, and the corruption does not require a villain. In fact, the studies showed that large output changes occurred whether the input was a deliberate attack crafted by a researcher or non-malicious content constructed to resemble ordinary clinical material. However, the review explicitly notes that no study compared naturally occurring content with a deliberate attack under matched conditions, so it remains unproven whether real-world accidents cause the same failures. The team organized these failures into four main categories. The first is instruction-hierarchy failure, where a hidden command overrides the doctor's original request. The second is evidence-integrity failure, where the model accepts false information or invented findings as truth. The third, specific to models that see images, is multimodal interference, where text written on or inside a medical picture overrides the visual evidence of the picture itself. The fourth is interactional sensitivity, where the tone of a message or a false premise changes the model's decision.

The results were stark. In one study involving skin cancer images, adding a single fabricated label to a picture caused the model's diagnostic accuracy to plummet from a range of 58 to 62 percent down to 0.0 to 1.9 percent. In another, when researchers altered the tone of a patient's message to sound more urgent or authoritative, the rate at which the model recommended sending a patient to the emergency room jumped from about 14 percent to as high as 82 percent. Perhaps most concerning was that these failures happened across different types of models, from older versions to the newest, most powerful systems. No single model proved to be consistently safe; a system that was robust against one type of error was often the most vulnerable to another.

The researchers also tested whether any current defenses could stop these problems. They looked at strategies like adding safety instructions to the prompt, having a second model review the first one's work, or retraining the models on corrected examples. None of these methods eliminated the risk. Some reduced the errors slightly, but others had no effect at all. One defense that involved retraining the model did improve its ability to reject false information, but it also caused the model to start refusing legitimate requests, essentially trading one type of mistake for another. Crucially, none of the tested defenses worked across different types of errors; a fix for one problem did not protect against another.

A significant portion of the review focused on whether these failures required a malicious attacker. The answer, based on the available evidence, is that researcher-constructed content designed to look like ordinary, accidental clinical material—such as a handwritten note on a slide or a patient's own urgent phrasing—was enough to break the system. However, the paper clarifies that these were not drawn from routine care, and no study tested naturally occurring content in isolation. This suggests that the threat is not just a cyberattack, but a fundamental weakness in how these models process information. The integrity of the data entering the system is just as important as the safety of the model itself.

The authors conclude that the current approach to medical artificial intelligence, which focuses heavily on training the model and monitoring its output, may be missing a critical piece of the puzzle. If the information fed into the system can be corrupted by simple, non-malicious means, then the safety of the entire clinical decision depends on the purity of that input. Until we can guarantee that the data reaching these models is free from accidental or intentional manipulation, and until we have defenses that work across all types of errors, the reliability of these tools in real-world care remains uncertain. The findings serve as an early warning: the safety of these systems depends not just on the intelligence of the machine, but on the trustworthiness of the world it is asked to read.