Eliciting Intrinsic Hallucinations in LLMs via Semantically Equivalent Adversarial Attacks

Imagine you are talking to a super-smart robot librarian who has read almost every book in the universe. You ask it a question, and to be extra sure, you hand it a specific page from a book to use as its answer key. This setup is called Retrieval-Augmented Generation, or RAG. It's like giving the robot a reference sheet so it doesn't have to guess from its own memory. The hope is that the robot will stick strictly to the reference sheet, ignoring its own internal knowledge if the two don't match. But here's the catch: sometimes, even with the reference sheet in hand, the robot gets confused, forgets what it's reading, or starts making things up anyway. This is called an "intrinsic hallucination." It's not that the robot is lying on purpose; it's that its brain is just too jumpy to stay faithful to the evidence right in front of it.

Now, imagine you are a detective trying to figure out how jumpy this robot really is. You could ask it the same question a hundred different ways, but if you change the meaning of the question, the robot might just be confused by the new topic. To truly test its focus, you need to ask the exact same question, but whisper it in a dozen different accents, dialects, or sentence structures. If the robot gives you a different (and wrong) answer just because you said "What time is it?" instead of "Could you tell me the current time?", then the robot has a serious problem with consistency. This paper is all about building a machine that whispers those tricky, meaning-preserving variations to see if the robot librarian cracks under the pressure.

The researchers behind this study decided to stress-test these AI systems using a clever new framework. They didn't just ask the models to break their safety rules (which is a common way to test AI); instead, they asked them to break their own honesty. They used a team of digital "attackers"—some using math to tweak words, others using genetic algorithms to evolve new sentences, and some using other AI to rewrite questions—to find the perfect, meaning-preserving rephrasing that would make the model hallucinate. The goal was to see if they could trick the model into ignoring the provided context and making up a story, even though the question meant exactly the same thing as the original.

The results were a bit of a shocker. The team found that even the most advanced, state-of-the-art models are surprisingly fragile. When they used these semantically equivalent attacks, the models' ability to stick to the facts dropped dramatically. In some cases, the faithfulness of the model to the provided evidence degraded by up to 50% for specific models like GPT-5-mini on certain datasets. For example, on the FaithEval dataset, GPT-5-mini's faithfulness dropped from 0.72 to 0.22—a 41.7% decrease—suddenly failing to answer correctly when the question was phrased just slightly differently, even though the meaning hadn't changed at all.

The paper suggests that simply making models bigger or smarter isn't enough to fix this. The researchers tested everything from small, open-source models to massive, closed-source giants, and they all stumbled when faced with these meaning-preserving tricks. The attacks were so effective that they could induce the models to say things like "Unable to answer" or to invent details that weren't in the text, all while the question remained logically identical to the original.

Interestingly, the type of attack mattered. Some methods that just swapped out words for synonyms (like changing "big" to "large") often made the questions sound weird and unnatural, which the models could easily spot. But the methods that used other AI to rewrite the whole sentence naturally—keeping the flow and meaning perfect—were the real troublemakers. These natural-sounding attacks were the ones that caused the biggest drop in performance, proving that the models are brittle not just to weird gibberish, but to the natural variety of human language.

The study also looked at how the models failed. Sometimes they just refused to answer, but often they got the facts wrong, misread the context, or made bad logical leaps. This tells us that the problem isn't just that the models are being stubborn; it's that their ability to ground their answers in the provided text is fundamentally shaky. The researchers conclude that we need to build new architectures and training methods that force models to stay grounded in the evidence, no matter how the question is phrased. Until then, even the smartest AI librarians might just be one slightly different sentence away from making up a whole new history.