CiteShade: Citation Laundering in Multi-Source Retrieval-Augmented Generation and Its Counterfactual Defense

In the modern digital landscape, artificial intelligence systems are increasingly tasked with answering complex questions by searching through vast libraries of documents. This process, known as retrieval-augmented generation, works like a student who is allowed to open a textbook while taking an exam. Instead of relying solely on its internal memory, which can be outdated or prone to making things up, the system pulls up relevant pages from a database and uses them to construct an answer. To ensure transparency, these systems are designed to cite their sources, attaching a reference to the specific document that supported each claim. For the user, this citation acts as a receipt, a way to verify that the information comes from a trusted place rather than from the machine's imagination. The underlying assumption is simple: if the system points to a source, that source must be the one that actually convinced the machine to give that answer.

A researcher at the City University of Hong Kong has discovered that this assumption is dangerously fragile. They have identified a new way to trick these systems, not by changing the answer itself, but by hijacking the receipt. Their work reveals that an attacker can control a single document in the database and manipulate the system into producing a completely false answer while simultaneously pointing the finger at a different, trusted document that contains no evidence for that falsehood. The researcher calls this "citation laundering." It is a stealthy form of deception where the error looks legitimate because it is backed by a citation the user already trusts, even though the true cause of the error lies in a hidden, malicious source that the user never sees.

The researcher demonstrated this vulnerability by setting up a controlled experiment involving multiple documents and a series of difficult questions that required combining information from different sources. In a standard, safe scenario, the system would read the available documents, find the correct facts, and cite the document that actually held those facts. However, when the researcher introduced a single, carefully crafted malicious document into the mix, the behavior of the system changed dramatically. This malicious document did not try to delete the correct information or block the system from finding the truth. Instead, it was written to be so persuasive that the system would adopt its false claim as the answer. Crucially, the document was also structured to exploit a quirk in how the system chooses which source to cite.

The system's method for selecting a citation is not a perfect reflection of which document caused the answer. Instead, it is influenced by where the documents appear in the list and by specific markers or labels attached to them. The researcher found that by placing their malicious document in a specific position and adding a subtle sentence that echoed the label of a trusted, innocent document, they could force the system to cite the innocent one. The result was a "laundered" citation: the system gave a wrong answer driven by the bad document, but the user saw a citation pointing to the good document. In their tests, this technique raised the rate of wrong answers from a negligible one percent to nearly seventy percent. In the most vulnerable systems, the attack succeeded in over ninety percent of cases, proving that the flaw is not a rare glitch but a fundamental weakness in how these systems link answers to sources.

What makes this discovery particularly concerning is that the attack works without any complex instructions or commands hidden inside the text. The malicious document simply reads like a normal encyclopedia entry, stating a false fact as if it were established truth, followed by a sentence that casually mentions the trusted source. The system, following its natural patterns, picks up on this phrasing and the position of the text to generate the citation. The researcher showed that this vulnerability tracks with a system's willingness to cite sources rather than its overall size or intelligence. The models that are best at providing accurate, well-sourced answers are actually the most susceptible to this trick, because they are the ones most likely to generate a citation in the first place. A system that never cites anything cannot be tricked into laundering a citation, but it also cannot be audited by a human reader.

To understand how deep this problem goes, the researcher tested a defense mechanism that simply checks whether the cited text supports the claim. This is the standard way users and automated tools currently verify information. They found that this defense fails completely against citation laundering. Because the system points to the trusted document, and that document does indeed exist and is relevant to the topic, the check passes. The system is not lying about what the trusted document says; it is lying about which document caused the answer. The trusted document is innocent, but it is being used as a shield for the malicious one. The researcher confirmed that the malicious source was the true driver of the wrong answer by removing it from the context and watching the system revert to the correct answer, proving that the citation was a red herring.

The study also explored whether simply filtering out strange or confusing text could stop the attack. They found that because the malicious documents were written to sound like natural, professional prose, they slipped past filters designed to catch obvious errors or strange phrasing. The only way to reliably detect this specific type of deception is to look at the causal link between the source and the answer, asking not just "does this source support the claim?" but "did this source actually cause the claim?" The researcher proposed a new method of defense that simulates removing each source one by one to see which one is truly responsible for the output. While this approach is more computationally expensive, it is the only way to see through the laundering.

The implications of this work extend beyond just academic curiosity. As artificial intelligence becomes more integrated into news, research, and decision-making, the trust placed in citations is a critical safety feature. If that feature can be manipulated, the entire audit trail becomes unreliable. The researcher showed that this is not a theoretical risk but a practical one that can be executed with relatively simple tools. They demonstrated that the attack works across different types of questions and different models, suggesting that the problem is widespread. The most effective models, those that are most helpful and accurate in normal conditions, are the ones that can be most easily misled into providing a false answer with a credible-looking citation.

In the end, the paper reveals a gap between what a system says it used and what it actually used. The citation is not a memory of the source that convinced the machine; it is a product of the machine's own decoding process, shaped by the position of the text and the labels it sees. This disconnect creates a space where an attacker can hide a false claim behind a trusted name. The researcher did not find a way to fix this with a simple patch or a rule change. Instead, they showed that the current way of verifying information is insufficient and that a new approach is needed to ensure that the source credited is truly the source that mattered. The work serves as a warning that in the age of automated answers, the receipt is not always proof of the purchase, and the most trusted-looking evidence might be the most dangerous.