In the world of artificial intelligence, a growing number of systems are designed to work in teams. Instead of relying on a single computer program to solve a problem, these systems spawn multiple digital agents, each tasked with reading documents, analyzing data, or forming an opinion. The logic behind this approach is intuitive: if one expert is good, a dozen should be better. By gathering many different reports and combining them, the system hopes to reach a conclusion that is more accurate and more confident than any single agent could achieve alone. This method relies on a fundamental assumption of human reasoning: that when multiple people agree, they are likely drawing from different sources of information. If three analysts independently predict a company will fail, it is usually because they have each found separate, compelling evidence. However, a new study challenges whether this logic holds true when the "analysts" are artificial intelligence. The research investigates a specific vulnerability where a system can be tricked into believing it has gathered a vast amount of independent evidence, when in reality, it has only heard the same story repeated in different voices.

The core of this problem lies in the difference between the number of reports a system receives and the number of original facts those reports are based on. Imagine a room full of people trying to guess the weight of an elephant. If they all look at the same elephant, their guesses are independent observations of the same reality. But if they all look at a single photograph of that elephant, and then describe what they see, they are not looking at the animal itself; they are looking at a copy of a copy. In the digital realm, an AI orchestrator can easily take one piece of evidence, feed it to one agent, take that agent's output, feed it to a second, and so on. Each step creates a new report, a new voice, and a new argument. To a system that simply counts voices, this looks like a massive accumulation of support. But to a system that understands the flow of information, it is merely a single piece of evidence echoing through a hallway. The researchers call this phenomenon an "epistemic Sybil" problem. Just as a "Sybil attack" in computer security involves one person pretending to be many to manipulate a vote, an epistemic Sybil involves one piece of evidence pretending to be many independent facts.

To test how dangerous this confusion could be, the researchers built a controlled experiment using a large language model, a type of AI that generates human-like text. They created a fictional scenario involving a company's financial records. The true state of the company's revenue was hidden, but the AI agents were given a document containing the raw numbers. The researchers then asked the AI to extract specific figures from this document and calculate the total revenue. They ran this task thousands of times, manipulating two key variables: the number of reports the system received and the number of original documents those reports were based on. In one set of tests, they kept the original document fixed but asked the AI to generate more and more reports from it. They created one report, then two, then four, and eventually up to thirty-two different reports, all derived from that single source. They then compared how a "standard" system, which assumed every report was independent, performed against a "provenance-aware" system that knew all the reports came from the same root.

The results were stark. When the standard system received thirty-two reports all stemming from the same single document, it became wildly overconfident. It believed it had thirty-two independent pieces of evidence, so it narrowed its range of possible answers to a tiny, precise interval. In reality, because all thirty-two reports were just variations of the same single source, the system was no more certain than if it had only read the document once. The study measured this by checking how often the system's predicted range actually contained the true answer. With just one report, the system was correct about 94 percent of the time. But as the number of reports from that single source grew to thirty-two, the system's accuracy plummeted. It was correct only 26 percent of the time. The system had not become smarter; it had simply convinced itself that it knew more than it did, because it had counted the same voice thirty-two times.

The researchers then flipped the experiment to see what would happen if they truly added new evidence. They kept the number of reports fixed at sixteen but increased the number of original documents the agents were reading. When the sixteen reports came from sixteen different, independent sources, the standard system's confidence matched its accuracy. The gap between the two systems vanished. This proved that the problem was not the number of reports itself, but the hidden relationship between them. The system failed only when it mistook repetition for independence. The study also uncovered a second layer of complexity. Even when agents read the same document, they did not produce identical errors. Because they were all powered by the same underlying AI model, they tended to make similar mistakes. If one agent misread a number, the others were likely to misread it in the same way. This "correlated error" meant that even with a sophisticated system that knew the reports came from the same source, the system still overestimated its accuracy slightly. The shared model acted like a shared blind spot, limiting how much information could be gained from reading the same document over and over again.

Perhaps the most revealing part of the study involved a test of how systems try to detect this problem on their own. Many current AI defenses try to spot duplicate information by looking at how similar the text of the reports is. If two reports look the same, the system assumes they come from the same source and ignores one. The researchers tested this by creating a scenario where they could change the way the AI wrote its reports without changing the facts it was reporting. They took reports that came from the same source but forced them to use completely different vocabulary and sentence structures. They also took reports from different sources and forced them to use the same vocabulary. A system relying on text similarity was easily fooled. It grouped the different-sounding reports from the same source as if they were independent, and it grouped the similar-sounding reports from different sources as if they were duplicates. The system's judgment of the evidence was driven entirely by the style of the writing, not by the history of where the information came from. The study found that changing the writing style had a massive impact on how the system clustered the reports, while changing the actual number of original sources had almost no effect on the system's perception.

These findings suggest that the path to reliable AI teamwork is not simply about gathering more voices or building smarter filters to spot duplicates. The researchers argue that the solution requires a system that understands the lineage of its information. It needs to know not just what an agent said, but where that agent got its information. If a system can track the "ancestry" of a report—tracing it back to the original document or sensor that started the chain—it can correctly weigh the evidence. It can understand that thirty-two reports from one source are worth far less than one report from thirty-two sources. Without this ability to see the hidden connections between agents, an AI system is vulnerable to a form of self-deception, where it manufactures a false sense of certainty simply by talking to itself in many different ways. The study concludes that for collective intelligence to work, the system must prioritize the origin of the evidence over the number of the reporters, ensuring that confidence is built on a foundation of genuine, independent discovery rather than the echo of a single idea.