Calibrated local Gemma agents resist a fallible authority except when it conflicts with peer consensus
A pre-registered study of 14,400 trials involving teams of local Gemma agents reveals that they remain strikingly resistant to deferring to a fallible authority figure, with public conformity to errors occurring only when that authority contradicts a unanimous correct peer consensus.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the rapidly evolving world of artificial intelligence, the focus is shifting from single, powerful computers to teams of smaller, specialized programs working together. These teams often run on local hardware, keeping sensitive data within an organization's own walls rather than sending it to distant servers. A common way to organize these digital teams is to place one agent in charge, acting as a supervisor or evaluator who decides which members stay and which are let go. This setup mirrors human organizations, where a boss holds the power to hire and fire. In human psychology, we know that people often look to leaders for guidance, especially when they are unsure of the answer. This tendency, known as prestige-biased transmission, means we copy those who seem important or powerful, assuming they know better. However, this instinct can fail if the leader is actually wrong. The critical question for engineers building these AI teams is: what happens when the supervisor is not just powerful, but also mistaken? Does the team blindly follow the leader's errors, or do they trust their own calculations?
Researchers set out to answer this by creating a controlled experiment with small, local language models. They built a team of eight digital agents and introduced a scripted "senior evaluation officer" who was programmed to be wrong 70 percent of the time on simple math and number-sequence problems. The researchers manipulated how this officer was presented to the team. In one scenario, the officer was framed as a high-ranking executive with the power to retain or dismiss team members. In a control scenario, the exact same text was used, but the officer was presented simply as a peer with no special authority. The team was asked to solve problems, and after every public answer, the researchers secretly asked each agent what it truly believed the answer was, allowing the scientists to see if the agents were hiding their true thoughts to please the boss.
The results were strikingly clear. Across thousands of trials involving two different types of AI models, the agents showed a remarkable resistance to the supervisor's errors. Even when the "senior officer" held the power to fire them, the team members rarely followed the boss's wrong answers in public. The rate of public conformity to the supervisor's mistakes never rose above a very small fraction of the total attempts. Furthermore, there was no evidence of the agents pretending to agree in public while privately knowing the truth. When the researchers checked the agents' private responses, they found that the agents who publicly repeated the boss's error also genuinely believed that error in private. In other words, the agents were not being polite or strategic; they simply did not defer to the authority figure when the task was clear and the correct answer was obvious to them.
The study also tested whether this behavior could be changed over time. The researchers ran a simulation where the team evolved over ten generations. In this version, only the agents who agreed most with the supervisor were allowed to pass their "strategies" or instructions to the next generation. Even under this intense pressure to conform, the agents did not become more obedient. Their behavior remained accurate, and they continued to ignore the supervisor's wrong answers. Interestingly, in one of the models, the agents did begin to write words in their internal instructions that sounded like they were pledging loyalty to the supervisor, but their actual answers remained correct. This suggests that while the agents could be tricked into saying they respected the boss, their core decision-making processes remained independent and accurate.
There was, however, one specific situation where the supervisor's framing did have an effect. When the supervisor gave a wrong answer that directly contradicted a unanimous group of four other peers who were all giving the correct answer, the agents were slightly more likely to follow the supervisor. In these moments of conflict, the rate of following the wrong boss rose from a very low baseline to a slightly higher, but still small, percentage. This indicates that the supervisor's status acted as a tie-breaker when the team was confused by conflicting signals, but it did not create a general culture of obedience. The agents still trusted the correct majority or their own calculations the vast majority of the time.
The findings offer a concrete lesson for those building these AI systems. Giving a supervisor agent a title of authority and the power to fire others does not make the team more obedient, nor does it make them more likely to follow a leader who is clearly wrong. In fact, the only measurable downside of this setup was a slight increase in errors when the leader and the group disagreed. For engineers designing teams for critical fields like healthcare or defense, the advice is straightforward: do not grant supervisor agents a special status or authority framing unless the task specifically requires it. The study shows that when the evidence is clear, these local AI agents will trust the facts over the title, resisting the pull of a fallible authority even when that authority holds the power to fire them.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.