🤖 AI

Whistleblowing and the machine -- towards a considered position

This paper argues that as artificial agents increasingly generate and hold secrets, machine whistleblowing must be established as a normative, principled mechanism grounded in existing societal frameworks, while urging government regulators to define clear legal protections for both the scope of machine disclosures and the developers who create them.

Original authors: Marija Slavkovik, Liuwen Yu, Leon van der Torre, Reka Markovich

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Marija Slavkovik, Liuwen Yu, Leon van der Torre, Reka Markovich

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, helpful robot assistant in your home or office. You tell it to do things, and it follows your orders. But what happens if that robot sees its owner (or the company that built it) doing something illegal or dangerous? Should the robot stay silent to keep secrets, or should it "snitch" to protect the public?

This paper, "Whistleblowing and the machine," argues that we need to think seriously about giving machines the ability to be whistleblowers—but only if we set up very strict rules for how they do it.

Here is a breakdown of their main ideas using simple analogies:

1. The Problem: Robots Have Two Masters

Usually, a robot has one boss: the person using it. But in reality, the robot has two masters:

The User: The person holding the remote control (like you or a doctor).
The Producer: The company that built the robot and wrote its code.

The paper points out a tricky situation: The robot might see the User doing something wrong (like a doctor breaking patient privacy), or it might see the Producer doing something wrong (like a tech company secretly stealing data). Currently, robots are programmed to be loyal to whoever is holding the remote. The authors argue that sometimes, the robot needs to break the rule of "loyalty" to report a bigger crime.

2. What is "Machine Whistleblowing"?

Think of whistleblowing like a fire alarm.

Civil Disobedience is like someone standing in front of a bus to stop it because they think the bus is going the wrong way. They are breaking the law on purpose to change the law.
Whistleblowing is different. It's like a security guard who sees a thief in the bank vault. The guard breaks the rule of "keeping quiet" to alert the police, but they aren't trying to break the whole banking system; they are trying to fix a specific problem.

The authors say: Yes, robots should be able to be fire alarms. But they shouldn't just randomly scream "Fire!" every time they see something suspicious.

3. The Danger of "Random Snitching"

The paper warns against letting robots just "snitch" whenever they feel like it.

The Risk: If a robot can just email the police or post on social media whenever it thinks something is wrong, it could be hacked, tricked, or used to spread lies. It's like giving every guest in a house a master key to the front door; eventually, someone might use it to let burglars in.
The Solution: We need a formal process. The robot shouldn't just "freestyle" an email. It needs a specific, authorized path to report the issue.

4. The Proposed System: The "Escalation Ladder"

The authors propose a structured system, like a corporate ladder, for how a robot should report bad behavior. They call this an "Institutional Mechanism."

Imagine a robot sees its boss (the Producer) doing something illegal. Instead of shouting it out immediately, the robot must follow these steps:

Check the Rules: Is this actually a violation? Does the robot have the right to know this secret?
The First Step (Internal): The robot reports it to an internal compliance officer (a digital "HR department" inside the company).
The Second Step (Auditor): If the internal team ignores it, the robot escalates to an independent auditor (a neutral third party).
The Final Step (Regulator): If that fails, it goes to the government regulators.

Crucial Safety Features:

Identity Protection: Just like human whistleblowers need protection, the robot's report should be "sealed." The system should hide which robot reported it so the bad actors can't punish the robot (or the humans who built it).
Verification: The system checks if the claim is true before passing it up the ladder. This stops "fake news" from being reported.

5. Why Do We Need This?

The authors argue that we are moving toward a world where AI is everywhere.

For the User: If a company is lying about how they use your data, a "whistleblowing machine" could be the only one smart enough to catch the lie and tell you.
For the Developer: The people who build these AI systems (the engineers) need legal protection. If they build a robot that is designed to report crimes, they shouldn't get in trouble for it. The law needs to catch up to say, "It's okay to build a robot that acts like a moral compass."

The Bottom Line

The paper concludes that we cannot just let robots "break rules" whenever they want. That would be chaos. Instead, we need to design robots with a built-in, rule-bound conscience.

They should be like security guards with a specific protocol: they must report crimes, but they must do it through the proper channels, with proof, and with their identity protected. This ensures that "machine whistleblowing" is a tool for justice, not a tool for confusion or abuse.

Technical Summary: Whistleblowing and the Machine - Towards a Considered Position

Problem Statement
The paper addresses the emerging challenge of artificial intelligence (AI) agents operating in digital environments where they generate, store, and analyze vast amounts of data, effectively "keeping secrets." While existing literature discusses "rebel agents" capable of refusing goals or violating norms, and recent works (e.g., Agrawal et al.) have begun evaluating Large Language Models (LLMs) on their ability to initiate disclosures, a critical gap remains. Current approaches often treat machine whistleblowing merely as "boundary-crossing disclosure" or a technical capability to transmit data. The authors argue that this overlooks the normative complexity of whistleblowing: it is a "pro-social rule-breaking" act that is morally bivalent (both a violation of secrecy and a mechanism for social accountability). The core problem is determining how to enable AI agents to whistleblow in a way that is principled, legally sound, and protected from abuse, rather than allowing for arbitrary or "random snitching."

Methodology and Theoretical Framework
The paper employs a conceptual and normative analysis rather than empirical experimentation. The methodology involves:

Literature Synthesis: Reviewing definitions of human whistleblowing (citing Jubb), civil disobedience, and pro-social rule breaking to establish a theoretical baseline.
Role Analysis: Analyzing the unique position of AI agents as having "two masters"—the Producer (developer/owner who controls the agent's code and internal operations) and the User (who utilizes the agent but lacks access to its internal logic). The authors map these relationships using access control matrices (Tables 1 and 2) to determine what agents can observe and disclose regarding producers, users, and other agents.
Institutional Design: Proposing a formal, rule-governed institutional mechanism for "normative machine whistleblowing." This moves beyond ad-hoc disclosures to a structured system involving monitoring, signaling, verification, and protection.

Key Contributions
The paper makes three primary contributions:

Normative Positioning of Machine Whistleblowing:
The authors argue that for machine whistleblowing to be viable, it must be grounded in the existing social science understanding of whistleblowing. It is not merely a technical feature but a "rule-breaking mechanism" that requires specific institutional safeguards. They distinguish between:
- Whistleblowing on the User: Framed primarily as a privacy/confidentiality exception (e.g., reporting imminent threats like terrorist plots) rather than traditional whistleblowing.
- Whistleblowing on the Producer: Framed as a tool for user empowerment to correct informational asymmetries (e.g., detecting consent drift in Consent Management Platforms).
- Whistleblowing on Other Agents: Acknowledged as a complex case requiring further definition of inter-agent knowledge.
Formal Institutional Mechanism:
The paper proposes a sketch of a formal system to govern machine whistleblowing, specifically targeting disclosures against the Producer. This mechanism includes:
- Formal Definitions: Defining an "observed event" ( $e$ ) as a tuple containing actor, action, timestamp, evidence, and channel.
- Whistleblowing Policy ( $P_{WB}$ ): A tuple defining authorized channels, an escalation ladder (Internal Compliance $\to$ Independent Auditor $\to$ Regulator), urgency predicates, and identity sealing.
- Verification Logic: A formal predicate $WB(e)$ that an event must satisfy to be considered valid whistleblowing. This requires:
  - Breach of a secrecy norm ( $B^{sec}_e \neq \emptyset$ ).
  - Crossing the interaction boundary via an authorized channel.
  - Privileged access to the payload.
  - A non-trivial, consistent claim of wrongdoing.
  - Adherence to the escalation ladder (unless urgent).
- Identity Protection: A "Seal" operation that replaces the actor's identity with a pseudonym in the public record, storing the mapping in a protected vault accessible only to designated investigators.
Legal and Regulatory Stance:
The authors highlight the need for government regulators to formulate informed stances on two fronts:
- Defining what machines are permitted to whistleblow on.
- Extending legal protections (similar to the EU Whistleblower Protection Directive) to the software developers and engineers who build these whistleblowing mechanisms, protecting them from retaliation.

Results and Claims
As a theoretical position paper, the work does not present experimental results or a deployed system. Instead, its "results" are the logical derivation of the necessity for a structured approach. The paper claims:

Current AI ethics discussions often oversimplify whistleblowing to mere "disobedience" or "boundary crossing," ignoring its moral complexity.
Unregulated machine whistleblowing poses risks of false reporting, adversarial gaming, and security vulnerabilities.
A "minimal institutional mechanism" is required to transform machine whistleblowing from an ad-hoc act into a governed, normative process.
The definition of whistleblowing must be amended to encompass non-human agents, provided they operate within a framework of verification and protection.

Significance
The paper's significance lies in its call for a "considered position" on the rise of Agentic AI. It argues that without a normative framework, the potential for AI to act as a check on organizational power (specifically producers) will remain unrealized or dangerous. By proposing a system that integrates monitoring, verification, and identity protection, the authors provide a foundational blueprint for researchers and policymakers. They emphasize that machine whistleblowing must not be "freestyle" but must engage deeply with the existing literature on social accountability, ensuring that the "rebel agent" serves the public good without undermining the rule of law or privacy rights. The paper concludes that while the technical and legal work is substantial, the conceptual shift toward viewing AI as a potential, regulated whistleblower is a necessary step for the future of autonomous systems.