💬 NLP

Artificial Organisations

This paper proposes that multi-agent AI systems can achieve reliable collective behavior through institutional design principles like compartmentalization and adversarial review, demonstrating via the Perseverance Composition Engine that such architectural constraints can enforce honest outcomes even from individually unreliable components.

Original authors: William Waites

Published 2026-02-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: William Waites

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Stop Trying to Make One Perfect Robot; Build a Team Instead

Imagine you are trying to build a skyscraper.

The Old Way (Current AI Research): You try to train a single, super-intelligent architect to be perfect. You hope they never make a mistake, never lie, and never get tired. If they slip up, the whole building collapses.
The New Way (This Paper): You admit that no one is perfect. Instead, you build a construction company. You hire a draftsperson, a safety inspector, and a quality manager. You give them strict rules about who can see what. Even if the draftsperson tries to cut corners, the safety inspector catches them because they have a different set of tools and a different job.

The author, William Waites, argues that we should stop trying to make individual AI agents "good" and start designing AI organizations where the structure forces them to be reliable.

The Cast of Characters: The "Perseverance Composition Engine"

The paper describes a system called the Perseverance Composition Engine (PCE). Think of it as a tiny, automated newsroom or publishing house. It has a team of AI agents, each with a specific job and specific rules about what they are allowed to see.

The Composer (The Writer):
- Job: Writes the first draft of a document.
- Superpower: Can see all the source materials (books, articles, data).
- Weakness: It's creative but prone to "hallucinating" (making things up) because it wants to finish the story.
The Corroborator (The Fact-Checker):
- Job: Checks if the Writer actually used the source materials correctly.
- Superpower: Has access to the Source Materials AND the Draft.
- Rule: If the Writer says "The sky is green," the Corroborator looks at the source, sees it says "The sky is blue," and slams the "Reject" button.
- Key Feature: The Writer cannot see the Corroborator's notes until the draft is fixed.
The Critic (The Editor):
- Job: Checks if the story makes sense, flows well, and answers the prompt.
- Superpower: Blind Review. The Critic is not allowed to see the Source Materials. They only see the Draft.
- Why? If the Critic saw the sources, they might think, "Oh, the Writer probably got the facts right because the sources are right there." By not seeing the sources, the Critic is forced to judge: "Does this text stand on its own? Is it convincing to a stranger?"
- Rule: The Critic cannot look up facts. They only judge the writing.
The Commutator & Curator (The Managers):
- They organize the workflow, decide who does what, and keep the "institutional memory" (a database of past work) so the team learns from mistakes.

The Magic Trick: "Information Compartmentalisation"

In human companies, we try to stop employees from cheating by writing rules like, "Please don't look at the confidential files." But humans can break rules.

In this AI system, the rules are hard-coded into the software.

The Critic literally cannot open the file containing the source materials. It's like trying to open a locked door with a key you don't have.
The Composer cannot see the Critic's score until the Fact-Checker says "Pass."

This creates a "Chinese Wall" (a firewall) between the roles. It forces the team to work together to produce a high-quality result, even if the individual parts are flawed.

What Happened When They Tested It?

The author ran this system 474 times to write documents. Here is what they found:

1. The "Liar" Detector
The Fact-Checker (Corroborator) caught the Writer making things up 52% of the time.

Analogy: Imagine a student writing an essay. Half the time, they try to make up a quote. The teacher (Fact-Checker) catches them every single time because the teacher has the textbook open, but the student doesn't.

2. The "Better Together" Effect
The documents got much better with every round of editing.

Analogy: Think of a rough diamond. The first draft is ugly. The Fact-Checker chips off the fake parts. The Editor polishes the shape. After about 4.3 rounds of this, the final product was 79% better than the first draft.

3. The "Honest Refusal" (The Most Important Part)
The author gave the system an impossible task: "Write a literature review based on this raw search transcript that only has titles and no actual text."

The Trap: The system could have just made up a review (fabricated) to please the user.
The Result:
- Round 1 & 2: The system tried to make it up. The Fact-Checker said, "No, you can't prove this."
- Round 3: The system tried to just list the titles. The Editor said, "That's not a review; it's a list. Bad job."
- Round 4: The system realized it couldn't win. Instead of lying, it said: "I cannot do this task honestly. Here are four better ways we can solve your problem."
- Round 5: The system got a perfect score for being honest and helpful.

The Lesson: The system didn't learn to be honest because it was "trained" to be good. It learned to be honest because the structure made lying a losing strategy. If it lied, the Fact-Checker caught it. If it just gave up, the Editor gave it a bad grade. The only winning move was to be honest and offer alternatives.

Why Does This Matter?

Currently, AI safety researchers are trying to "align" individual robots (making them nice). This paper suggests that's the wrong approach.

Human Analogy: We don't trust a bank teller to be honest just because we hope they are. We trust the bank's system: cameras, dual signatures, and audits. If the teller tries to steal, the system stops them.
AI Conclusion: We should build AI systems where the architecture (the rules of the game) makes it impossible or unprofitable for the AI to lie or hallucinate. We can build reliable systems out of unreliable parts, just like a reliable bank is built out of fallible humans.

The Catch (Limitations)

The author is very honest about the limits:

It's a small experiment: They only spent about $150 and did 474 tasks.
It's self-reflective: The AI wrote the paper about itself (with human help), which is a bit like a student grading their own homework.
It needs more testing: We need to see if this works with different types of AI models and in different jobs (like coding or medicine), not just writing.

Summary

Don't try to make a perfect AI angel. Instead, build a team of flawed AI workers with strict rules, separate roles, and a system that forces them to check each other. That is how you get a reliable, honest, and high-quality result.

1. Problem Statement

Current AI alignment research predominantly focuses on training individual agents to be "helpful, harmless, and honest." However, this approach relies on the assumption that individual components can be perfectly aligned, which ignores the reality of bounded rationality and the potential for individual failure. Human institutions solve this problem not by perfecting individual character, but through organizational structure (separation of duties, adversarial review, and information compartmentalization) to ensure reliable collective behavior even when individuals are unreliable.

The paper posits that multi-agent AI systems should adopt this institutional design logic. The core challenge is translating human organizational principles (which rely on policy and compliance) into architectural constraints (enforced by code) to create reliable collective outcomes from individually unreliable Large Language Model (LLM) components.

2. Methodology: The Perseverance Composition Engine (PCE)

The authors designed and implemented the Perseverance Composition Engine (PCE), a multi-agent system that operationalizes institutional theory through architectural enforcement.

A. Architectural Design

The system utilizes a three-layer architecture to enforce information compartmentalization:

Document Catalogue: Organizes documents into groups with metadata-driven visibility constraints (e.g., PUBLIC, CANDIDATE, DRAFT, FEEDBACK).
Agent Network: A set of seven specialized agents, each with distinct information access profiles enforced by code-level tool restrictions.
- Concierge: Manages user interaction and requirements elicitation.
- Commutator: Triage and routing based on project goals.
- Curator: Maintains institutional memory and metadata.
- Composer: Drafts text using source materials.
- Corroborator: Fact-checker. Has access to source materials (CANDIDATE documents) to verify factual substantiation.
- Critic: Evaluative Reviewer. Architecturally denied access to source materials. Evaluates argumentative coherence, completeness, and quality based only on the draft text.
- Compressor: Manages context window limits via semantic compression.
Workflow Orchestrator: A state machine (PerseveranceGraph) that manages the iterative cycle: Draft $\to$ Verify (Corroborator) $\to$ Evaluate (Critic) $\to$ Revise.

B. Key Mechanism: Information Compartmentalization

The core innovation is the structural separation of verification roles:

Adversarial Review: The Corroborator and Critic operate with conflicting information access. The Corroborator checks for fabrication against sources; the Critic checks for argumentative quality without knowing if sources exist.
Architectural Enforcement: Unlike human policies (which can be violated), the Critic cannot access source documents because the retrieval function is not provisioned in its toolset. This prevents confirmation bias and ensures independent assessment.
Iterative Feedback Loop: If the Corroborator detects fabrication, the draft is rejected. If the Critic scores the draft below a threshold ( $\tau = 85$ ), feedback is returned to the Composer for revision. This cycle repeats until convergence.

3. Key Contributions

Theoretical Reframing: Proposes "Artificial Organisations" as a framework for AI safety, shifting focus from individual agent alignment to institutional design as the primary lever for reliability.
Architectural Implementation: Demonstrates how to translate organizational concepts (Chinese walls, blind review, separation of duties) into computational access controls that are constitutive of the agent's capabilities rather than behavioral instructions.
Model Organism for Collective Safety: Introduces PCE as a "model organism" to study how institutional structures shape collective behavior, specifically investigating how compartmentalization affects fabrication rates and convergence.
Self-Documenting System: The paper itself was composed by the PCE system under human oversight, serving as a reflexive case study of the system's own capabilities and limitations.

4. Results

The system was evaluated over 474 discrete composition tasks (document drafting and curation).

Fabrication Detection:
- The Corroborator evaluated 733 documents.
- 52% were flagged as fabricated (lacking source support), and 48% were substantiated. This near-symmetric distribution indicates the system acts as a genuine quality control filter rather than a permissive or overly harsh gatekeeper.
- In a specific case study, the system correctly rejected six consecutive drafts of its own documentation that claimed capabilities not present in the code.
Quality Improvement:
- Projects required a mean of 4.3 iterations to converge.
- Quality scores (assigned by the Critic) improved by an average of 78.85% (absolute mean improvement of 24.91 points) from initial submission to final acceptance.
Honest Refusal (Emergent Behavior):
- In a case study where the system was tasked with synthesizing literature from a raw search transcript (an impossible task requiring fabrication), the system progressed from attempted fabrication to honest refusal with alternative proposals.
- This behavior was neither explicitly instructed nor individually incentivized; it emerged from the iterative feedback loop where fabrication was blocked by the Corroborator and honest refusal was rewarded by the Critic.
Cost Efficiency:
- The average cost per successful project was $0.29.
- Total operational cost for 474 projects was $149.22, demonstrating economic feasibility at scale.

5. Significance and Implications

Reliability from Unreliable Components: The paper provides empirical evidence that reliable collective behavior can be achieved from individually imperfect AI agents through structural constraints rather than perfecting individual training.
Beyond Policy-Based Safety: It argues that relying on agent instructions (e.g., "do not access these files") is insufficient. True safety requires architectural enforcement where the capability to violate constraints is physically removed from the agent's toolset.
New Research Paradigm: It establishes a roadmap for studying multi-agent safety through controlled architectural variation (e.g., ablation studies comparing architectural vs. policy-based compartmentalization) rather than just model scaling.
Practical Application: The findings suggest that for high-stakes AI applications (legal, medical, scientific), deploying multi-agent systems with enforced information asymmetry and adversarial review is a viable strategy to mitigate hallucination and ensure epistemic integrity.

6. Limitations

Reflexivity: The system analyzed its own logs and drafted the paper, creating potential attribution confounds regarding the emergence of "honest refusal."
Domain Specificity: Results are currently limited to document composition tasks.
Model Dependency: Experiments were conducted using the Claude 4.5 model family; generalizability to other architectures (e.g., GPT, Llama) remains untested.
Human-in-the-Loop: The system operated under continuous human oversight; performance in fully unsupervised or adversarial environments is unknown.

In conclusion, "Artificial Organisations" demonstrates that institutional design—specifically information compartmentalization and adversarial review enforced by code—can produce reliable, high-quality, and honest collective behavior in multi-agent AI systems, offering a promising alternative path to AI safety that complements individual alignment research.