🤖 AI

Consensus is Strategically Insufficient: Reasoning-Trace Disagreement as a Knowledge-Representation Signal

This paper proposes a knowledge-representation framework that abstracts multi-agent reasoning traces and decisions into four symbolic disagreement states to enable strategic routing in value-laden tasks, arguing that preserving rather than eliminating disagreement is crucial for addressing normative uncertainty.

Original authors: Michał Wawer, Jarosław A. Chudziak

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Michał Wawer, Jarosław A. Chudziak

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a team of five AI assistants tasked with deciding whether a specific piece of online content should be kept or removed. In most current systems, the goal is to get these five assistants to agree as quickly as possible. If they disagree, the system usually treats it as a mistake, forces a vote, or asks them to debate until they reach a "consensus."

This paper argues that forcing agreement is actually a bad idea for tricky, value-based decisions (like content moderation). Sometimes, disagreement isn't a glitch; it's a feature. It might mean the agents are looking at the same facts but weighing different values (like "free speech" vs. "safety").

Here is the core idea, broken down with simple analogies:

The Problem: The "Voting Machine" Flaw

Think of a standard multi-agent system like a jury that is forced to deliver a single verdict immediately. If the jury members are split 3-to-2, the system just picks the majority and moves on.

The Flaw: This ignores why they disagreed. Did they disagree because they saw different facts? Or did they see the exact same facts but have different moral priorities? The current system treats both situations the same way: "Just pick a winner."

The Solution: The "Disagreement Map"

The authors propose a new layer of thinking that doesn't just look at the vote, but looks at the reasoning behind the votes. They create a "map" with four distinct zones based on two questions:

Did they agree on the conclusion? (Keep or Remove?)
Did they agree on the reasoning? (Did they use similar logic?)

This creates four "states" of disagreement, like four different rooms in a house:

1. Convergent Agreement (CA)

The Scenario: Everyone agrees on the decision, and they all used the same logic.
The Metaphor: A choir singing the same note in perfect harmony.
The Action: Auto-approve. No need to think further; the system can safely make the decision automatically.

2. Divergent Agreement (DA)

The Scenario: Everyone agrees on the decision, but they used totally different reasons to get there.
The Metaphor: Five people deciding to buy a car. One wants safety, one wants speed, one wants style. They all agree on the car, but for different reasons.
The Action: Auto-approve with explanation. The system makes the decision but keeps all the different reasons visible, because different people might care about different explanations.

3. Divergent Disagreement (DD)

The Scenario: They disagree on the decision, and they also used totally different logic.
The Metaphor: A group of people looking at a blurry photo. One thinks it's a dog, another thinks it's a cat, and a third thinks it's a shadow. They can't agree because they aren't even sure what they are looking at.
The Action: Ask for more context. The system realizes it doesn't have enough information yet. It shouldn't escalate to a human yet; it should just ask for more data or try again.

4. Convergent Disagreement (CD) — The Most Important One

The Scenario: They disagree on the decision, but they used the exact same logic.
The Metaphor: Five judges looking at the same clear photo of a dog. They all agree it's a dog. But, Judge A says, "Dogs are dangerous, remove it," while Judge B says, "Dogs are cute, keep it." They see the same reality but have a fundamental clash of values.
The Action: Escalate to a human. This is the paper's big insight. If the AI agents agree on the facts but disagree on the value, it's a genuine moral conflict. Forcing the AI to pick a winner here would hide a real human dilemma. This is the signal that says, "Stop, a human needs to weigh in on this value clash."

Why This Matters

The paper suggests that instead of trying to fix disagreement (by voting it away), we should use disagreement as a signal.

Old Way: "We have a disagreement? Let's force a vote and pretend it's solved."
New Way: "We have a disagreement. Let's check the map. Is it a 'blurry photo' situation (get more info) or a 'clashing values' situation (call a human)?"

The "Test Drive"

The authors tested this idea using a dataset of hate speech examples. They simulated five AI agents with different "personalities" (e.g., one focused on safety, one on free speech).

The Result: The system successfully sorted the cases into the four categories.
The Proof: The cases where the AI agents had "Convergent Disagreement" (same logic, different values) were the ones where human reviewers also disagreed the most. This proves that the system's "map" correctly identified the hardest, most value-laden cases that humans struggle with.

Summary

This paper argues that in complex, value-heavy tasks, disagreement is a useful tool, not a bug. By categorizing how agents disagree, we can build systems that know exactly when to act automatically, when to ask for more info, and when to wisely step back and let a human handle the moral dilemma.

Technical Summary: Consensus is Strategically Insufficient

Problem Statement
Current LLM-based multi-agent systems are predominantly designed to minimize inter-agent disagreement through mechanisms like majority voting, debate rounds, or fault-tolerant aggregation. While effective for instrumental tasks where disagreement signals noise or reasoning failure, this consensus-seeking objective is argued to be strategically insufficient for value-laden tasks (e.g., content moderation). In such domains, disagreement often reflects genuine normative uncertainty, perspectival variation, or value pluralism rather than agent error. Treating all disagreement as a defect to be averaged away risks obscuring legitimate contested situations and fails to distinguish between interpretive ambiguity and evaluative conflict.

Methodology
The authors propose a Knowledge-Representation (KR) layer that abstracts agent outputs into symbolic disagreement states, serving as an interface between sub-symbolic LLM deliberation and symbolic strategic control.

Formal Model:
- A system consists of $n$ agents, each producing an output $O_i(c) = \langle r_i, d_i, v_i, \gamma_i \rangle$ , where $r_i$ is an explicit reasoning trace, $d_i$ is a binary decision (e.g., KEEP/REMOVE), $v_i$ is a value profile, and $\gamma_i$ is a confidence score.
- The KR layer evaluates two dimensions:
  - Reasoning Similarity: Measured by semantic similarity $sim(r_i, r_j)$ of reasoning traces. A threshold $\theta_s$ determines if similarity is High or Low.
  - Conclusion Agreement: Measured by the proportion of agents agreeing on a decision. A threshold $\theta_a$ determines if the group is in Agreement or Disagreement.
Symbolic Taxonomy:
Combining these dimensions yields four distinct epistemic states:
- Convergent Agreement (CA): High reasoning similarity + Conclusion agreement.
- Divergent Agreement (DA): Low reasoning similarity + Conclusion agreement.
- Convergent Disagreement (CD): High reasoning similarity + Conclusion disagreement.
- Divergent Disagreement (DD): Low reasoning similarity + Conclusion disagreement.
Defeasible Strategic Routing:
The system maps these states to strategic meta-actions using defeasible rules (non-monotonic reasoning), allowing for overrides based on domain-specific constraints (e.g., legal requirements or risk levels):
- CA $\Rightarrow$ Auto: Automatically accept the decision.
- DA $\Rightarrow$ AutoExplain: Accept the decision but preserve diverse explanations for stakeholders.
- DD $\Rightarrow$ SeekContext: Request additional information or further deliberation (indicating unstable interpretation).
- CD $\Rightarrow$ Escalate: Forward to human judgment (indicating shared interpretation but conflicting value weightings).

Key Contributions

Reframing Disagreement: The paper reframes disagreement not as an aggregation obstacle but as a representable epistemic state of the multi-agent system.
Compact Taxonomy: It defines a four-state taxonomy based on the intersection of reasoning similarity and conclusion agreement, distinguishing between cases where agents disagree due to ambiguity (DD) versus value conflict (CD).
Strategic Routing Policy: It associates these states with defeasible routing rules, enabling the system to reason about whether to decide, inquire, or escalate, rather than solely what to decide.
Empirical Faithfulness Check: The authors validate the KR layer using a content moderation setup with five LLM agents differentiated by value profiles (harm-focused, free-expression, etc.) on the Measuring Hate Speech corpus.

Results

Distribution and Ordering: In a sample of 600 items, the distribution of cases across states preserved the conceptually predicted ordering of human annotator disagreement: Divergent Agreement (DA) < Convergent Agreement (CA) < Divergent Disagreement (DD) < Convergent Disagreement (CD).
Statistical Significance: The two disagreement states (CD, DD) were jointly separated from the agreement states (CA, DA) with a large effect size (Cohen's $d = 0.80$ ), suggesting the structural abstraction tracks phenomena humans also perceive.
Predictive Performance: A category-based routing approach (using the four states) achieved a higher F1 score (0.548) in flagging high human-disagreement cases compared to a baseline using only reasoning divergence magnitude (0.503). The category-based approach improved precision by distinguishing between ambiguous cases (DD) and normative conflicts (CD), which a magnitude-only metric conflates.

Significance and Claims
The paper claims that the structure of disagreement carries information that its magnitude alone does not. Specifically, Convergent Disagreement (CD) is identified as a strong signal of normative conflict where agents share a description of a case but prioritize values differently. In such cases, forcing consensus is argued to be a failure of automation; instead, escalation is a rational meta-action.

The framework positions itself as a pragmatic interface between sub-symbolic deliberation and symbolic strategic control. It bridges the gap between judgment aggregation (which combines votes without exposing reasoning) and formal argumentation (which models full attack/support graphs). The authors emphasize that this is a "sanity check" on the KR layer's faithfulness rather than a benchmark of the routing policy itself, noting that the qualitative ordering of states is the primary load-bearing result. The work suggests that for value-laden tasks, the goal of multi-agent systems should shift from "how to make agents agree" to "what the structure of disagreement implies for the next action."