Imagine two people sitting at a table, looking at the exact same pile of documents, charts, and news reports. Yet, one person concludes, "We need to stop this project immediately," while the other says, "We should double down and move faster."

In the real world, we often react to this by saying, "One of them is crazy," "They are lying," or "They just don't get it." We assume the problem is a character flaw.

This paper argues that we are looking at the wrong thing. It suggests that the disagreement isn't about who is looking, but how they are looking. The author, Toru Takahashi, proposes that when people share the same facts but reach different conclusions, it's not a defect in their brains—it's a mathematical inevitability called non-identifiability.

Here is the paper's argument broken down into simple concepts and analogies.

1. The Core Idea: The "Same Input, Different Output" Problem

The paper starts by rejecting the idea that there is only one "correct" way to think (which it calls the Single Intelligence Assumption). Instead, it suggests that thinking is like a machine with many dials. Even if two people feed the exact same data into their brains, if they turn the dials differently, they will get different answers.

The author splits this into two levels of "glitches":

Level 1: The Settings Glitch ( $\theta$ -level). Imagine two chefs using the exact same recipe and the exact same ingredients. One chef decides to add a pinch of salt, cook it for 5 minutes, and taste it immediately. The other chef adds no salt, cooks it for 20 minutes, and tastes it slowly. They end up with different dishes, not because the ingredients were bad, but because their settings were different.
Level 2: The Memory Glitch ( $W$ -level). Now, imagine those chefs keep cooking every day. The first chef only ever cooks dishes that are salty and fast. The second only cooks slow, bland dishes. Over time, their memory of what "good food" is changes. They have built different internal models of the world. Now, even if you give them the same new ingredient, they will interpret it differently because their past experiences have shaped their brains to expect different things.

2. The Four Dials of Thinking

To explain why people think differently, the author introduces a "Thinking Profile" with four adjustable dials. Think of these as the settings on a camera or a video game:

Reference (R): What do you trust?
- Do you trust hard numbers, logs, and legal text (things you can show a friend and say, "Look, it's right here")? Or do you trust gut feelings, unspoken risks, and intuition (things that are hard to explain)?
- Analogy: One person drives by looking strictly at the speedometer and GPS. The other drives by looking at the road, the wind, and a "feeling" that something is wrong.
Exploration (E): How many possibilities do you keep open?
- Do you quickly decide on one answer and stick to it? Or do you keep many "what if" scenarios running in your head at once?
- Analogy: A detective who immediately arrests the first suspect vs. a detective who keeps a list of ten suspects and investigates all of them.
Stabilization (S): How hard is it to change your mind?
- When new info arrives, do you instantly update your plan? Or do you stick to your original rule unless the new info is overwhelming?
- Analogy: A thermostat that changes the temperature the second the room feels a degree warmer vs. one that waits until the room is freezing before turning on the heat.
Horizon (D): How far into the future do you look?
- Do you care about what happens next week? Or next decade?
- Analogy: A farmer who plants crops for next month's market vs. one who plants trees that won't bear fruit for 20 years.

3. Why Do We Argue About the Same Three Things?

You might think there are infinite ways to disagree. But the paper argues that because our brains have limits (we can't process infinite data, we can't see everything, and we have to talk to each other), these four dials tend to collapse into just three main arguments:

Abstract vs. Concrete:
- The Conflict: One person wants to talk about big, general principles (Abstract). The other wants to talk about specific, messy details (Concrete).
- The Cause: Our brains have to compress information to fit it in. Sometimes we compress too much (losing details), and sometimes we hold onto too much detail (losing the big picture).
External vs. Internal:
- The Conflict: One person says, "Show me the data!" (External). The other says, "You just don't understand the risk I feel!" (Internal).
- The Cause: It's hard to share your internal feelings. It's easy to share a spreadsheet. People argue over whether the "feelings" count as valid evidence.
Order vs. Freedom:
- The Conflict: One person wants strict rules and consistency (Order). The other wants flexibility and new ideas (Freedom).
- The Cause: We have to balance stability (not changing our minds every second) with adaptability (changing our minds when we learn something new).

4. A Real-World Example: AI Regulation

The paper uses the debate over regulating Artificial Intelligence to show how this works.

The Shared Facts: Everyone sees the same reports on AI accidents, economic growth stats, and technical benchmarks.
The "Precautionary" Group:
- Reference: They focus on hard-to-externalize fears (e.g., "What if we lose control?").
- Exploration: They keep "worst-case scenarios" alive in their minds.
- Stabilization: They want strict, unchangeable rules.
- Horizon: They look 50 years into the future.
- Conclusion: "Ban it or regulate it heavily."
The "Promotion" Group:
- Reference: They focus on externalizable data (e.g., "Look at these economic numbers").
- Exploration: They focus on the most likely, positive scenarios.
- Stabilization: They want flexible rules that can change as tech evolves.
- Horizon: They look at the next 2–5 years.
- Conclusion: "Let it grow; we can fix problems later."

The paper says: Neither side is "crazy." They are just using different settings on their thinking machine.

5. The Solution: Stop Blaming, Start Tuning

The paper's main takeaway is that we should stop calling people "irrational" or "bad faith" when they disagree. Instead, we should treat disagreement like a technical problem.

If two people disagree, we shouldn't ask, "Who is stupid?" We should ask:

"Are you looking at different parts of the data?" (Reference)
"Are you holding onto different possibilities?" (Exploration)
"Are you looking at different timeframes?" (Horizon)

By identifying which "dial" is turned differently, we can design better ways to talk. We can agree to look at the same timeframe, or agree to share the same "gut feelings" as data. This turns a moral fight into a solvable engineering problem.

In short: Disagreement isn't a sign of a broken brain; it's a sign of different settings on the same machine. If we understand the settings, we can fix the disagreement.

Technical Summary: Formalizing World-Model Non-Identifiability via an Inference Profile $\theta$

1. Problem Statement

The paper addresses the phenomenon where distinct agents, sharing identical observations (documents, statistics, logs, or incidents), reach divergent conclusions. In traditional discourse, such divergence is often attributed to the cognitive defects, irrationality, or bad faith of the opposing party. This attribution relies on the Single Intelligence Assumption (SIA), which posits that intelligence is centralized in logical reasoning, that deviations from this norm are failures, and that rational agents should converge on the same conclusion given identical inputs (commutability).

The paper argues that this framing blocks productive inquiry. Instead, it proposes that conclusion divergence is a structural feature of non-identifiability in world-model estimation. Under conditions of finite data, partial observability, and representational constraints, multiple models or inference policies can remain compatible with the same observations. The paper seeks to reframe disagreement not as a moral or personality defect, but as a computational problem of non-identifiability occurring at two distinct levels:

$\theta$ -level: Divergence arising from differences in inference settings despite a shared world model ( $W$ ).
$W$ -level: Divergence arising because repeated inference operations bias data exposure and update rules, causing the learned world models themselves to diverge over time.

2. Methodology and Framework

2.1 The Inference Profile $\theta$

To operationalize the sources of divergence, the paper introduces the Inference Profile $\theta = (R, E, S, D)$ , a four-dimensional vector representing the operational degrees of freedom in the inference process:

Reference ( $R$ ): The weighting of grounds (evidence) used for inference. This is modeled as a weighted composition of partial grounds $\{e_i\}$ . The weight $w_i$ depends on an externalizability score $x_i$ (how easily a ground can be shared and audited) and a parameter $\beta_R$ . High $\beta_R$ prioritizes auditable grounds (logs, statistics); low $\beta_R$ allows high-description-cost grounds (tacit knowledge, intuition) to influence the conclusion.
Exploration ( $E$ ): The retention of alternative hypotheses. This is characterized by the entropy $H(h|o)$ of the hypothesis distribution. High exploration maintains multiple possibilities (high entropy), while low exploration concentrates on a single conclusion.
Stabilization ( $S$ ): The inhibition of updates. This is governed by a threshold $\tau$ or regularization strength $\lambda$ . High stabilization resists change (order), while low stabilization allows rapid adaptation to new information (freedom).
Horizon ( $D$ ): The temporal center of evaluation, controlled by a discount factor $\gamma$ . High $\gamma$ emphasizes long-term consequences; low $\gamma$ emphasizes immediate, local outcomes.

2.2 Two Levels of Non-Identifiability

$\theta$ -Level Non-Identifiability: Even if two agents share the same world model parameters $\phi$ (and thus the same $W_\phi$ ), their conclusions $y$ may differ if their inference profiles $\theta_A \neq \theta_B$ . Formally: $y = \text{Infer}(W_\phi, o_{\le t}; \theta)$ .
$W$ -Level Non-Identifiability: Inference operations are repeated over time. The choice of $\theta$ biases which data is observed and how the model is updated ( $\phi_{t+1} = U(\phi_t, o_t, \theta_t)$ ). Consequently, agents with different initial $\theta$ values may develop fundamentally different world models $W_A$ and $W_B$ , leading to divergent causal attributions and expectations even when presented with the same new input.

2.3 Projection onto Three Bases

The paper posits that the four operational dimensions of $\theta$ tend to project onto three recurrent axes of disagreement due to three fundamental constraints common to learning systems:

Computational Constraints ( $C_{comp}$ ): Finite capacity and resources.
Observational Constraints ( $C_{obs}$ ): Partial observability and noise.
Coordination Constraints ( $C_{coop}$ ): Requirements for accountability, reproducibility, and auditability.

These constraints induce three trade-offs:

Abstract vs. Concrete: Driven by $C_{comp}$ (Rate-Distortion theory). High abstraction compresses information; high concreteness preserves detail. Horizon ( $D$ ) projects here.
Externalizability vs. Internalization: Driven by $C_{obs}$ and $C_{coop}$ . Externalizable grounds are shareable; internalized states (e.g., anxiety, tacit risk) are costly to communicate. Reference ( $R$ ) projects here.
Order vs. Freedom: Driven by $C_{comp}$ and $C_{coop}$ (Plasticity-Stability dilemma). Order implies low entropy and reproducibility; freedom implies high entropy and retained alternatives. Exploration ( $E$ ) and Stabilization ( $S$ ) jointly project here.

2.4 Structural Correspondence in Deep Learning

The framework is grounded in deep representation learning concepts:

Reference corresponds to the selection of representation layers (e.g., lower layers for concrete features vs. higher layers for abstract concepts in Transformers).
Externalizability relates to latent-state estimation, where hidden states are non-identifiable without inductive biases or supervision, requiring externalization procedures (probing, logging) to communicate.
Order/Freedom corresponds to the trade-off between regularization (stability) and exploration (diversity) in learning and inference (e.g., temperature sampling).

3. Key Contributions

Formalization of Non-Identifiability: The paper distinguishes between $\theta$ -level (inference setting) and $W$ -level (model learning) non-identifiability, providing a unified framework for short-term misalignment and long-term epistemic fragmentation.
The Inference Profile $\theta$ : It introduces a compact, four-component representation ( $R, E, S, D$ ) to locate divergence in identifiable operational points rather than vague personality traits.
Projection Mechanism: It explains why diverse inference settings collapse into three recurrent bases of disagreement (Abstract/Concrete, Externalizability, Order/Freedom) via computational, observational, and coordination constraints.
Computational Grounding: It connects these bases to deep learning mechanisms (representation hierarchy, latent-state estimation, regularization), shifting the discourse on disagreement from rhetorical or psychological explanations to computational design problems.

4. Results and Illustration

The paper does not present empirical experimental results but offers a case study on AI regulation debates (specifically the EU AI Act formation) to illustrate the framework:

Shared Observations: Stakeholders share incident reports, benchmarks, and economic forecasts.
$\theta$ -Level Divergence:
- Precautionary actors prioritize hard-to-externalize concerns (low $\beta_R$ ), retain worst-case scenarios (high $H$ ), favor institutional fixation (high $\tau$ ), and emphasize long-term irreversibility (high $\gamma$ ).
- Promotion-oriented actors prioritize externalizable benefits (high $\beta_R$ ), focus on mainline scenarios (low $H$ ), allow flexible revision (low $\tau$ ), and emphasize medium-term opportunity costs (medium $\gamma$ ).
$W$ -Level Divergence: Actors learn different causal sequences from history (e.g., "innovation leads to improvement" vs. "lack of regulation leads to accidents"), causing them to interpret the same new evidence through different causal structures.
Resolution Strategy: The framework suggests that resolving disagreement requires designing discriminative observations or interventions (e.g., A/B tests, measurable indicators of trust) that maximize predictive differences between models, rather than attributing the conflict to moral failure.

5. Significance and Claims

The paper claims that disagreement often possesses an identifiable computational structure. By locating divergence in inference operations and world-model learning, the problem can be shifted from moralized evaluation (accusing the other of defect) to designable coordination.

The significance lies in:

Methodological Shift: Adopting the Multiple Inference Assumption (MIA), which treats diversity in inference as a consequence of non-identifiability rather than a defect to be eliminated.
Operational Clarity: Providing a vocabulary ( $R, E, S, D$ ) to diagnose where the inference process differs.
Practical Application: Offering a path to resolve conflicts by aligning operational settings (for $\theta$ -level issues) or designing specific interventions to test competing world models (for $W$ -level issues).

The paper remains modest, noting that the projection of four dimensions onto three bases is a "structural tendency" supported by theory rather than a strict theorem, and that the framework is a computational account rather than a normative celebration of diversity. Future work is identified as extracting profiles from empirical data and quantitatively validating the three-basis reduction.

Why Conclusions Diverge from the Same Observations: Formalizing World-Model Non-Identifiability via an Inference