A Bayesian Framework for Human-AI Collaboration: Complementarity and Correlation Neglect

1. Problem Statement

The paper addresses a critical gap in the literature regarding Human-AI collaboration: while AI systems are increasingly capable, their integration into human decision-making workflows does not always improve outcomes. In some cases, AI assistance degrades performance compared to human-only or AI-only decision-making.

The central problem is to determine when and why AI assistance leads to:

• Augmentation: AI improves human performance.
• Impairment: AI worsens human performance.
• Complementarity: The combined system outperforms both human-only and AI-only baselines.
• Automation: The AI alone outperforms the human (with or without AI assistance).

The authors argue that existing literature often focuses on AI capability alone, neglecting two crucial factors: the informational overlap between human and AI knowledge, and human behavioral biases (specifically, how humans imperfectly combine signals).

2. Methodology

The authors develop a decision-theoretic Bayesian framework to model the interaction between a human decision-maker and an AI system.

A. Core Model Setup

• State: An uncertain state $Y$ (e.g., disease presence).
• Signals:
  • $H$: Human private information (expertise, observations).
  • $A$: AI signal (algorithmic recommendation).
• Loss Function: A Bregman loss function (e.g., quadratic loss for regression, log loss for classification) measuring the error of the decision $d$ relative to the true state $Y$.
• Decision Rule: The human observes $H$ and $A$ and chooses a decision $\delta(H, A)$. Crucially, the human is not assumed to be a Bayesian-optimal integrator.

B. Decomposition of AI Value

The paper introduces a fundamental decomposition of the realized benefit of AI assistance:
$$\text{Realized Gain} = \underbrace{v(A|H)}_{\text{Marginal Informational Value}} - \underbrace{\text{Behavioral Penalty}}_{\text{Misspecification Cost}}$$

• Marginal Value ($v(A|H)$): The incremental decision value the AI provides beyond what the human already knows.
• Behavioral Penalty: The loss incurred because the human uses a suboptimal decision rule (e.g., misinterpreting the correlation between signals).

C. Micro-founded Model of Information Overlap

To quantify overlap, the authors construct a model where both $H$ and $A$ are aggregates of a common pool of primitive information cues ($S$).

• Overlap Coefficient ($\lambda$): Defined as $\lambda = \frac{|A \cap H|}{|A|}$. This represents the fraction of the AI's information that is redundant given the human's expertise.
• Key Insight: $\lambda$ is treated as a domain-specific parameter that remains relatively stable as AI capability ($\tau_A$) grows, provided the AI draws from the same accessible data pool. This allows the authors to vary AI capability and overlap independently.

D. Behavioral Misspecification: Correlation Neglect

The paper focuses on Correlation Neglect as the primary behavioral bias.

• Definition: The human treats the AI signal $A$ and their own signal $H$ as conditionally independent given $Y$, even though they share common evidence sources.
• Consequence: The human "double-counts" the shared information, overweighting the redundant evidence and underweighting the unique innovation.

3. Key Contributions

1. General Decomposition Theorem: The authors prove that the impact of AI assistance is the difference between the marginal informational value and a behavioral penalty. AI impairs performance precisely when the behavioral penalty (due to misuse) exceeds the marginal informational gain.
2. Micro-founded Overlap Measure: They introduce $\lambda$ as a rigorous, interpretable metric for information redundancy. They demonstrate that $\lambda$ is largely invariant to AI capability growth, allowing for a clear separation of "AI getting smarter" vs. "AI knowing what the human already knows."
3. Phase Transition Analysis: By combining the overlap model with correlation neglect, the authors characterize three distinct phase-transition patterns based on the degree of overlap ($\lambda$):
   • Low Overlap ($\lambda < 0.5$): AI always helps (Augmentation). As AI gets stronger, the system transitions smoothly from Complementarity to Automation.
   • Intermediate Overlap ($0.5 < \lambda < \bar{\lambda}$): A "danger zone" exists. Weak AI impairs human performance because the behavioral penalty (double-counting) outweighs the small marginal value. Only when AI capability surpasses a specific threshold ($\tau_{aug}$) does Complementarity emerge. Eventually, as AI becomes very strong, Automation becomes optimal.
   • High Overlap ($\lambda > \bar{\lambda}$): The behavioral penalty is so severe that AI assistance is never optimal. The system transitions directly from Human-Only (Impairment) to AI-Only (Automation). Complementarity is impossible.
4. Reversal of the Value of Information: The paper shows that for a Bayesian-rational agent, more information is always better. However, under correlation neglect, revealing an informative AI signal can strictly worsen human decision-making if the overlap is high and the AI is weak.

4. Key Results

• The "Impairment" Region: In domains with intermediate to high overlap, introducing a weak AI system is detrimental. The human's tendency to double-count shared evidence leads to a net loss in decision quality compared to acting alone.
• Automation is Inevitable (under stagnation): If human behavior (specifically correlation neglect) does not improve, increasing AI capability will eventually make automation (AI acting alone) the optimal strategy, as the human's behavioral penalty prevents them from effectively utilizing even highly capable AI.
• Thresholds: The authors derive closed-form expressions for the thresholds separating these regimes:
  • $\tau_{aug}(\lambda)$: The AI capability level required to overcome the impairment region.
  • $\tau_{auto}(\lambda)$: The AI capability level where automation becomes optimal.
• Design Implications:
  • AI Training: AI should be trained to predict what humans miss (residual learning) rather than what they already know, to minimize effective $\lambda$.
  • Human Interface: Reducing correlation neglect (via training or UI design that highlights unique vs. shared evidence) can expand the range of AI capabilities where complementarity is possible, pushing back the automation frontier.

5. Significance

This paper provides a rigorous theoretical foundation for understanding the non-monotonic relationship between AI capability and human-AI team performance.

• Paradigm Shift: It challenges the assumption that "better AI" always leads to "better human-AI teams." It demonstrates that without addressing informational overlap and human cognitive biases, powerful AI can actually degrade human performance or make human involvement obsolete.
• Policy and Deployment: The findings suggest that deploying AI in high-overlap domains (e.g., medical diagnosis where AI uses the same test data as doctors) requires careful management of the "impairment" phase. Simply adding AI to a workflow without addressing how humans process the signal can be counterproductive.
• Future Research: The framework offers a new lens for empirical studies, suggesting that measuring the "overlap coefficient" and "correlation neglect" is essential for predicting the success of AI integration in specific domains.

In summary, the paper argues that the future of Human-AI collaboration depends not just on the raw power of the AI, but on the structural alignment between human and AI knowledge and the cognitive mechanisms humans use to integrate them.