📈 economics

The Wisdom of the Crowd and Higher-Order Beliefs

The paper introduces Population-Mean-Based Aggregation (PMBA), a novel procedure that enables a principal to accurately infer the true state of the world from agents' beliefs and their expectations of the population average without needing to understand the underlying information structure, a method that is theoretically robust under weak assumptions and empirically outperforms existing approaches.

Original authors: Yi-Chun Chen, Manuel Mueller-Frank, Mallesh M Pai

Published 2026-04-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yi-Chun Chen, Manuel Mueller-Frank, Mallesh M Pai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Why the Crowd Sometimes Gets It Wrong

Imagine you are trying to guess the weight of a giant pumpkin at a county fair. You ask 100 people to guess.

The Old Way (Standard Wisdom of the Crowd): You take all 100 guesses, add them up, and divide by 100. The theory says this average will be very close to the real weight.
The Problem: What if everyone is looking at the same bad photo of the pumpkin? Or what if everyone is using the same wrong rule of thumb? In that case, the "average" of their guesses will be confidently wrong. The crowd is wise only if their individual errors cancel each other out. If they all share the same blind spot, the crowd is just a big group of people making the same mistake.

The authors of this paper ask: Can we fix the crowd's wisdom even if we don't know why they are making mistakes or how they got their information?

The Solution: Asking "What Do You Think Others Think?"

The authors propose a new method called PMBA (Population-Mean-Based Aggregation). It's like upgrading the question you ask the crowd.

Instead of just asking, "How heavy is the pumpkin?" (First-Order Belief), they also ask a specific subset of people: "What do you think the average person in the crowd will guess?" (Second-Order Belief).

The Magic Analogy: The Detective and the Map

Imagine a detective trying to find a hidden treasure (the "True State").

The Clues (First-Order Beliefs): The detective asks 100 locals, "Where is the treasure?" Everyone points in different directions. The detective takes the average direction. But the detective doesn't know the map, so they don't know if the average points to the treasure or a swamp.
The Twist (Second-Order Beliefs): The detective then asks a few of those locals, "If you were to stand in the middle of the crowd and look at where everyone is pointing, where would that average point?"
The Breakthrough: By comparing what people think the average is, versus what the actual average is, the detective can mathematically reverse-engineer the map. They can figure out the true location of the treasure without ever seeing the map themselves.

How It Works (The "Linear Regression" Trick)

The paper explains that this process is actually a fancy version of Linear Regression (a common math tool used to find patterns).

The Inputs:
- X (The Belief): "I think there is a 70% chance the pumpkin is heavy."
- Y (The Expectation): "I think the average person will say there is a 60% chance it's heavy."
The Connection: The math shows that your expectation of the crowd (Y) is just a weighted combination of what the crowd actually thinks in different scenarios (Heavy vs. Light).
The Result: By running this math on the data, the system can "solve" for the hidden scenarios. It figures out: "Ah, when the pumpkin is actually heavy, the crowd averages 80%. When it's light, they average 40%." Once the system knows these "true averages," it can look at the current crowd's average and instantly know which scenario is real.

Why This Is Better Than Old Methods

The paper compares their method to two other popular ways of aggregating information:

Majority Voting: Just asking "Is it heavy or light?" and taking the most popular answer. (Fails if the crowd is confused).
"Surprisingly Popular" (SP): Asking people what they think the majority will say, and picking the answer that is more popular than people expected. (This works well for simple Yes/No questions but struggles when there are many options, like 4 different price ranges).

The PMBA Advantage:

Handles Complexity: It works even if there are many possible answers (not just Yes/No).
Robustness: It works even if people are confused about how the world works. The paper shows that even if individuals have "misspecified" beliefs (they are wrong about how others think), the math still averages out the errors and finds the truth, as long as the errors aren't massive.
Real-World Proof: The authors ran an experiment where people guessed the price ranges of NFTs (digital art).
- The Result: PMBA was more accurate than simple majority voting and more accurate than the "Surprisingly Popular" method. It successfully predicted the true price ranges better than the other methods.

The "Misspecification" Safety Net

A key part of the paper is that it doesn't require people to be perfect geniuses.

The Fear: What if people don't understand the game? What if they think everyone else is smarter or dumber than they really are?
The Fix: The math is designed to be "robust." It assumes that while individual people might be confused about the crowd's mindset, the average confusion cancels out. As long as the "noise" in their thinking isn't louder than the actual signal, the method still finds the true state.

Summary

The paper proposes a simple but powerful trick: Don't just ask people what they think; ask them what they think others will think.

By combining these two layers of information, a principal (like a researcher or a market maker) can uncover the true state of the world—even if they don't know the rules of the game, even if the crowd is confused, and even if there are many possible outcomes. It turns the "Wisdom of the Crowd" from a fragile concept into a robust, mathematically guaranteed tool.

1. Problem Statement

The paper addresses the fundamental challenge of information aggregation: how a principal (an aggregator) can infer the true state of the world ( $\omega$ ) from a population of agents, each holding private, noisy signals, without knowing the underlying information structure (i.e., the distribution of signals, the correlation structure, or the common prior).

The Limitation of First-Order Beliefs: Existing literature (e.g., Arieli, Babichenko, & Smorodinsky, 2017) establishes that knowing only the first-order beliefs (agents' posterior probabilities about the state) is generally insufficient to identify the true state, even with an infinite population. This is an identification problem: multiple states can generate the same distribution of first-order beliefs.
The Gap: Most existing methods (like Prediction Markets or Surprisingly Popular) either require strong assumptions (e.g., binary states, independent signals) or ask for complex, often infeasible data (e.g., state-contingent beliefs or conditional probabilities of signals).
The Goal: Develop a robust procedure that aggregates information to identify the true state using only:
1. Agents' first-order beliefs ( $\mu_i$ ).
2. Agents' expectations of the population mean belief (higher-order beliefs).
3. Minimal knowledge of the information structure (no need to know the prior or signal distribution).

2. Methodology: Population-Mean-Based Aggregation (PMBA)

The authors propose a novel procedure called Population-Mean-Based Aggregation (PMBA). The core intuition relies on the Law of Iterated Expectations.

A. The Core Mechanism

Let $\Omega = \{\omega_1, \dots, \omega_L\}$ be the set of $L$ possible states.

First-Order Elicitation: The principal collects the belief vector $\mu_i$ from every agent $i$ . The empirical mean $\hat{\mu} = \frac{1}{n}\sum \mu_i$ converges to the true state-dependent population mean $\bar{\mu}(\omega)$ .
The Identification Problem: The principal observes $\hat{\mu}$ but does not know the mapping from states $\omega$ to the expected population means $\bar{\mu}(\omega)$ .
Higher-Order Elicitation: The principal selects $L$ $L$ agents (or $L$ $L$ groups of agents) and asks them for their expectation of the population average belief.
- Let $\alpha_i = E[\bar{\mu} | s_i]$ be agent $i$ 's expectation of the population average.
- By the Law of Iterated Expectations, $\alpha_i$ is a weighted average of the state-dependent population means, where the weights are the agent's own first-order beliefs:
  $\alpha_i = \sum_{\omega \in \Omega} \mu_i(\omega) \cdot \bar{\mu}(\omega)$
System of Equations: If the principal collects these reports from $L$ agents with linearly independent belief vectors, they form a system of linear equations:
$\mathbf{\alpha} = \mathbf{M} \cdot \mathbf{\bar{\mu}}$
Where $\mathbf{M}$ is the matrix of first-order beliefs and $\mathbf{\bar{\mu}}$ is the vector of unknown state-dependent population means.
Inversion: Since $\mathbf{M}$ is invertible (under specific rank conditions), the principal can solve for $\mathbf{\bar{\mu}} = \mathbf{M}^{-1}\mathbf{\alpha}$ .
State Identification: The principal compares the observed empirical mean $\hat{\mu}$ to the recovered state-dependent means $\bar{\mu}(\omega)$ . The true state is the $\omega$ for which $\bar{\mu}(\omega)$ is closest to $\hat{\mu}$ .

B. Robustness and Misspecification

The paper extends the model to handle misspecification, where agents may have incorrect beliefs about the information structure (e.g., they misunderstand how others form beliefs).

Approximate PMBA: The procedure is reinterpreted as a linear regression problem.
- Dependent Variable: Agent's second-order belief ( $\alpha_i$ ).
- Independent Variables: Agent's first-order beliefs ( $\mu_i$ ).
- Coefficients: The unknown state-dependent population means.
Misspecification Handling: Even if agents have biased views of the population average (additive noise $\zeta$ ), the procedure remains valid provided the bias is small relative to the separation between true state means. The Law of Large Numbers ensures that averaging over large groups cancels out individual misspecification errors.

3. Key Contributions

General Identification without Structure: PMBA is the first procedure to guarantee state identification for finite $L > 2$ states without requiring knowledge of the signal structure, correlation, or common prior. It overcomes the identification failure of first-order beliefs alone.
Robustness to Misspecification: The authors prove that the procedure works even when agents are "wrong" about the aggregate information structure, as long as the aggregate error is not too large. This makes the method applicable to real-world settings where agents have limited cognitive models of the crowd.
Linear Regression Interpretation: By framing PMBA as a linear regression, the authors unlock a century of statistical tools (confidence intervals, hypothesis testing, finite-sample error bounds). This transforms a theoretical matrix inversion into a practical, actionable statistical tool.
Finite-Sample Guarantees: The paper derives non-asymptotic error bounds, showing that the estimation error decreases at the standard $O(1/\sqrt{n})$ rate, dependent on the diversity of beliefs and the noise level.
Experimental Validation: The authors conduct a novel experiment involving NFT price prediction (a complex, uncertain market).
- Result: PMBA significantly outperforms Majority Voting (based on first-order beliefs) and the Surprisingly Popular (SP) method.
- Significance: SP struggles with more than two states (binary limitation), whereas PMBA successfully handles multi-state (4-range) predictions with higher accuracy.

4. Results

Theoretical:
- Theorem 1: Under weak regularity conditions (limited correlation, full rank of beliefs), PMBA recovers the true state in probability as $n \to \infty$ .
- Theorem 3: PMBA recovers the true state almost surely even under misspecified beliefs, provided the bias in agents' expectations is smaller than the distance between the true state-dependent population means.
- Proposition 1: Provides finite-sample guarantees, establishing the sample size $n$ required to identify the state with probability $1-\delta$ .
Empirical (NFT Experiment):
- Setup: 199 participants predicted NFT price ranges (4 states) and reported their expectations of the group's average prediction.
- Performance:
  - PMBA Mean Absolute Error (MAE): 1.25
  - Majority Voting (First-Order) MAE: 1.50
  - Majority Voting (Second-Order) MAE: 1.40
  - Surprisingly Popular (SP): Struggled to distinguish states effectively in the 4-state setting; often reduced to binary distinctions.
- Statistical Significance: PMBA's superiority over first-order majority voting is statistically significant ( $p < 0.05$ ).

5. Significance and Implications

Practical Applicability: Unlike theoretical models requiring complex state-contingent reports (which humans find difficult to compute), PMBA only asks for beliefs and "what do you think the average person thinks?"—questions that are easily elicited in surveys.
Beyond Binary States: Many existing "wisdom of the crowd" mechanisms (like SP) are limited to binary choices. PMBA generalizes this to any finite number of states, making it applicable to complex economic, political, and scientific forecasting.
Robustness to Human Error: The procedure's reliance on higher-order beliefs allows it to correct for individual cognitive biases and misspecifications, effectively "averaging out" the noise in human reasoning.
Incentive Compatibility: The paper discusses how to incentivize truthful reporting using Bayesian Truth Serum mechanisms, linking the aggregation procedure to payment rules based on the recovered state and population averages.

In summary, Chen, Mueller-Frank, and Pai demonstrate that by leveraging higher-order beliefs (expectations of the crowd's average), a principal can fully aggregate dispersed information and identify the true state of the world, even without knowing the underlying information structure and even when agents are imperfectly informed about the crowd's behavior.