Technical Summary: Building Social World Models with Large Language Models

1. Problem Statement

The paper addresses the fundamental challenge of modeling and predicting how social beliefs evolve in response to real-world events (e.g., policy changes, scientific breakthroughs, or political elections). While Large Language Models (LLMs) possess commonsense knowledge and social intelligence, applying them to this domain faces three specific hurdles:

• Quantifiability and Data Scarcity: Social beliefs are semantic and difficult to capture as structured, high-fidelity time-series data. Traditional sources like surveys or social media suffer from noise, bias, and lack of representativeness.
• Semantic Complexity: Social transitions are driven by nuanced psychology and cultural context rather than explicit, codified laws. Traditional statistical or symbolic models often fail to capture these implicit transition rules.
• Lack of Explicit Attribution: Even when belief shifts are observed, the causal link between a specific event and the shift is often obscured. Without explicit "event-to-shift" labels, models struggle to learn the underlying mechanisms of social dynamics.

The core research question is: Can LLMs model the dynamics of social beliefs in response to events?

2. Methodology: The Social World Model (SWM)

The authors propose the Social World Model (SWM), a generative framework that treats collective belief as a semantic state ($s_t$) and social events as exogenous actions ($e_t$). The model learns a state-transition function $P_\theta(s_{t+1} | s_t, e_t)$.

2.1 Data Foundation: SWM-Bench

To solve the data scarcity issue, the authors introduce SWM-Bench, a benchmark derived from real-world prediction markets (Kalshi and Polymarket).

• Rationale: Prediction market prices serve as a high-fidelity, aggregated signal of public opinion, filtering out noise and reflecting rational decision-making based on financial stakes.
• Format: A social belief is defined as a binary proposition $q$ with a market-implied probability $v_t \in [0, 1]$. The state $s_t$ is a trajectory window of recent prices, and the event space $E_t$ consists of candidate news articles.

2.2 Architecture and Training Paradigm

The SWM framework utilizes LLMs as the backbone for three core modules, employing a posterior-guided training paradigm to solve the attribution labeling problem without human annotations.

1. Latent Event Attribution: The true event driving a belief shift is treated as a latent variable $Z_t$. The model approximates transitions by conditioning on a single hypothesized event.
2. Prior Attributor ($P_\eta$): A forward-looking model that scores candidate events based on context available before the shift occurs.
3. Posterior Attributor ($Q_\phi$): A frozen, hindsight LLM that observes the realized outcome ($s_{t+1}$) and determines which candidate event best explains the shift. This acts as a "teacher" providing pseudo-labels.
4. Social World Model ($P_\theta$): The transition engine that learns to predict $s_{t+1}$ given $s_t$ and a specific attributed event.

Training Objective:
The training minimizes a loss function composed of two terms:

• World Model Loss ($L_{wm}$): Trains $P_\theta$ to predict the transition $s_{t+1}$ conditioned on the event attributed by the posterior $Q_\phi$.
• Attributor Loss ($L_{attr}$): Distills the hindsight attribution from $Q_\phi$ into the prior attributor $P_\eta$ using KL divergence.

This approach allows the model to learn causal mechanisms by performing inverse inference: the posterior identifies the cause, and the prior learns to predict that cause without access to the future.

2.3 Inference Modes

The decoupled architecture supports two distinct modes:

• Forecasting: Marginalizes over the prior attributor's uncertainty to predict future market trajectories ($\hat{s}_{t+1} = \sum P_\eta \cdot E[P_\theta]$).
• Simulation: Conditions directly on a specific (real or hypothetical) event to estimate the "what-if" belief shift, bypassing the attributor.

3. Key Contributions

1. SWM-Bench: The first evaluation benchmark curated from real-world prediction markets, transforming social beliefs into a quantifiable, time-series format covering politics, finance, and cryptocurrency.
2. Social World Model Framework: A novel architecture leveraging LLMs to capture semantically driven, implicit transition rules of social dynamics, moving beyond traditional time-series forecasting.
3. Posterior-Supervised Training: A training paradigm that solves the attribution labeling problem by decoupling social reasoning (attribution) from dynamics modeling, enabling learning without explicit human annotations.

4. Experimental Results

Experiments were conducted on SWM-Bench (Kalshi and Polymarket datasets) comparing SWM against time-series models (e.g., Autoformer, iTransformer) and LLM-based baselines (prompting and fine-tuned).

• Performance: SWM achieves state-of-the-art results on the Kalshi dataset, outperforming strong baselines including GPT-5.5 and various time-series models. On the Polymarket dataset, it demonstrates competitive performance, particularly in directional accuracy.
• Directional Accuracy: On the attributed subset (where a clear news driver exists), SWM (prior) achieves the highest directional accuracy (0.685 on Polymarket, 0.845 on Kalshi).
• Ablation Studies:
  • Model Size: Larger LLM backbones (0.6B to 8B) consistently improve performance, suggesting capacity is crucial for translating textual events into numerical dynamics.
  • Attribution: The primary bottleneck is the prior attributor's ability to isolate the correct causal shock. When an oracle posterior is used for inference, performance improves significantly, indicating the transition model ($P_\theta$) is highly accurate once the event is identified.
  • Event Sparsity: Causal signals are often concentrated in a single event; posterior-guided selection with a single event ($m=1$) matches full-set performance.

5. Significance and Claims

The paper claims that SWM provides a new pipeline for modeling and predicting the evolution of collective consensus. By grounding social dynamics in prediction markets and leveraging the reasoning capabilities of LLMs, the framework offers a scalable toolkit for quantitative digital sociology.

The authors position this work as a step toward parametric social world models. They emphasize that while the model excels at capturing belief shifts driven by clear external events (as seen in Kalshi), it faces challenges in markets with high endogenous or algorithmic noise (as seen in Polymarket). The work does not claim to solve all social forecasting problems but rather establishes a foundational framework for understanding how specific events drive shifts in public opinion, offering interpretable insights into the mechanisms of social belief dynamics.

Ethical Note: The authors acknowledge that such models carry risks if used maliciously (e.g., for targeted disinformation) and advocate for transparent, defensive deployment to detect unnatural shifts in collective belief rather than to manipulate them.