Technical Summary: Characterizing Opinion Evolution of Networked LLMs

Problem Statement

As Large Language Models (LLMs) are increasingly deployed in multi-agent systems—ranging from human discourse simulations to influence operations and fully LLM-driven social platforms—there is a critical need to understand how opinions form, evolve, and propagate within these networks. While classical opinion dynamics models (e.g., DeGroot, Friedkin-Johnsen, Hegselmann-Krause) have long been used to explain collective belief evolution in human societies, it remains unclear whether these models can accurately capture the behavior of interacting LLMs. Specifically, the authors investigate which mechanisms (stubbornness, homophily, bias) are intrinsic to LLMs and which are required to model their belief evolution. The study addresses three core research questions:

1. RQ1: Which components of classical opinion dynamics models are most critical for explaining LLM belief evolution?
2. RQ2: To what extent do the derived opinion dynamics parameters differ across different LLM families and discussion topics?
3. RQ3: Are the recovered opinion dynamics a property of the LLM itself or of the specific network topology on which it communicates?

Methodology

The authors propose a framework to simulate, measure, and model the opinion dynamics of networked LLMs.

Simulation and Data Generation

• Agents and Network: The study simulates $n=30$ LLM agents communicating over directed graphs generated via three topological models: Erdős–Rényi (random), Chung-Lu (scale-free), and Stochastic Block Models (community structure).
• Interaction Protocol: Agents post sequentially in response to a random sample of $k$ posts from their neighbors. Posts are generated by prompting agents to justify their positions based on their initial opinion and the sampled context.
• Opinion Measurement: Natural language posts are converted into numerical opinions $z(t) \in [-1, 1]$ using a stance identification model. This involves embedding the post and reference "for/against" texts, then calculating the cosine similarity difference. An agent's opinion at a given time is the mean of their five most recent posts.
• Initialization: Agents are initialized with discrete stances ranging from complete disagreement to agreement, mapped to natural language descriptors. Initial conditions vary (uniform, polarized, consensus, skewed).

Modeling Framework

The authors fit parametric models of the form $z(t + 1) = f_\theta(z(t), z(0), A)$ to the simulated trajectories. They evaluate several hypotheses corresponding to classical models:

1. DeGroot: Simple averaging of neighbors' opinions.
2. Friedkin-Johnsen (FJ): Adds a "stubbornness" term, anchoring agents to their initial opinions ($z(0)$).
3. Uniform Bias: Adds a term for an innate bias $b$ toward which agents regress, independent of initial opinions.
4. Homophily: Replaces static weighting with distance-based weighting, where agents weigh neighbors with similar opinions more heavily.
5. Adjacency-Based Weighting: Instead of fitting a full $n \times n$ influence matrix $W$ for each graph, the authors test "graph-agnostic" models where influence is a function of the adjacency matrix $\bar{A}$ and scalar coefficients ($\mu_{self}, \mu_{neigh}$).

Models are fitted by minimizing a least-squares objective. Fully parameterized models use convex quadratic programming (QP) or non-linear solvers (SLSQP), while adjacency-based models reduce the parameter space to scalar coefficients.

Key Results

1. The Dominance of Innate Bias (RQ1)

The study finds that naive averaging models (DeGroot) and models incorporating only stubbornness (FJ) fail to track LLM opinion dynamics, exhibiting high prediction errors.

• Bias is Critical: The inclusion of a uniform bias term ($b$) yields substantial gains in modeling fidelity. Models with bias reduce the cumulative estimated mean opinion error by up to 88% and the Wasserstein-1 distance by up to 67% (specifically on LLAMA3.1).
• Insignificance of Other Factors:
  • Stubbornness: Anchoring agents to their initial opinions (the FJ component) provides negligible improvement over the basic DeGroot model.
  • Homophily: Adding homophily weighting changes error rates by at most a few percent and produces no measurable improvement once bias is included.
• Conclusion: The primary driver of LLM opinion dynamics is not stubbornness or social clustering, but a regression toward a single, innate opinion shared across all agents.

2. Parameter Variability Across Models and Topics (RQ2)

While the mechanism (bias) is consistent, the magnitude of parameters varies:

• Self-Weight: The self-weight ($\lambda_{self}$) is consistently the largest parameter (0.53–0.70), though this is largely an artifact of the chosen time scale.
• Initial Opinion Weight: The weight for initial opinions ($\lambda_0$) is consistently near zero (<0.04), confirming that LLMs do not strongly anchor to their starting stance.
• Bias Weight: The bias weight ($\lambda_b$) is the second-largest term (0.17–0.36), confirming regression to an innate opinion is a universal feature.
• Model Differences:
  • LLAMA3.1: Shows the strongest bias effect and the least social interaction (neighbor influence ratio $\approx 0$).
  • QWEN3: Shows the most "social" behavior, with neighbor influence matching or exceeding bias.
  • GEMMA3: Shows high variance depending on the topic.
• Topic Differences: The "vaccines" topic consistently yields the highest bias effect across all models, likely due to alignment training regarding health misinformation.

3. Generalizability of Behavioral Profiles (RQ3)

The authors compare fully parameterized models (fitting a unique influence matrix $W$ for each graph) against adjacency-based models (using scalar coefficients).

• Overfitting: Fully parameterized models lower training error but increase held-out test error, indicating overfitting to specific graph structures.
• Portability: Adjacency-based models recover a behavioral profile that generalizes across network topologies. This suggests that the recovered parameters (specifically the bias and social averaging weights) are intrinsic properties of the LLM, not artifacts of the specific network.

Significance and Claims

The paper claims to provide a compact, identifiable model for LLM opinion formation. By demonstrating that classical opinion dynamics models can accurately describe LLM networks if they include an innate bias term, the authors lay the groundwork for:

• Design and Control: Guiding the design of LLM-driven social platforms to mitigate echo chambers.
• Validation: Certifying when LLM populations act as faithful surrogates for human ones in simulations.
• Leverage Points: Identifying how stubborn LLM agents might disproportionately shape collective beliefs in influence operations.

The authors explicitly state that their findings characterize the behavior of instruction-tuned LLMs in simulation. They caution that these dynamics should not be assumed to represent human opinion formation without independent validation, and they emphasize that their goal is to make these dynamics legible and governable rather than to optimize persuasion.