Technical Summary: Digital Pantheon – Simulating and Auditing Coalition Formation with LLM Agents

1. Problem Statement

The formation of political coalitions involves complex negotiations driven by concrete policy objectives and deep-seated ideological convictions. While Large Language Models (LLMs) offer new avenues for computational political science, standard models trained with Reinforcement Learning from Human Feedback (RLHF) are optimized for neutrality and helpfulness. These inherent biases prevent LLMs from sustaining steadfast partisan behavior, which is essential for realistic political simulation. Existing approaches face four critical gaps:

1. Unsteered Neutrality: Debate agents drift toward consensus or dominant-party positions, and prompt-based personas degrade reasoning and lack factual grounding.
2. Generic Ideology: When ideology is steered, it often follows a coarse left-right axis rather than specific party manifestos; when coalitions are simulated, policy content is often abstract rather than grounded in official documents.
3. Black-Box Deliberation: Political simulators often predict terminal outcomes without attributing specific provisions to their ideological origins or the negotiation process.
4. Opaque Adjudication: Evaluation relies on opaque LLM-judge committees rather than auditable accounts of how agreements were reached.

2. Methodology

The authors propose a multi-agent framework that reconciles factual grounding with ideological alignment, operationalized on the 2019 Flemish election (simulating the N-VA, CD&V, and Open Vld coalition). The pipeline consists of three core stages:

A. Data Processing and Chunking

The corpus comprises official 2019 electoral manifestos. The authors employ an Agentic–Structural Hybrid (ASH) procedure to partition documents:

• Hard-Gate: Deterministic regex rules detect formal section boundaries (e.g., numeric subsections, headers) to ensure structural integrity.
• Soft-Gate: A probabilistic check using Gemma3 (27B) analyzes trailing context to determine topic transitions, ensuring semantic coherence.
• Preference-Pair Generation: Each chunk is processed to generate (Prompt, Chosen, Rejected) triplets. The Chosen response rewrites manifesto resolutions in a first-person "Method Actor" persona; the Rejected response offers a counterfactual from an opposing ideology or a neutral stance.

B. Two-Stage Ideological Alignment

To overcome RLHF-induced neutrality, the framework uses a two-stage alignment process on Gemma-3 (27B):

1. Supervised Fine-Tuning (SFT): Adapts the model to the conversational structure and first-person style of the party ("we will..."), establishing the "Method Actor" persona.
2. Direct Preference Optimization (DPO): Maximizes the implicit reward gap between Chosen and Rejected responses. This penalizes encyclopedic neutrality and rewards combative partisan rhetoric, ensuring the model does not revert to a neutral prior under adversarial prompting.

C. Retrieval-Augmented Generation (RAG) and Negotiation Arena

• RAG: Each agent is equipped with an isolated vector database (ChromaDB) containing its party's manifesto chunks. During negotiation, agents retrieve top-k chunks to enforce strict ideological boundaries, preventing hallucinations or concessions that contradict core promises.
• Hub-and-Spoke Negotiation: A central formateur (instantiated from the largest party's tuned model but system-prompted as a neutral broker) arbitrates a structured, multi-turn debate. Agents present standpoints, identify red lines, and propose syntheses over four rounds.
• Power Dynamics: The formateur utilizes Banzhaf power indices to reflect realistic leverage among parties.

3. Key Contributions and Evaluation Framework

The paper introduces a novel evaluation suite to make the generative process transparent and auditable:

A. Multi-Layered Information Lineage Topology (MILT)

MILT traces every clause in the final agreement (L3) backward through the agents' initial standpoints (L1) to the retrieved manifesto chunks (L0). It classifies provenance into five states:

1. Direct Lineage: Ideological preservation; the clause survives from L0 to L3 intact.
2. Diluted Lineage: Concession; the clause is weakened during negotiation.
3. Synthesized Lineage: Logrolling; a novel hybrid generated from competing L0 chunks.
4. Pipeline Artifact: Initialization bias; present in L1/L3 but absent from L0 (prompt-induced noise).
5. Orphan: Debate hallucination; appears in L3 with no L0/L1 grounding.

B. Coalition Influence Score (CIS)

A metric aggregating traceable contributions to identify the dominant party. It assigns weights based on MILT states:

• Direct: +3 points
• Diluted: +2 points
• Synthesized: +1 point
• Artifacts/Orphans: 0 points (to prevent hallucinations from inflating scores).

C. Real-World Grounding Pass

The simulated provisions are benchmarked against the historically adopted 2019 coalition agreement. Each provision is graded as Present (exact/functional equivalent), Partially Present, or Absent. This validates whether manifesto-anchored lineage predicts real-world materialization.

4. Results

The framework was executed across three independent simulations (N = 2,629 classified points) covering 22 policy dossiers:

• Stability and Ranking: The framework yields a stable winner and ranking across all runs: N-VA > CD&V > Open Vld. N-VA commands approximately 40.0% of attributable influence, followed by CD&V (31.8%) and Open Vld (28.2%).
• Provenance and Grounding:
  • Provisions with genuine manifesto provenance (Direct, Diluted, Synthesized) account for 71.1% of classifications and are significantly more likely to match the historical agreement than ungrounded inventions.
  • Systemic Errors: "Orphan" (hallucinated) and "Pipeline Artifact" clauses are overwhelmingly absent from the real-world agreement (71.6% Absent rate for Systemic Errors vs. ~48% for Ideological Retention).
  • Realization Rates: 45.8% of provisions reached a real-world counterpart (Present or Partial). Realization is highest in programmatic dossiers (e.g., Education, G = 0.491) and lowest in fiscal/competence-split dossiers (e.g., Finance, G = 0.147).
• Failure Modes: The dominant failure mode is not thematic reasoning but the fabrication of operational details (specific budgets, percentages, deadlines) which appear as "Orphans" or "Pipeline Artifacts."

5. Significance and Claims

The paper claims to provide a transparent, scalable testbed for the ex-ante exploration of party compatibility and formateur-mediated compromise. Its primary significance lies in:

1. Behavioral Steering: Demonstrating that SFT + DPO + RAG can create agents that maintain partisan consistency under extended negotiation, overcoming the neutrality bias of standard LLMs.
2. Explainability: Moving beyond "black box" outcomes by using MILT to attribute every clause to a specific party and manifesto origin, distinguishing between valid negotiation dynamics and system noise.
3. External Validity: Showing that manifesto-anchored lineage reliably predicts real-world materialization, whereas hallucinated content does not.
4. Methodological Rigor: Providing a reproducible framework where the "winner" is a property of the simulator's dynamics rather than a single stochastic draw, validated against historical data.

The authors conclude that while the simulation is closed (omitting socio-economic and media channels) and relies on a single historical referent, the alignment between internal lineage and external grounding makes the topology an actionable, ex-ante reliability signal for political strategy.