Elite Polarization in European Parliamentary Speeches: a Novel Measurement Approach Using Large Language Models

Technical Summary: Elite Polarization in European Parliamentary Speeches

Problem Statement
Comparative political science has long recognized that mutual emotional hostility among political elites can destabilize democracies more rapidly than mass-level grievances. However, existing measurement tools fail to capture this specific phenomenon directly. Current comparative measures typically focus on either mass affective polarization (public sentiment toward parties) or elite ideological distance (policy positions). There is a significant methodological gap regarding directed mutual elite evaluation: the specific sentiment one political actor expresses toward another within official discourse. Traditional methods, such as survey aggregation, are limited by frequency and country scope, while manual coding or dictionary-based Natural Language Processing (NLP) lacks the scalability, cross-lingual capability, and contextual nuance required to analyze millions of parliamentary references across multiple languages and decades.

Methodology
The paper introduces the Elite Polarization Score (EPS), a novel measurement approach utilizing Large Language Models (LLMs) to analyze parliamentary speeches. The methodology involves a multi-stage pipeline:

1. Unit of Analysis Disaggregation: Unlike traditional sentiment analysis that assigns a single score to an entire speech, this approach disaggregates the unit of analysis to the individual reference. The LLM identifies every instance where a politician mentions another political actor.
2. Actor-Target-Sentiment Extraction: Using a standardized prompt and Chain-of-Thought reasoning, the LLM extracts:
   • The speaker and the target actor.
   • The specific context of the reference.
   • The sentiment directed specifically at the target actor (distinguishing the actor's evaluation from the evaluation of their circumstances).
   • Sentiment is scored on a scale from −5 (strongly dislikes) to +5 (strongly likes).
3. Standardization and Entity Linking: Heterogeneous references (e.g., "Prime Minister," "Orbán," "Fidesz") are standardized into party dyads. This process combines deterministic lookup scripts with LLM-assisted fuzzy matching to resolve named entities to their specific political parties within a given country-year context.
4. Aggregation: The system calculates a weighted average of out-party evaluations.
   • Party Level: Evaluations are weighted by the reference share (how often the target party is mentioned).
   • Parliament Level: Party-level scores are aggregated using vote share weights.
   • The final score reverses the sign so that higher values indicate greater negativity (polarization).
5. Validation Protocol: The authors employ a "supergold" union standard constructed via LLM-assisted fuzzy matching. Human coders and the AI annotate the same sample of speeches; the union of valid references identified by either serves as the ground truth. This avoids penalizing the AI for identifying valid references humans missed.

Key Contributions

• Conceptual Innovation: The EPS occupies a previously unmeasured quadrant in polarization typology: elite-level affective polarization based on directed out-party evaluations in speech. It is distinct from ideological distance, general sentiment, incivility, and mass affective polarization.
• Methodological Scalability: The approach leverages pre-trained LLMs (specifically GPT-4o and open-weight alternatives) that operate across languages (Hungarian, Italian, English) without task-specific training or fine-tuning. This enables the first tractable, multilingual, longitudinal panel of elite polarization.
• Technical Validation: The study demonstrates that LLMs can achieve high sensitivity (recalling more references than human coders in some settings) and high sentiment agreement (Mean Absolute Error of 0.85–1.48 on a 10-point scale) with human coders. Crucially, the False Discovery Rate (FDR) was zero across all validation samples, confirming that every AI-identified reference had a genuine textual anchor.

Results
The methodology was applied to parliamentary corpora from the United Kingdom (1987–2020), Hungary (2002–2023), and Italy (2013–2022).

• Hungary: The EPS captures the consolidation of the regime and the reconfiguration of the party system. Polarization peaked in 2018–2019, rather than during the initial phase of executive aggrandizement (2010–2011), suggesting the measure reflects the reaction of the opposition and the system's new equilibrium more than the immediate institutional takeover.
• United Kingdom: The index shows a U-shaped trend (declining post-1997, rising post-2010). It is empirically distinct from mass affective polarization and ideological polarization indices; notably, during the Brexit period, elite polarization did not spike despite high public and ideological tension, indicating that parliamentary rhetoric remained constrained by institutional norms.
• Italy: The measure is highly responsive to coalition dynamics. Out-party negativity drops when former rivals enter government together (e.g., the Draghi unity government) and rises when they return to opposition.
• Validation: The approach successfully distinguishes between different types of polarization. In the UK case, the EPS showed low correlation with mass affective polarization in later years, confirming it measures a distinct elite-level phenomenon.

Significance and Claims
The paper claims that the Elite Polarization Score provides a scalable, cross-national basis for future research into the causes and consequences of elite hostility.

• Distinctiveness: The measure is conceptually and empirically distinct from existing metrics, offering a new lens to study democratic stability without relying on US-centric data or mass surveys.
• Feasibility: The study proves that LLMs can perform fine-grained political text analysis (actor detection and sentiment attribution) at a scale previously impossible, moving beyond the need for expensive, manual coding or narrow, single-country studies.
• Future Utility: The authors posit that this measure allows scholars to treat elite polarization as both a dependent variable (studying what produces it, such as elections or crises) and an independent variable (studying what it produces, such as democratic backsliding or populist escalation). The pipeline is designed to be extended to all 29 EU member states via the ParlaMint corpus, potentially creating a twenty-year EU-wide panel.