🤖 machine learning

Quantifying Political Partisanship for Cross-Platform Analyses

This paper introduces a platform-portable, text-based methodology that leverages transformer embeddings and external news-credibility signals to reliably measure and compare political partisanship across diverse social media platforms, demonstrating its effectiveness through a cross-platform analysis of Bluesky and Truth Social.

Original authors: Fathima Ameen, Christopher G. Healey

Published 2026-07-27

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Fathima Ameen, Christopher G. Healey

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Great Digital Echo Chamber

Imagine the internet as a massive, chaotic town square where billions of people shout their opinions. In recent years, scientists have noticed something strange: people in this square seem to be shouting louder and louder, but only at people who agree with them. This phenomenon is called political polarization. It's like two groups of neighbors building a wall between their houses, refusing to talk to each other, and believing the other side is dangerous.

To understand if this wall is getting higher, researchers need a way to measure exactly how "left" or "right" a person's opinion is. But here's the tricky part: the internet isn't just one square anymore. It's fragmented into different neighborhoods, or platforms. Some are the big, established cities like X (formerly Twitter), while others are newer, smaller towns like Bluesky or Truth Social, built specifically for people who feel the big cities don't listen to them.

The problem is that the "language" of these neighborhoods is different. A joke in one town might be a serious insult in another. A hashtag that means "I support this" in one place might mean something totally different in another. For a long time, scientists could only measure opinions within a single town. If they tried to compare the "left-ness" of Bluesky to the "right-ness" of Truth Social, the results were like comparing apples to oranges because the measuring sticks were different. This paper asks a big question: Can we build a single, universal ruler that works in every neighborhood, no matter how different they are?

The Universal Ruler for Online Opinions

In this study, Mia Ameen and Christopher G. Healey from North Carolina State University decided to build that universal ruler. They wanted to see if they could measure political bias in social media posts using only the words people wrote, without needing to know who their friends were or what specific hashtags they used. Their goal was to create a method that could travel across different platforms and give a fair score to every post.

To do this, they treated every social media post like a piece of paper in a giant library. First, they used a super-smart computer brain (a transformer-based sentence encoder) to read the text and turn it into a mathematical "fingerprint." This fingerprint captures the meaning of the words, not just the spelling. Next, they grouped these fingerprints into clusters, like sorting books by genre.

Here is the clever part: to figure out which cluster was "left" and which was "right," they didn't ask the users. Instead, they looked at the news links inside the posts. They used a pre-existing tool called AllSides, which rates news outlets on a scale from "far-left" to "far-right." If a cluster of posts mostly linked to left-leaning news, the whole group got a left-leaning label. If they linked to right-leaning news, they got a right-leaning label.

Once they had these labeled groups, they drew a line through the mathematical space connecting the "left" groups to the "right" groups. This line became their partisanship axis. Now, they could take any post, even one with no news links, and project it onto this line to see where it landed. A score on the left side meant the post was likely left-leaning; a score on the right meant it was right-leaning.

They tested this new ruler on a massive collection of about 1.3 million posts collected from Bluesky and Truth Social during the six months leading up to the 2024 U.S. presidential election (from May to October 2024). They also tested it on a separate dataset from X (Twitter) to see if the ruler worked on a platform it had never seen before.

What They Found

The results were promising. The ruler worked surprisingly well. When they checked the posts that did have news links, the scores the ruler gave matched the actual ratings of those news outlets. For the Bluesky and Truth Social posts, the match was strong (a correlation of 0.365). Even more impressively, when they applied the ruler to the X (Twitter) posts—a completely different platform it wasn't trained on—the scores still pointed in the right direction (a correlation of 0.193). This suggests the ruler isn't just memorizing the "style" of one specific app; it's actually measuring the underlying political sentiment.

The study also revealed some fascinating dynamics that a simple "platform identity" check would have missed. For example, on Bluesky (a platform generally known for left-leaning users), posts mentioning President Biden started out with left-leaning scores. But as the summer of 2024 went on, those scores drifted toward the center and even became slightly right-leaning by September, right after Biden dropped out of the race. This showed that the users' feelings about Biden were changing, not just the platform's general vibe. Similarly, posts about Kamala Harris on Bluesky became increasingly left-leaning starting in July, peaking around the first presidential debate.

On the other side, the study looked at the language used by the most extreme groups. They found that the most strongly left-leaning posts often named specific politicians (like Biden, Harris, and Trump). In contrast, the most strongly right-leaning posts tended to use words related to civic duty and morality, such as "people," "God," "country," "America," and "need."

Why It Matters

The authors suggest that this method is a big step forward because it allows scientists to compare political moods across different digital worlds. Before this, it was hard to know if a shift in opinion on Truth Social was real or just a change in how the platform worked. Now, they can see that the "wall" between groups is real and shifting in response to actual events, like a candidate dropping out of a race.

However, the authors are careful to note that this ruler has limits. It was built specifically for the U.S. two-party system using American news ratings, so it might not work in countries with many political parties. Also, it relies on the fact that people share news links; if a platform is full of people who never share links, the ruler can't calibrate itself. But for the platforms where it works, it offers a new way to watch the political temperature of the internet without getting lost in the noise of different apps.

Technical Summary: Quantifying Political Partisanship for Cross-Platform Analyses

Problem Statement

Research on political polarization in social media is hindered by the inability to reliably measure partisanship in a way that generalizes across different platforms. Existing methodologies are typically tailored to platform-specific properties, such as structural affordances (e.g., Reddit's subreddit system, Twitter's retweet graphs) or linguistic conventions (e.g., specific hashtags or slang). As the social media ecosystem fragments and "alt-tech" platforms (e.g., Truth Social, Bluesky) emerge alongside mainstream ones, these platform-specific approaches fail to provide comparable metrics. This limitation prevents rigorous cross-platform analysis of polarization, echo chambers, and radicalization, particularly when comparing ideologically asymmetric environments.

Methodology

The authors propose a text-based, platform-portable methodology for measuring political partisanship that relies solely on post text and external media bias ratings, avoiding reliance on follower graphs, community structures, or platform-specific conventions. The pipeline consists of four stages:

Data Collection and Preprocessing:
- Corpus: Approximately 1.3 million posts collected from Bluesky and Truth Social between May 1 and October 31, 2024 (the six months preceding the 2024 U.S. presidential election).
- Cleaning: Posts were deduplicated (removing near-identical content), filtered for substantive text (minimum three tokens after removing URLs/hashtags), and stripped of HTML/emojis.
- Embedding: Cleaned posts were embedded using a transformer-based sentence encoder (qwen3-embedding:8b), producing 128-dimensional vectors. These were L2-normalized and reduced via Principal Component Analysis (PCA) to retain 95% of the variance.
Cross-Platform Clustering and Labeling:
- Clustering: Posts from both platforms were jointly clustered using MiniBatch k-means ( $k=36$ ).
- Labeling via External Signal: The authors extracted URLs from posts and mapped them to news domains. Using AllSides media bias ratings (a 5-point scale from -2 [far-left] to +2 [far-right]), they aggregated bias scores for cited domains within each cluster.
- Stabilization: To handle clusters with few citations, Empirical Bayes shrinkage was applied to the raw mean bias scores, contracting them toward a global prior mean. Clusters were labeled as left-leaning ( $\le -0.5$ ), right-leaning ( $\ge 0.5$ ), neutral, or no signal.
Social Dimension Construction:
- Seed Pair Selection: The authors algorithmically selected four pairs of opposing clusters (left vs. right) based on high cosine similarity between their centroids, ensuring the pairs differed primarily along the partisan dimension rather than content topic.
- Axis Definition: A partisanship axis ( $\mathbf{d}$ ) was defined as the mean vector difference between the L2-normalized centroids of these opposing seed pairs: $\mathbf{d} = \frac{1}{n} \sum (\hat{\mathbf{r}}_i - \hat{\mathbf{l}}_i)$ .
- Scoring: Individual posts were scored by projecting their embeddings onto this axis. Scores were standardized to z-scores, where positive values indicate right-leaning alignment and negative values indicate left-leaning alignment.
Validation:
- In-Distribution: Tested on held-out posts from Bluesky and Truth Social that cited news sources but were not part of the seed clusters.
- Out-of-Distribution: Tested on an independent corpus of X (Twitter) posts from the same election period, which was never used to construct the axis.
- Sensitivity Analysis: Compared the performance of the selected seed pairs against all 630 possible cluster pairings to ensure the signal was not an artifact of arbitrary selection.

Key Results

Cross-Platform Correlation: The constructed partisanship axis showed a significant positive correlation with the AllSides bias scores of cited news sources in both evaluation sets:
- Bluesky/Truth Social (In-Distribution): $r = 0.365$ ( $p < 0.001$ ).
- X (Out-of-Distribution): $r = 0.193$ ( $p < 0.001$ ).
- The persistence of a positive correlation on X, a platform unseen during axis construction, demonstrates that the method captures a general partisan signal rather than platform-specific noise.
Sensitivity Analysis: The axis derived from the authors' selected seed pairs outperformed all other possible cluster pairings. It fell in the 100th percentile for in-distribution performance and the 99.5th percentile for out-of-distribution (X) performance, confirming the robustness of the seed selection procedure.
Dynamic Partisan Trends: The method successfully recovered within-platform dynamics that platform identity alone could not explain:
- On Bluesky (typically left-leaning), posts mentioning Biden shifted from neutral/left to distinctly right-leaning (critical) following his July withdrawal, while posts mentioning Kamala Harris became increasingly left-leaning, peaking around the first presidential debate.
- On Truth Social, mentions of Biden and Harris spiked rightward in July, reflecting Republican reaction to the candidate transition.
Linguistic Asymmetry: Qualitative analysis revealed that strongly left-leaning posts referenced specific candidates (Biden, Harris, Trump), while strongly right-leaning posts were dominated by civic and moral vocabulary (e.g., "people," "god," "country," "america").

Key Contributions

A Portable Measurement Method: A text-based partisanship metric informed by external media bias ratings, validated to produce comparable scores across three distinct platforms (Bluesky, Truth Social, and X).
First Cross-Platform Comparison: The first comparative analysis of political discourse on Bluesky and Truth Social during the 2024 U.S. presidential election, addressing a gap in literature regarding "alt-tech" platforms.
Empirical Evidence of Dynamic Sentiment: Demonstration that the approach captures shifting partisan dynamics (e.g., the erosion of support for Biden on Bluesky) rather than merely recovering static platform identities.

Significance and Claims

The paper claims that its primary contribution is a methodological framework that enables rigorous cross-platform comparison of political partisanship. By anchoring the measurement in external, consistent bias ratings (AllSides) and relying solely on text embeddings, the method overcomes the generalizability limitations of prior work.

The authors assert that this approach allows researchers to:

Investigate how different platforms respond to the same political events.
Track how partisan attention shifts over time within and across platforms.
Analyze how the same content is interpreted on ideologically opposing platforms.

The paper concludes that while the method is currently U.S.-centric (due to reliance on AllSides ratings) and sensitive to clustering choices, it provides a foundational tool for studying temporal polarization dynamics, cross-platform information flow, and platform-motivated radicalization in a fragmented social media ecosystem.