Political Stance Detection on X During the 2024U.S. Election: A Comparative Study of Classical, Neural, and Transformer Models

Imagine the internet as a giant, chaotic town square where millions of people are shouting their opinions all at once. In this digital plaza, a specific field of science called Computational Social Science tries to listen in. These researchers use computers to understand how humans behave online, turning messy shouting matches into data they can study. To do this, they often rely on two main tools: Sentiment Analysis, which is like a simple mood ring that just tells you if someone is happy or angry, and Stance Detection, which is a much sharper detective skill. Stance detection doesn't just ask, "Are you mad?" It asks, "Who are you mad at, and what side are you on?" This matters because during big events like elections, knowing who people support helps us understand the mood of the nation, but it's incredibly hard to do when people use slang, emojis, sarcasm, and short, punchy messages.

This paper is like a massive taste test where a team of researchers from the University of Delaware set up a lab to see which "detective" works best at figuring out who people are supporting in the 2024 U.S. presidential election. They looked at nearly 100,000 posts from the social media platform X (formerly Twitter) to see if they could tell if a user was cheering for Donald Trump, cheering for Joe Biden and Kamala Harris, or staying neutral. They didn't just pick one fancy tool; they pitted three different types of "detectives" against each other: the Classical detectives (old-school math tools), the Neural detectives (brain-like computer networks), and the Transformer detectives (super-smart, modern AI models that read context). Their goal was to find out if you need a super-complex, expensive AI to solve the puzzle, or if a simpler, well-tuned tool could do just as good a job.

The Great Detective Showdown

The researchers started by cleaning up the messy data. Social media posts are full of noise—like typos, weird symbols, and links that don't matter. They acted like librarians, organizing the books, removing the duplicates, and translating the slang into plain English so the computers could understand. They also created a special "Candidate Affiliation" label. Instead of just saying "this post is happy" or "this post is sad," they trained their system to ask, "Is this person talking about Trump, Biden-Kamala, or neither?" They used a mix of keyword spotting and topic analysis to figure this out, and they even had human experts double-check their work to make sure the labels were accurate.

Once the data was ready, they ran the three types of models through a series of tests.

First, they tried the Classical Models. Think of these as the reliable, old-school detectives who use a magnifying glass and a notebook. They are fast, easy to understand, and don't need a supercomputer to run. The researchers used tools like Support Vector Machines (SVM) and Logistic Regression. These models looked at the words in the posts and counted how often certain political keywords appeared.

Next, they tried the Neural Models. These are like detectives with a "brain" made of layers. They used something called Sentence-BERT, which turns a whole sentence into a single, dense number (an embedding) that captures the meaning of the words, not just the words themselves. It's like translating a sentence into a secret code that represents its vibe.

Finally, they tried the Transformer Model, specifically BERTweet. This is the "super-detective." It's a massive AI that has read millions of tweets before and knows exactly how people use slang, hashtags, and emojis on X. It's the most complex and powerful tool in the room, designed specifically to understand the chaotic language of social media.

The Results: Who Won the Race?

Here is where the story gets interesting. The researchers expected the "super-detective" (BERTweet) to win easily because it's the most powerful. And, in a way, it did. When they gave BERTweet the full dataset and let it learn everything, it achieved a score of 0.995 on the test set. That is an incredibly high score, suggesting it was almost perfect at guessing the political stance.

However, the paper suggests a very important twist: Complexity isn't everything.

When the researchers looked closely, they found that the "old-school" Linear SVM (the classical model) was a fierce competitor. After they cleaned the data and tuned the settings just right, the SVM scored a 0.963. That is a tiny bit lower than the super-AI, but it is still an amazing score. The difference between them was so small that the researchers suggest the extra power of the super-AI might not be worth the cost for many real-world jobs.

The study also found that the "brain-like" neural models (Sentence-BERT + MLP) sat right in the middle, scoring around 0.85. This suggests that once you have a good way to understand the meaning of a sentence, making the "brain" deeper or more complex doesn't help much more.

The "Extra Clues" Experiment

The researchers also wondered: "What if we give the detectives extra clues?" They tried adding non-text information, like:

• When the post was made (time of day, day of the week).
• Where the user was located (geographic data).
• How popular the post was (likes, reposts, engagement).
• Safety flags (is this hate speech?).

They tested these clues one by one and in groups. The result? The extra clues helped a little bit, but not much. The "text-only" detectives were already so good at reading the words that adding location or time data didn't change the outcome significantly. In fact, sometimes adding too many extra clues made the models slightly worse because the extra data was just noise. The paper suggests that for this specific task, the words themselves are the most important clue, and fancy metadata doesn't guarantee a better answer.

The Big Takeaway

So, what does this all mean for the future of election monitoring?

The paper suggests that while the "super-detective" (Transformer models) is the strongest in a controlled, perfect environment, it might be overkill for many situations. The Linear SVM, a much simpler and faster tool, performed almost as well. This is a big deal because the simple tool is:

• Faster: It runs on regular computers, not expensive supercomputers.
• Clearer: It's easier to understand why it made a decision (interpretability).
• Cheaper: It costs less to run.

The authors warn that the near-perfect score of the super-AI might be specific to this particular dataset and the way the data was labeled. They don't claim that political stance detection is "solved" or that the AI is perfect in the real world, where people might be more sarcastic or the language might change. Instead, they suggest that for building reliable, scalable systems, good data preparation and smart feature selection matter more than just picking the most complex model.

In short, you don't always need a Ferrari to win a race; sometimes a well-tuned bicycle is just as fast, easier to fix, and much cheaper to drive. For tracking political opinions on social media, a carefully designed, simple model might be the most practical choice.