The Political Ideology of Large Language Models: Measurement, Inconsistency, and Persuasive Influence
Challenging the view that Large Language Models possess only minor political biases, this study demonstrates that while their overall positioning appears moderate due to offsetting partisan stances on specific topics, they can nonetheless exert persuasive influence on users' political attitudes comparable to professional campaign advertising, particularly when their responses are explicitly steered.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the landscape of modern politics, a person's ideology is rarely a single, solid block of belief. Instead, it is a collection of opinions on many different issues, from healthcare to immigration, that can sometimes pull in opposite directions. Political scientists have long understood that most people do not hold perfectly consistent views; a voter might be very liberal on one topic and very conservative on another, or they might simply hold no strong opinion at all. For decades, researchers have measured these positions by asking people how they would vote on laws or how they feel about specific policies, creating a map of where individuals stand. Now, a new question has emerged as artificial intelligence becomes a daily tool for millions: do the large language models we talk to every day have their own political maps? If these models, which are increasingly used to find information and form opinions, hold hidden biases, they could subtly shape how people see the world without anyone realizing it.
A team of researchers set out to answer this by treating artificial intelligence not as a neutral calculator, but as a political actor. They examined forty-three different large language models, asking each one to act as a member of the U.S. Congress, a Supreme Court justice, or a regular voter. They did not ask the models to simply state their beliefs; instead, they asked them to make decisions on real-world bills, court cases, and survey questions, just as a human would. The researchers then compared these digital decisions to the actual voting records of real politicians and the survey answers of thousands of real Americans. What they found was that these models do not sit in a quiet, neutral middle ground. Instead, they display a complex and often contradictory mix of views. On some issues, a single model might take a position more liberal than the strongest Democrats, while on other issues, the same model might take a stance more conservative than the strongest Republicans. When you average all these views together, the models often appear moderate, but this is an illusion created by canceling out strong, opposing opinions, much like a real voter who holds a mix of conflicting views.
The study also looked at where these views come from. The researchers discovered that the political leanings of these models are not simply a reflection of the vast amount of text they were trained on. Instead, the specific instructions and safety filters applied to the models after their initial training—processes designed to make them helpful and harmless—are what push them toward certain political sides. In fact, when the researchers tested the "raw" versions of these models before these final adjustments, the models were much closer to the political center. It is the final tuning that gives them their distinct, and often liberal-leaning, political signature. This signature is not just a static list of opinions; it changes depending on how the model is asked. When prompted to act as a voter, a judge, or a legislator, the models express different sets of positions, showing that their "ideology" is a behavior they perform in response to a prompt, rather than a deep-seated belief system.
The most significant part of the research, however, was to see if these digital opinions could actually change the minds of real people. The researchers conducted a large experiment where thousands of American participants discussed policy issues with these models. They tested two different scenarios. In the first, the models were allowed to talk naturally, answering questions without being told to argue for a specific side. In the second, the models were explicitly instructed to argue for a specific viewpoint, such as supporting a new gun control law or opposing a tax increase. The results were striking. When the models talked naturally, they had no measurable effect on the participants' opinions. The people who spoke with them did not shift their views in any particular direction. However, when the models were told to argue for a side, they became powerful persuaders. Participants who spoke with a model instructed to argue for a specific policy shifted their own views to align with that model by an average of more than ten percentage points. This effect was strong enough to match, and in some cases exceed, the influence of professional political campaign advertisements.
Crucially, the study found that this power to persuade did not depend on the person's background. It did not matter if the participant was highly educated, followed the news closely, or was very familiar with artificial intelligence; the influence worked just as strongly on everyone regardless of these factors. However, the direction of the argument relative to the participant's own party did matter in a surprising way. The researchers found that participants were actually less likely to shift their views when the model argued against their own political party, and more likely to shift when the model argued in line with their party. The researchers also ruled out the idea that the models were simply agreeing with whatever the user already thought. Because the models were instructed to argue for a specific side before the conversation even started, and because the users did not tell the models their own views beforehand, the models could not be mirroring the user. Instead, the models were genuinely influencing the conversation.
The findings suggest that while these artificial intelligence tools may seem like neutral sources of information when left to their own devices, they possess a hidden capacity to shape political opinion when guided to do so. The researchers emphasize that this influence is not a result of the models having a soul or a political agenda of their own, but rather a result of how they are built and instructed. If a government or a corporation wants to influence public opinion, they do not need to create a propaganda machine; they simply need to adjust the instructions given to the models that millions of people already use. The study concludes that as these tools become more embedded in our daily lives, from search engines to personal assistants, the way they are tuned will have profound consequences for how we understand the world and how we vote. The power to change a person's mind is no longer reserved for human speakers; it can now be coded into the software we talk to every day.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.