From Entity Mentions to Tone: An LLM-Based Pipeline for Media Bias Analysis

In the modern world, the news we read rarely arrives as a neutral stream of facts. Instead, it is filtered through the choices of editors and the algorithms that decide what we see. These filters act as gatekeepers, determining which stories reach us and how they are framed. A story about a political event might be presented as a triumph by one outlet and a failure by another, not because the facts changed, but because the tone and focus shifted. This phenomenon, known as media bias, is difficult to track on a large scale because it involves subtle shifts in language, the selection of specific details, and the emotional weight assigned to different people and events. For decades, understanding these patterns required teams of human experts to read and analyze articles one by one, a slow and expensive process that could not keep up with the speed of the internet.

Researchers have begun to turn to artificial intelligence to solve this problem, hoping to automate the detection of these subtle biases. However, a significant hurdle remains for many parts of the world: most advanced language tools are built for major languages like English, leaving smaller languages without reliable software to analyze their own news. In these regions, the lack of specialized tools means that even basic questions—such as which news sources covered a specific event or how different outlets described the same public figure—remain difficult to answer consistently. Without these tools, it is hard to see the full picture of how information is being shaped and distributed.

A team of researchers from the University of Tirana in Albania has developed a new approach to bridge this gap. They created a system that uses a powerful, general-purpose language model to read thousands of news articles written in Albanian, a language with very few dedicated digital tools. The system does not just read the text; it breaks it down into three distinct layers of analysis. First, it groups articles into topics and specific events to see what stories are being told. Second, it identifies the names of people and organizations mentioned in those stories. Third, and perhaps most importantly, it measures the emotional tone of the writing, determining whether the coverage of a person or event is positive, negative, or neutral. By combining these layers, the pipeline can map out not only what a news source chooses to cover, but also how it describes the people involved in those stories.

To test their system, the researchers gathered a collection of 8,358 news articles from 124 different Albanian news sources, published in April 2026. They fed these articles into their pipeline, which used a local version of an advanced language model to extract the necessary data. The goal was to see if this automated method could produce reliable results without needing a massive, pre-existing database of human-labeled examples. The researchers compared their system's output against the annotations provided by GDELT, a large global database that also uses automated tools to track news. The comparison revealed a moderate level of agreement between the two systems, suggesting that the new pipeline was working in a similar direction to established methods. More importantly, the researchers found that their system identified many mentions of people that the GDELT system had missed. This ability to catch additional names is crucial for bias analysis, because missing a public figure can distort the understanding of how different outlets treat that person.

The study also tested two different sets of instructions, or prompts, given to the language model to see which would work better. One set of instructions was strict, requiring the model to double-check its own work to ensure that the emotional labels it assigned matched the numerical scores it gave. The other set was simpler and faster. The strict instructions did eliminate inconsistencies between labels and scores, but they also slowed the process down significantly and caused the system to skip labeling nearly half of the articles it processed. The simpler instructions, while producing a few more inconsistencies, allowed the system to analyze a much larger portion of the news collection and run much faster. The researchers concluded that for a system designed to monitor news continuously, the faster, broader approach was more practical, as the few inconsistencies could be cleaned up later with simple rules.

Using the data generated by this pipeline, the researchers were able to create detailed profiles of how different news sources operate. They found that some outlets consistently framed topics with a more favorable tone, while others were systematically more critical. The system could also track specific public figures, showing how the same person might be described with a positive tone in one newspaper and a negative tone in another. For example, the analysis showed that coverage of certain political figures varied widely depending on the source, with some outlets offering almost entirely positive coverage while others were predominantly critical. Additionally, the system could identify "gatekeeping" events—stories that were covered by only a small handful of sources while being ignored by the rest of the media landscape. This allowed the researchers to see not just the tone of the news, but also the reach of specific stories and where information was being concentrated or withheld.

The work demonstrates that it is possible to build a structured, automated system for analyzing media bias in low-resource languages without needing specialized, language-specific training data. The researchers emphasize that their tool is not intended to replace human judgment or expert review, but rather to act as a fast starting point. It turns a massive collection of articles into clear signals that can be inspected more easily. By automating the initial extraction of names, topics, and tones, the system reduces the amount of material that humans need to read from scratch. This is particularly valuable in environments where continuous monitoring is difficult and where specialized language tools are scarce. The researchers suggest that this approach provides a reproducible blueprint that can be applied to other languages and regions, offering a way to visualize how news is framed and who is being heard in the public conversation.