Explainability and Interpretability Methods for Large Language Models: An Exploratory Systematic Mapping Study
This systematic mapping study analyzes 1,252 research papers on explainability methods for large language models, revealing that most techniques require inaccessible internal model data and that evidence regarding human utility and real-world deployment remains critically scarce.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the last few years, a new kind of computer program has reshaped how we interact with technology. These systems, known as large language models, can write stories, solve math problems, and hold conversations that feel remarkably human. They work by predicting the next word in a sentence, layer upon layer, until a complete thought emerges. But because these programs are built with billions of adjustable settings, their internal workings are often a mystery. When such a system gives an answer, it is difficult to know if that answer came from a deep understanding of the facts or simply from a clever guess based on patterns it has seen before. This uncertainty has created a pressing need for explainability: the ability to look inside the machine and understand why it made a specific choice. Researchers are trying to build tools that can trace a decision back to its source, revealing the logic, the data, or the specific parts of the computer's brain that led to the final output.
A team of researchers at the University of Piraeus in Greece has taken a massive step toward clarifying this field. They conducted a comprehensive survey of nearly 8,000 scientific reports published between 2019 and August 2026, narrowing them down to 1,252 studies that specifically tried to explain how these large language models work. Their goal was not to invent a new method, but to map the entire landscape of existing methods. They wanted to see what kinds of explanations scientists are building, what parts of the computer they are allowed to look at, and whether these explanations actually help people understand the machine or just look convincing. To manage this enormous task, the researchers used a carefully designed artificial intelligence system to read and categorize every paper, with human experts checking the work to ensure accuracy.
The most striking discovery from their map is a fundamental mismatch between the tools scientists are building and the reality of how these models are used today. The vast majority of the methods they found—nearly 70 percent—require deep, internal access to the computer's brain. To use these tools, a researcher needs to see the raw numbers flowing through the system or even change the settings of the model itself. However, the most powerful language models that people use every day are hosted by companies as closed services. Users can send a message and get a reply, but they cannot see the internal numbers or change the settings. This means that the most sophisticated ways of explaining these models are effectively useless for the very systems where people need explanations the most. Only about 23 percent of the studies described methods that could work with the limited access available to the public, such as just reading the text output or the probability scores of the next word.
The researchers also looked at what these explanations were actually proving. They found that most studies focused on "faithfulness," which is a technical way of asking if the explanation truly matches the computer's internal logic. About 70 percent of the papers tried to show that their method correctly identified the cause of a decision. However, the study found that very few researchers tested whether these explanations were actually useful to human beings. Only about 3 percent of the studies involved real people testing the explanations to see if they helped them make better decisions, understand the system, or trust the output. It is as if engineers have spent years perfecting a map of a city's underground tunnels, but have never asked a commuter if the map helps them find their way.
Another significant gap appeared in how the researchers tested their ideas. While many studies claimed their methods were robust, fewer than a third of them used a rigorous control group to prove their results were not just random chance. Furthermore, the connection to real-world rules and regulations was almost non-existent. Despite the frequent mention of laws regarding artificial intelligence transparency, only a tiny handful of the studies actually linked their technical methods to specific legal requirements. The researchers noted that while the field is growing rapidly, with nearly half of the included studies published in just the last year, it remains heavily focused on open-source models that anyone can download, rather than the closed, proprietary systems that dominate the market.
The study concludes that the field of explaining large language models is at a crossroads. The tools currently available are powerful but often inaccessible, and they prioritize technical correctness over human utility. The researchers suggest that future work must focus on creating methods that work with the limited access provided by commercial services and on testing these explanations with real people. Until then, the most advanced explanations remain locked away in the labs of researchers who have full access to the machines, leaving the general public with systems that are increasingly powerful but increasingly difficult to understand.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.