Imagine you've built a super-smart robot that can predict the weather, diagnose illnesses, or even write poetry. It works amazingly well, but it's a "black box": you press a button, it spits out an answer, and you have no idea why it chose that answer. This is the world of Artificial Intelligence (AI). To fix the mystery, scientists created a field called Explainable AI (XAI). Think of XAI as a translator that tries to explain the robot's secret thoughts to us humans. We have tools that highlight which words mattered most in a sentence, or which pixels in a photo made the robot think "cat." But here's the catch: just because the robot can give us a translation doesn't mean the translation actually helps us do anything useful. We need to know if these explanations are trustworthy, if they match the robot's real brain, and most importantly, if they actually help us make better decisions or fix the robot when it goes wrong.

This is exactly the problem a new paper tackles. The authors argue that the AI community has been running a marathon in the wrong direction. Instead of building a solid foundation for how and why we explain things, everyone has been frantically inventing new, flashy "flashlights" (methods) to shine on the black box. The paper suggests that while these new flashlights are cool, they often don't help us navigate the dark because we haven't agreed on what a "map" even looks like, how to check if the map is accurate, or what to do once we have one.

The "Flashlight" Problem

The authors, a team of researchers from top universities and tech companies, looked at hundreds of recent papers on Explainable AI and talked to dozens of people who actually use these tools in real jobs. They found a worrying pattern: the field is obsessed with creating new methods. In fact, 77% of the papers they analyzed were just proposing a new way to explain AI, while only 11% bothered to give a clear, formal definition of what an "explanation" actually is.

It's like a group of chefs trying to invent new recipes without ever agreeing on what "soup" is. Some say soup must be hot; others say it must have carrots; some just throw everything in a pot and call it soup. Because everyone is using different definitions, the results are a mess. One researcher might say their method is "faithful" (meaning it tells the truth about the AI), while another uses a different test and says the same method is a lie. The paper argues that we are stuck in a loop of making things up as we go, rather than building a reliable system.

The Four Missing Pillars

The authors say we need to stop and build four specific foundations before we keep inventing new flashlights. They call these the "Four Challenges":

1. The Definition Problem (What is an explanation?)
Right now, the word "explanation" is used for everything. Sometimes it means a list of important features; other times it means a story the AI tells itself. The paper argues we need to stop guessing. An explanation should be defined by who it is for and what they need to do with it. If a doctor needs to know why a patient was denied insurance, the explanation needs to be different than if a robot engineer needs to fix a bug. Without a clear definition, we can't even start to measure if we are doing a good job.

2. The Property Problem (What makes a good explanation?)
We need to agree on the rules. A good explanation must be "faithful"—it has to actually match what the AI is doing, not just look pretty. But faithfulness isn't enough. An explanation also needs to be "complete" (it shouldn't miss the most important parts) and "robust" (it shouldn't change wildly if you tweak the input slightly). The paper points out that many current methods fail these tests. For example, a method might look great on a test but fail to help a human make a decision. The authors suggest that we need to test these properties rigorously, not just assume they work because the math looks nice.

3. The Evaluation Problem (How do we test if it works?)
This is where things get really messy. The paper found that while 56% of researchers claim to test if their explanations are "faithful," very few actually talk to humans. Only 11% of the papers they reviewed included a study with real people. Most researchers just run computer simulations or check if the numbers look stable. But an explanation is useless if a human can't understand it or doesn't trust it. The authors argue that we need to stop treating explanations like a math problem and start testing them like a product: does it actually help a human do their job better?

4. The Action Problem (What do we do with the explanation?)
This is the biggest gap. Imagine a doctor gets an explanation that says, "The AI thinks this patient has a rare disease because of their age." Great, but what does the doctor do with that? Can they change the AI's mind? Can they ask for more tests? The paper found that 38.2% of practitioners struggle to integrate these explanations into their actual workflows. Currently, explanations are often just "show-and-tell" items that you look at and then throw away. The authors argue that explanations need to be part of a loop where humans can give feedback, fix the AI, and improve the system. Without this, the explanation is just a decoration.

What the Data Says

To back up their claims, the authors did two big things. First, they used an AI (an LLM) to read 617 papers from top conferences like NeurIPS and ICML. The AI found that the field is driven almost entirely by "new method" papers, with very little focus on definitions or human testing.

Second, they surveyed 34 real-world practitioners—people who actually use these tools in their jobs. The results were eye-opening:

• Confusion: 44.1% of them said they didn't know how to check if an explanation was any good.
• Choice Paralysis: 23.5% admitted they didn't know which method to pick, and 26.5% didn't know how to choose the right settings (parameters) for the tools they used.
• The "Trust" Trap: While many people said these tools helped them "trust" the AI, the paper warns that trust without understanding is dangerous. In fact, 44.1% of practitioners said the biggest challenge was explaining the results to the people who actually have to make the decisions.

The Path Forward

The authors aren't saying we should stop making new tools. They aren't saying XAI is broken forever. They are saying we need to change our priorities. Instead of rushing to build the 100th new flashlight, we need to agree on what a map looks like, how to draw it accurately, and how to use it to get to our destination.

They propose a simple checklist for anyone writing a new paper on Explainable AI. Before you publish, ask yourself:

1. Did you define your goal? (Who is this for, and what are they trying to do?)
2. Is your claim testable? (Can someone prove you wrong?)
3. Did you test for faithfulness? (Does it actually match the AI's brain?)
4. Is it useful? (Does it help someone make a real decision?)
5. Did you test it with humans? (Did real people use it in a real situation?)

The paper concludes that if we start focusing on these foundations, we can move from "cool science experiments" to "reliable tools" that actually help humans and machines work together. Until then, we might keep building flashlights that shine brightly but lead us in circles.