Software Frameworks for Explainable AI in Time Series Classification: A Systematic Review
This systematic review analyzes existing software frameworks for explainable AI in time series classification, revealing significant fragmentation and limitations in frequency-domain support, specialized evaluation metrics, and cross-framework consistency, while advocating for the development of unified, time-series-specific XAI solutions.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern world, machines are increasingly asked to make sense of data that flows over time, such as the rhythm of a heartbeat, the fluctuation of stock prices, or the sound of a spoken word. This type of data, known as a time series, is everywhere in healthcare, industry, and finance. To predict what happens next, scientists often use powerful computer programs called deep learning models. These programs are excellent at finding patterns and making accurate predictions, but they operate like black boxes: they produce an answer without explaining how they reached it. In high-stakes situations, such as diagnosing a patient or approving a loan, knowing the reason behind a decision is just as important as the decision itself. This need for clarity has given rise to a field called explainable artificial intelligence, which aims to open the black box and show the human user which parts of the data influenced the machine's choice.
A team of researchers recently set out to examine the tools available to help scientists build these explanations specifically for time-based data. They conducted a systematic review of the software frameworks—essentially the toolkits and libraries that programmers use to generate and test these explanations. While many tools exist for explaining decisions based on images, the researchers found that the landscape for time series is fragmented and still developing. They searched through thousands of open-source projects, filtering them down to six major frameworks that claim to support time series data. Their goal was to understand what these tools can actually do, how well they work, and whether they are ready for real-world use.
The investigation revealed a significant gap between the complexity of time series data and the capabilities of the current software. While the six frameworks identified offer a wide array of methods to generate explanations, very few of these methods were actually designed with time series in mind. Most of the tools simply take methods created for images or other data types and apply them to time-based signals without specific adaptation. The researchers found that only a small fraction of the available methods explicitly account for the unique nature of time, such as the fact that past events influence future ones, or that different sensors in a single device often depend on each other. For instance, in a medical context, an electrocardiogram signal from one part of the heart is deeply connected to signals from other parts; current tools often struggle to explain these complex, simultaneous relationships across multiple channels.
A particularly striking limitation discovered was the lack of support for analyzing data in the frequency domain. Time series data can be understood not just by looking at how it changes over seconds or minutes, but also by looking at the underlying frequencies, much like how a musical chord can be broken down into individual notes. In many tasks, such as distinguishing between a male and a female voice or identifying specific types of heart conditions, the crucial clues often hide in these frequency patterns rather than in the raw signal itself. The review found that out of all the methods available in the six frameworks, only one was capable of generating explanations in the frequency domain. This means that for many important applications, the tools are currently blind to the very patterns that might be most critical for understanding the decision.
The researchers also tested whether these tools produce consistent results. They took the same mathematical approach to explaining a model's decision and ran it through two different software frameworks. Surprisingly, the results were not the same. In one case, using a specific framework, the explanation failed to highlight the characteristic pattern that a medical expert would expect to see, while the other framework correctly identified it. This suggests that the choice of software tool can fundamentally change the explanation a user receives, even when the underlying method appears to be the same. This inconsistency poses a serious challenge for reliability, as it implies that two scientists could reach different conclusions about why a machine made a decision simply by using different software packages.
Furthermore, the study highlighted a shortage of reliable ways to test these explanations. To know if an explanation is good, researchers need to compare it against a known truth, a concept called ground truth. The review found that most of the available datasets used for testing do not include this ground truth information, making it difficult to verify if the explanations are actually correct. Only two specific metrics were found that were designed specifically for time series data, while the vast majority of testing tools were borrowed from other fields. This lack of specialized testing tools means that the quality of explanations for time-based data is often assumed rather than proven.
Ultimately, the paper concludes that while the community has made progress in creating software for explaining time series decisions, the current tools are not yet fully ready for the job. They rely heavily on generic methods that were not built for the unique challenges of time-based data, such as cross-channel dependencies and frequency patterns. The researchers argue that the field needs unified, purpose-built frameworks that can generate faithful, reproducible, and time-aware explanations. Until such tools are developed and standardized, users must proceed with caution, understanding that the explanations provided by current software may be incomplete or inconsistent, and that the reliability of these insights depends heavily on the specific tool chosen.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.