🤖 machine learning

Concept Extraction for Time Series with ECLAD-ts

This paper introduces ECLAD-ts, a novel post-hoc concept extraction and localization method tailored for time series classification that generates global explanations by clustering CNN activation maps to reveal how models encode input subsets at different abstraction levels.

Original authors: Antonia Holzapfel, Andres Felipe Posada-Moreno, Sebastian Trimpe

Published 2026-08-03

📖 9 min read🧠 Deep dive

CC BY 4.0

Original authors: Antonia Holzapfel, Andres Felipe Posada-Moreno, Sebastian Trimpe

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but your only witness is a brilliant, super-fast robot that refuses to speak. This robot can look at a long, wiggly line of data and tell you exactly what is happening—like predicting if a machine will break or if a patient is sick—but it won't tell you how it figured it out. In the world of science, this is called a "black box." We know the robot is smart, but because we can't see its thought process, we don't know if it's using real clues or if it's just relying on tiny, accidental patterns that happen to look like the answer. This is a big problem because if the robot is relying on these patterns, it might make dangerous mistakes when things change. Scientists have been trying to build "flashlights" to shine into these black boxes, especially for pictures, but time-based data (like heartbeats or stock prices) has been much harder to crack because it moves and changes in ways pictures don't.

This paper introduces a new flashlight called ECLAD-ts, designed specifically to peek inside the brains of robots that analyze time series. The researchers built this tool to find "concepts"—which are just the specific patterns or shapes the robot has learned to recognize. Think of it like teaching a child to identify a dog: you don't just say "it's a dog"; you point out the floppy ears, the wagging tail, and the wet nose. ECLAD-ts does this for robots, but instead of ears and tails, it finds wiggles, bumps, and slopes in the data. The team tested their new tool on both made-up data (where they knew the answer beforehand) and real-world data from factories and motion sensors. They found that ECLAD-ts is much better at spotting the real clues the robot is using and ignoring the "shortcuts" or accidental patterns. It even figured out that some robots were ignoring the actual important parts of the data and just guessing based on the background noise. By showing us exactly which parts of the timeline the robot is looking at, ECLAD-ts helps us trust the robot more, or fix it if it's looking at the wrong things.

The Problem: The Silent, Shortcut-Using Robot

Imagine you are training a robot to tell the difference between a "good" day at a factory and a "bad" day. You feed it thousands of recordings of machine vibrations. The robot gets really good at its job, getting 100% right on the tests. But then, you ask it, "Why did you think that machine was broken?" The robot just stares back. It's a black box.

The scary part is that robots are notorious for taking shortcuts. Instead of learning that a machine is broken because of a specific crunching sound in the vibration, the robot might learn that "bad days always happen on Tuesdays" or "bad days always have a little bit of static noise at the very beginning." If the robot relies on these shortcuts, it will fail miserably the moment the schedule changes or the static noise disappears. This is called "shortcut learning," and it's dangerous because the robot looks smart but isn't actually smart.

To fix this, scientists need to open the black box. They need to see what the robot is actually "thinking." For pictures, we have tools that can highlight the part of the image the robot is looking at. But for time series—data that flows over time like a heartbeat or a stock chart—these tools were missing or didn't work well. Existing methods were like trying to find a needle in a haystack by just guessing where the needle might be, or they required humans to manually label every single pattern, which is slow and boring.

The Solution: ECLAD-ts, the Time-Traveling Detective

The authors of this paper, Antonia Holzapfel and her team, created a new tool called ECLAD-ts. Think of this tool as a detective that can freeze time and zoom in on specific moments in a data stream to see what the robot is paying attention to.

Here is how it works, using a simple analogy:
Imagine the robot's brain is a giant library of notes. Every time the robot looks at a piece of data, it writes down a note about what it sees. ECLAD-ts goes into this library and groups similar notes together. These groups are called concepts.

Concept 1 might be "a sharp spike in the middle."
Concept 2 might be "a slow, gentle slope at the start."
Concept 3 might be "random noise."

The magic of ECLAD-ts is that it doesn't just find these concepts; it tells you how important they are. It asks the robot, "If I remove this 'sharp spike' from the data, does your answer change?" If the answer changes a lot, the concept is very important. If the answer doesn't change, the concept is just background noise.

Crucially, this tool is designed for multivariate time series. This means the data often comes from multiple sensors at once (like a car having a speed sensor, a temperature sensor, and a pressure sensor all recording at the same time). ECLAD-ts is special because it can tell you which sensor the robot is looking at. It can say, "The robot is looking at the 'sharp spike' on the temperature sensor, but it's ignoring the speed sensor." Other tools often mix all the sensors together, making it impossible to tell which one matters.

The Experiments: Testing the Detective

To prove their tool works, the team ran a series of tests.

1. The Fake Data Test (The "Ground Truth" Check)
First, they created three fake datasets where they knew exactly what the answer was.

SyntheticL2: They made data where one class had a specific "bump" and the other didn't. They also added a "background" pattern that looked like a square wave.
SyntheticL4: They made data with an "upward bump" for one class and a "downward bump" for another.
SyntheticLm: This was the tricky one. They had two channels (sensors). One channel had a "bump" for Class A, and the other channel had a "bump" for Class B.

They trained robots on these fake datasets and then used ECLAD-ts (and other tools) to see what the robots learned.

The Result: ECLAD-ts correctly identified the "bumps" as the important concepts. It even spotted that some robots were relying on shortcuts! In one case, the robot learned to ignore the "bump" and just look at the "background" to guess the answer. ECLAD-ts caught this and gave the background a high importance score, revealing the robot's shortcut.
The Comparison: They compared ECLAD-ts to other tools like ConceptShap and MultiVISION. These other tools often failed. They either couldn't find the specific "bumps" or they got confused by the background noise. ECLAD-ts was the only one that could clearly point to the right spot in time and on the right sensor.

2. The Real World Test
Next, they tried it on real data.

GunPoint Dataset: This data records the hand movements of actors pretending to hold a gun. The key features are a specific "overshoot" motion (when lowering the arm) and an "extra lift" (when raising it). ECLAD-ts successfully highlighted these exact movements. The other tools just pointed to broad, vague areas of the timeline.
P2S Dataset: This is data from a metal stamping factory. The goal is to tell if a production run is normal or faulty. Experts know the important clues are on the "slopes" before the curve in the middle of the graph. ECLAD-ts found these slopes and gave them high importance scores. It even looked at a mistake the robot made and showed that the robot had confused a slope from a "good" run with a slope from a "bad" run. This helps humans understand why the robot failed.

What They Found (And What They Didn't)

The paper shows that ECLAD-ts is a powerful new way to understand how robots think about time.

It works: The tool successfully extracted meaningful patterns (concepts) from the robot's brain that matched what humans knew was important.
It catches shortcuts: It can detect when a robot is using shortcuts or looking at the wrong things (like background noise instead of the actual signal).
It handles multiple sensors: It is the first tool that can clearly separate which sensor is important for a specific pattern, which is a huge deal for complex data.
It's better than the rest: In their tests, ECLAD-ts consistently outperformed the other tools (ECLAD, ConceptShap, and MultiVISION) in finding the right patterns and assigning the right importance to them.

However, the authors are careful not to say this is a perfect, finished product. They note that:

It needs tuning: The tool has settings (like how many concepts to look for) that need to be adjusted. If you pick the wrong number, the results might not be as clear.
It's still a simulation in some ways: While they tested it on real data, the "ground truth" (knowing the absolute right answer) was only available in the fake datasets. In the real world, we can't be 100% sure if the robot is looking at the only right thing, but ECLAD-ts gives us the best view we have.
It's for CNNs: This tool is designed for a specific type of robot brain called a Convolutional Neural Network (CNN). It might not work for other types of robots without changes.

Why This Matters

This paper doesn't just give us a new tool; it gives us a way to build trust. If a robot is diagnosing a patient or controlling a factory, we need to know it's not just guessing. ECLAD-ts acts like a translator, turning the robot's silent, complex thoughts into a clear story: "I am predicting 'faulty' because I see this specific slope on the temperature sensor, and I am ignoring the noise on the pressure sensor."

By making these black boxes a little less black, the authors hope that engineers and doctors can build better, safer, and more reliable systems. They aren't claiming to have solved every problem in robot intelligence, but they have definitely turned on a very bright light in a very dark room.

Technical Summary: Concept Extraction for Time Series with ECLAD-ts

Problem Statement

Convolutional Neural Networks (CNNs) are increasingly deployed for Time Series Classification (TSC) in critical domains such as quality prediction and medical diagnosis. However, their "black box" nature hinders the understanding of their prediction processes, making it difficult to detect if models rely on harmful biases, spurious correlations, or "shortcut learning" rather than task-relevant features. While Concept Extraction (CE) methods exist to provide human-understandable explanations by identifying patterns (concepts) a model uses, existing approaches are predominantly designed for the image domain.

In the time series domain, current explanation methods face significant limitations:

Prototype-based methods do not reflect how the model encodes information internally.
Concept testing methods (e.g., TCAV) require manual labeling of concepts, introducing human bias and labor, and may not align with the concepts the model actually learned.
Existing CE methods for time series (e.g., MultiVISION) often rely on the receptive field of network layers. In deep architectures with large receptive fields (e.g., InceptionTime), this can result in concept patches covering the entire input, rendering the "clustering" ineffective.
Image-based CE methods cannot be directly transferred due to domain-specific characteristics, such as multi-channel data where different channels encode distinct types of information (e.g., different sensors).

There is a critical gap for an automatic, global CE method tailored to time series that can localize concepts both temporally and across channels without relying on manual annotation or fixed receptive fields.

Methodology: ECLAD-ts

The authors propose ECLAD-ts, a post-hoc global explanation method adapted from the image-domain algorithm ECLAD. It extracts and localizes concepts by analyzing the internal representations of CNNs. The method operates in three primary steps:

1. Encoding of the Latent Space (Local Aggregated Descriptors)

Instead of relying on fixed receptive fields, ECLAD-ts utilizes Local Aggregated Descriptors (LADs).

The method aggregates activation maps from a predefined set of CNN layers ( $L$ ).
These maps are up-scaled via linear interpolation to match the input dimensions ( $w \times ch$ ).
A descriptor $d_{x_i}$ is formed by concatenating these up-scaled maps.
A LAD is defined as the timestep-wise slice of this descriptor ( $d_{x_i}(b)$ ), representing how the model encodes a specific time step at various levels of abstraction. This approach reduces dimensionality while preserving temporal and channel-wise information.

2. Mining of Patterns and Visualization

Clustering: A mini-batch k-means algorithm is applied to the LADs extracted from a set of inputs to identify clusters of similarly encoded time subsequences. The centroids of these clusters ( $\gamma_{c_j}$ ) define the concepts.
Localization: To visualize a concept $c_j$ within an input $x_i$ , a mask $m_{x_i}^{c_j}$ is generated. This mask identifies timesteps where the LAD is closest to the concept centroid.
Channel Expansion: Unlike image-based methods, time series often have multiple channels with distinct information. ECLAD-ts expands the single-channel mask to match the input's channel count, creating separate masks for each channel to enable channel-wise localization.

3. Relevance Computation (Importance Scoring)

A novel importance scoring mechanism is introduced to quantify the relevance of concepts, specifically addressing the multi-channel nature of time series.

Instance-wise Sensitivity: The method computes the gradient-based sensitivity of the model output with respect to the input, masked by the concept's location: $R_{x_i}^{c_j} = \nabla_x g(f(x_i)) \odot m_{x_i}^{c_j}$ .
Wrapper Function: For multiclass classification, the wrapper $g(y)$ is defined as the squared norm of the difference between the output vector and its mean, ensuring the metric captures the model's confidence in distinguishing classes.
Sign Preservation: The importance score aggregates sensitivity over timesteps while preserving the sign of the gradient.
- A positive score indicates the concept helps distinguish the predicted class.
- A negative score indicates the concept reduces the distance between logits (making the model "less sure"), representing features that contradict the prediction.
- A score near zero implies the concept is unimportant.
Aggregation: The final Importance Score (IS) per channel is the mean relevance of the concept across the dataset, scaled to the range $[-1, 1]$ .

Key Contributions

ECLAD-ts Algorithm: A novel method for automatic concept extraction and localization in the time series domain that captures both temporal and channel-wise information, overcoming the limitations of receptive-field-based methods.
Channel-Wise Importance Scoring: A modified metric that quantitatively assesses the relevance of concepts across specific channels, accounting for the fact that different sensors may encode different types of information.
Empirical Evaluation: Extensive experiments on synthetic and natural datasets comparing ECLAD-ts against vanilla ECLAD, ConceptShap, and MultiVISION.

Results

The method was evaluated on three synthetic datasets (with known ground-truth primitives) and two natural datasets (GunPoint and Production Press Sensor Data).

Synthetic Datasets

Localization Accuracy: ECLAD-ts successfully localized concepts corresponding to the ground-truth primitives (e.g., specific bumps or background interruptions) in both time and channel dimensions.
Channel Differentiation: In the multi-channel synthetic dataset ( $syntheticL_m$ ), ECLAD-ts was the only method capable of distinguishing that primitive $p_0$ belonged to channel 0 and $p_1$ to channel 1. Other methods (ECLAD, ConceptShap, MultiVISION) failed to differentiate channels or produced ambiguous results.
Metrics: ECLAD-ts achieved the highest Representation Correctness (RC) and Importance Correctness (IC) scores across all models (InceptionTime, ResNet18, DenseNet121).
Shortcut Learning Detection: The method successfully identified when models relied on "uninterrupted background" (a shortcut) rather than the intended primitive features, evidenced by high importance scores assigned to background concepts.

Natural Datasets

GunPoint Dataset: ECLAD-ts correctly identified the "overshoot" and "extra lifting" motions known to be critical for classification, whereas other methods produced broad, inconclusive masks.
P2S Dataset (Industrial): The method localized features on the slopes before concavity in production sensor data, aligning with expert knowledge. It also provided insights into misclassified samples by highlighting conflicting features (e.g., a slope resembling the wrong class).

Significance and Claims

The paper claims that ECLAD-ts represents a significant step toward enhancing the transparency of CNNs for time series classification. Its primary significance lies in:

Bridging the Domain Gap: Successfully adapting image-based CE concepts to the specific challenges of time series (dimensionality and channel-specific information).
Automatic and Global: Providing explanations without the need for manual concept labeling or human expertise in the application domain.
Robustness to Architecture: Unlike methods dependent on receptive fields, ECLAD-ts remains effective even in deep architectures with large receptive fields.
Actionable Insights: By revealing whether a model uses relevant features or shortcuts, the method allows stakeholders to assess model alignment with human expectations, improve model robustness, and build appropriate trust.

The authors note that while ECLAD-ts is efficient and scalable, challenges remain regarding the visualization of complex features and sensitivity to hyperparameters (specifically the number of concepts). They suggest future work could focus on adaptive techniques and the integration of CE into real-time training.