🤖 machine learning

CENDRe: Concept Extraction with Natural Domain Representations

CENDRe is a novel concept extraction method for CNNs that overcomes existing limitations by automatically discovering the optimal number of time-frequency concepts through silhouette-guided clustering and gradient-based localization, thereby providing interpretable evidence for critical time-series classification tasks like fault diagnosis.

Original authors: Antonia Holzapfel, Andres Felipe Posada Moreno, Sebastian Trimpe

Published 2026-08-03

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Antonia Holzapfel, Andres Felipe Posada Moreno, Sebastian Trimpe

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize different sounds, like a dog barking versus a car horn. You feed it thousands of audio clips, and the robot, using a special kind of brain called a Convolutional Neural Network (CNN), gets really good at guessing the right answer. But here's the catch: the robot is a "black box." It gives you the answer, but it won't tell you why. Did it hear the bark because of the pitch? The rhythm? The sudden start? In the real world, where mistakes can be dangerous—like a doctor diagnosing a heart condition or a factory predicting a machine failure—we can't just trust the robot's guess. We need to peek inside its brain to see what patterns it's actually looking for. This is the world of "Explainable AI" (XAI). Scientists have developed methods to pull these patterns out, but until now, they've been like flashlights that only shine in one direction. They could show you when a sound happened (the time domain), but they were blind to the pitch or frequency of the sound, which is often the most important clue.

This paper introduces a new tool called CENDRe (Concept Extraction with Natural Domain Representations) that fixes these blind spots. Think of CENDRe as a detective that doesn't just look at the timeline of an event but also listens to the musical notes being played. The researchers found that old methods had three main problems: they only looked at time, they forced the robot to guess a specific number of patterns (like saying "there are exactly 3 clues" before even looking), and they sometimes pointed to the wrong parts of the signal. CENDRe solves this by automatically figuring out how many patterns exist, finding them in both time and frequency, and pinpointing exactly where in the signal those patterns live. When they tested it on fake data with known answers, CENDRe found the right clues much better than the old methods. When they tried it on real data from broken machine bearings, it successfully identified the specific "humming" frequencies that experts use to diagnose the problem, proving that the robot isn't just guessing randomly but is actually listening to the right things.

The Detective's New Toolkit

Imagine you are a detective trying to solve a mystery by listening to a recording of a noisy street. You have a super-smart AI assistant that tells you, "I know it's a car crash!" But you need to know how it knows. Is it the screech of tires? The crunch of metal? Or maybe just the sudden silence?

Old detective tools (the previous AI methods) were a bit clumsy. First, they only listened to the timing of the sounds. If the crash happened at 2:03 PM, they could tell you that, but they couldn't tell you if the sound was a high-pitched squeal or a low rumble. Second, they were stubborn about the number of clues. You had to tell them, "Look for exactly 3 clues," before they started. If the real mystery had 4 clues, they would miss one or mash two together. Third, they were bad at pointing. They might say, "The clue is somewhere in this whole minute," when the actual clue was just a tiny split-second sound.

CENDRe is like a detective who brings a new set of glasses. These glasses let them see the sound in two ways at once: as a timeline (when things happen) and as a spectrum (what pitches are present).

1. The Magic of "Natural" Glasses

The paper introduces the idea of "Natural Domain Representations." Imagine looking at a painting. You can look at it from the front (time), but you can also look at it through a special filter that shows you the brushstrokes (frequency). CENDRe uses a mathematical trick called a "virtual inspection layer." It's like inserting a transparent, reversible filter into the AI's brain. The AI doesn't even know the filter is there; it just keeps doing its job. But CENDRe can look at the AI's thoughts through the filter. This allows the detective to see the "frequency" of the sound—like the specific hum of a broken bearing—without changing how the AI thinks.

2. Letting the Clues Count Themselves

One of the biggest headaches for old detectives was deciding how many clues to look for. If you told them to find 5 clues, but there were only 3 real ones, they would invent fake clues to fill the gap. If there were 7, they would miss the last two.

CENDRe uses a clever trick called silhouette-guided aggregation. Imagine you have a pile of mixed-up puzzle pieces. Instead of guessing how many pictures are in the pile, CENDRe starts by grouping similar pieces together into tiny clusters. Then, it asks a question: "How well do these groups fit together?" It uses a score called a "silhouette score" (think of it as a "happiness score" for the groups) to decide when to stop merging. If the groups are happy and distinct, it stops. This means CENDRe automatically figures out, "Ah, there are actually 4 distinct patterns here," without you having to guess. It replaces the human guess with a math-based discovery.

3. Pinpointing the Exact Moment

Once CENDRe finds a pattern (a "concept"), it needs to show you exactly where it is in the sound. Old methods often drew big, fuzzy boxes around the whole sound, saying, "The clue is somewhere in here!" CENDRe is much more precise. It uses a technique called gradients to trace the path of the clue back to the source.

Think of it like a heat map. If you turn up the volume on a specific part of the sound, does the AI's confidence in its answer go up? CENDRe calculates this for every single moment in time and every single frequency. It creates a mask that highlights only the parts of the sound that matter. If the AI is listening to a high-pitched squeal at 2000 Hz for 0.5 seconds, CENDRe draws a tight, bright box around exactly that spot, ignoring the rest of the noise.

What They Found

The researchers tested CENDRe in two ways: with made-up data where they knew the answers, and with real data from broken machines.

The Fake Data Test:
They created synthetic sounds with hidden "primitives" (like a square wave or a specific frequency band) that the AI was trained to recognize.

Time Domain: When looking for time-based shapes (like a square pulse), CENDRe found them just as well as the best existing tools.
Frequency Domain: This is where CENDRe shined. When the clues were hidden in specific frequency bands (like a high-pitched tone), the old tools were completely blind. They couldn't see the frequency clues at all. CENDRe, however, found them perfectly. It localized the exact frequency bands the AI was using, matching the "ground truth" (the real answer) almost perfectly.
Importance: CENDRe didn't just find the clues; it correctly ranked them. It knew which clues were the most important for the AI's decision, whereas other methods sometimes got confused about which clues mattered.

The Real-World Test:
They took the tool to a real factory setting, looking at data from bearing faults (broken parts in machines).

The AI was trained to tell the difference between a healthy bearing and one with a crack on the inside or outside.
CENDRe extracted concepts that aligned with what human experts know. For example, it found that the AI was focusing on a sharp peak around 2000 Hz on one sensor and specific bands around 800–1000 Hz on another.
These aren't random numbers; they are the exact "signature" frequencies that engineers use to diagnose broken bearings. This proves that CENDRe isn't just making up patterns; it's revealing the actual logic the AI is using to make its predictions.

Why This Matters

The paper suggests that by combining time and frequency views, and by letting the AI tell us how many patterns it sees, we can trust these black-box models more. In critical fields like healthcare or industrial safety, we can't afford to have an AI guess correctly for the wrong reasons. CENDRe gives us a way to verify that the AI is looking at the right "clues"—whether that's a specific heartbeat rhythm or a specific machine vibration—before we let it make life-or-death decisions.

The authors are careful to note that while CENDRe works great on the types of AI models they tested (1D CNNs), it relies on the AI having a specific structure (translation equivariance). They suggest that future work will need to adapt this for other types of AI, like Transformers, which are becoming very popular. But for now, CENDRe offers a powerful new lens to see inside the machine's mind, turning a mysterious guess into a clear, explainable story.

Technical Summary: CENDRe (Concept Extraction with Natural Domain Representations)

Problem Statement

Convolutional Neural Networks (CNNs) are widely deployed for time-series classification in critical domains such as industrial condition monitoring and healthcare. However, their "black box" nature necessitates explanations that are both faithful to the model's behavior and interpretable by domain stakeholders. Existing Concept Extraction (CE) methods for time series suffer from three primary limitations:

Domain Restriction: They operate exclusively in the time domain, failing to explain frequency-domain features which are often critical for signal interpretation (e.g., fault diagnosis).
Manual Hyperparameterization: They require the number of concepts ( $K$ ) to be fixed manually, introducing user bias and requiring exhaustive sweeps to find optimal results.
Misaligned Localization: Their localization methods often produce masks that either miss the specific regions the model relies on or cover irrelevant regions, failing to concentrate attribution on the driving evidence.

Methodology: CENDRe

CENDRe is a post-hoc concept extraction method designed for time-series CNNs that extracts and visualizes concepts across any "natural domain" (any invertible transform of the input where signal content is directly interpretable, such as time or frequency). The method proceeds in three stages:

1. Latent Representation via Local Aggregated Descriptors (LADs)

To capture patterns at multiple levels of abstraction, CENDRe constructs a per-timestep descriptor. It selects a subset of internal layers as probe layers, linearly upsamples their activation maps to a common length, and concatenates them. This results in a timestep-wise latent vector $z_t(x)$ that leverages the approximate translation equivariance of CNNs, ensuring similar local patterns yield similar descriptors regardless of position.

2. Concept Discovery via Clustering

CENDRe discovers concepts by clustering the LADs in two stages to handle scalability:

Micro-clustering: A mini-batch k-means algorithm compresses the dataset of LADs into $J$ micro-centroids.
Concept Aggregation: These micro-centroids are clustered into $K$ concepts.
Automatic $K$ Selection: Unlike prior work, CENDRe does not require a fixed $K$ . It employs silhouette-guided hierarchical aggregation (or density-based clustering like HDBSCAN) to automatically select the number of concepts that maximizes the average silhouette score. This replaces the user-defined $K$ with the micro-cluster count $J$ , which the method is less sensitive to.

3. Cross-Domain Localization and Scoring

Contrastive Presence Score: For each concept, a presence score is calculated by contrasting the similarity of a LAD to its assigned micro-centroid against the mean similarity to micro-centroids of other concepts.
Gradient-Based Localization: To localize concepts, CENDRe computes gradients of the presence score with respect to the input.
Virtual Inspection Layers: To extend these localizations to other domains (e.g., frequency), CENDRe inserts a differentiable, invertible transform (such as the Fast Fourier Transform) into the computational graph. By backpropagating gradients through this transform, it generates concept masks in the frequency domain without retraining the model.
Importance Scoring: A relevance score is computed by measuring the overlap between the concept mask and the model's sensitivity (gradient) for a specific class, quantifying how much a concept drives a prediction.

Key Contributions

First Cross-Domain CE: CENDRe is the first CE method to extract and visualize concepts in both time and frequency domains simultaneously, addressing the limitation of time-only explanations.
Domain-Transferable Localization: It introduces a mechanism to backpropagate gradients of a contrastive presence score through differentiable invertible transforms, yielding concept masks in any natural domain.
Automatic Concept Number Selection: It proposes a procedure to automatically identify the number of relevant concepts ( $K$ ) from the latent structure using silhouette scores, eliminating the need for manual sweeps and reducing inductive bias.
Comprehensive Evaluation: The method is evaluated on synthetic benchmarks with ground-truth concepts in both domains and on real-world bearing-fault datasets.

Experimental Results

The authors evaluated CENDRe on synthetic datasets (SYNTHETICLOCAL and SYNTHETICFREQUENCY) and real-world bearing-fault datasets (CWRU and BearingPD).

Synthetic Time-Domain (SYNTHETICLOCAL): CENDRe achieved representation correctness (sRC) comparable to the state-of-the-art ECLAD-ts but demonstrated significantly higher importance correctness (sIC). This indicates that while ECLAD-ts might cover the full primitive, CENDRe's masks better align with the specific regions the model deems important at each timestep. MultiVISION performed poorly on both metrics.
Synthetic Frequency-Domain (SYNTHETICFREQUENCY): Since baselines lack frequency-domain capabilities, only CENDRe variants were tested. They successfully localized concepts to the ground-truth frequency bands, achieving high sRC and sIC. Time-domain masks for these datasets showed no interpretable patterns, whereas frequency masks clearly isolated the discriminative bands.
Real-World Bearing Faults (CWRU & BearingPD): On real data, CENDRe extracted concepts that separated cleanly by class. Crucially, the frequency-domain masks identified characteristic fault frequencies (e.g., specific bands around 2000 Hz or structural resonance regions) that domain experts use for diagnosis. This provides evidence that the model is reasoning based on physically meaningful signal features rather than spurious correlations.

Significance and Claims

The paper claims that CENDRe addresses the practical limitations of existing time-series CE methods by:

Expanding Interpretability: Moving beyond the time domain to include frequency, which is essential for signal processing applications.
Reducing User Bias: Automating the selection of the number of concepts, ensuring the explanation reflects the model's learned structure rather than user expectations.
Improving Fidelity: Producing localized masks that concentrate on the specific regions driving the concept, leading to higher importance correctness.

The authors conclude that CENDRe enables domain experts to verify that models are making predictions based on plausible signal features (e.g., specific fault frequencies) rather than artifacts. While the current implementation relies on the translation equivariance of 1D CNNs, the framework is presented as a pipeline applicable to other invertible representations (like time-frequency distributions) and future architectures.