Electrocardiogram Classification with Transformers Using Koopman and Wavelet Features

This paper demonstrates that while wavelet features excel in binary ECG classification, a transformer-based model utilizing Koopman operator features derived from an optimized Extended Dynamic Mode Decomposition (EDMD) with a radial basis function dictionary achieves superior performance in multi-class ECG classification, outperforming both wavelet-only and hybrid approaches.

The Gist

Imagine your heart is like a complex, rhythmic drum solo. Sometimes it plays a perfect beat, and sometimes it stumbles, skips, or goes off-key. Doctors use an ECG (electrocardiogram) to listen to this drum solo and diagnose problems. But listening to thousands of these recordings manually is exhausting, so scientists want computers to do it.

This paper is about teaching a super-smart computer (an AI) to listen to these heartbeats and figure out what's wrong, using two very different "languages" to describe the music.

The Two Languages of Heartbeats

The researchers tried two main ways to translate the heart's rhythm into data the computer could understand:

1. The Wavelet Translator (The "Microscope")
Think of a Wavelet as a high-powered microscope that looks at the heartbeat. It zooms in to see the tiny, sharp spikes and the smooth curves of the rhythm. It's great at spotting where and when a specific weird beat happens.

• The Result: This method was excellent at a simple job: telling the difference between a "Healthy Heart" and a "Sick Heart." It's like a security guard who is really good at spotting if someone is wearing a red shirt or a blue shirt.

2. The Koopman Translator (The "Orchestra Conductor")
The Koopman method is more abstract. Instead of just looking at the spikes, it tries to understand the physics of the rhythm. Imagine the heart rhythm as a complex dance. The Koopman method breaks this dance down into its fundamental "moves" or "modes" (like a spinning turn, a jump, or a slide). It treats the non-linear, messy heartbeat as if it were a set of simple, predictable waves.

• The Result: Initially, this method was a bit confused. But once the researchers tuned it perfectly (like tuning a guitar), it became a master at a harder job: distinguishing between four different types of heart problems (like telling the difference between a skipped beat, a fast beat, and a blocked beat).

The Big Experiment

The team combined these methods with a powerful AI model called a Transformer (the same type of brain behind modern chatbots). They set up four teams to compete:

1. Team Wavelet: Just the microscope data.
2. Team Koopman: Just the "dance move" data.
3. Team Hybrid: They tried to feed both the microscope and the dance moves into the AI at the same time, hoping for the best of both worlds.
4. Team Tuned Koopman: They took the Koopman method and carefully adjusted its settings (the "knobs" and "dials") to make it perfect.

What Happened?

• The Hybrid Surprise: You might think "Team Hybrid" would win because it had more information. But it actually did the worst. It's like trying to listen to a violin and a drum at the exact same time through a single earbud; the signals got confused and drowned each other out. The two methods were speaking slightly different dialects that didn't mix well.
• The Winner: Team Tuned Koopman took the gold medal. By carefully adjusting the settings, they taught the AI to see the underlying "dance" of the heart so clearly that it could distinguish between four different types of heart issues better than anyone else.
• The "Why": The researchers also built a "reconstruction" tool. They used the Koopman math to rebuild the heartbeat from scratch. The fact that the rebuilt heart looked almost identical to the real one proved that the AI had truly understood the heart's rhythm, not just memorized patterns.

The Takeaway

This paper teaches us a valuable lesson about AI and medicine: More data isn't always better.

Simply throwing every possible feature into a computer doesn't work. Sometimes, you need to find the right mathematical lens to view the problem. In this case, viewing the heartbeat as a set of mathematical "dance moves" (Koopman) and tuning that view perfectly allowed the AI to become a better doctor than the traditional methods, especially when the diagnosis gets complicated.

It's a step forward in making AI not just a pattern-matcher, but a true understanding of the physics behind our beating hearts.

Technical Summary

Here is a detailed technical summary of the paper "Electrocardiogram Classification with Transformers Using Koopman and Wavelet Features."

1. Problem Statement

Electrocardiogram (ECG) analysis is critical for diagnosing cardiac conditions, but automated classification remains challenging due to the high complexity, variability, and non-stationary nature of physiological signals.

• Limitations of Current Methods: Traditional handcrafted features lack robustness, while deep learning models (CNNs, RNNs) often struggle with interpretability and capturing long-range physiological dynamics.
• Research Gap: While Wavelet transforms (for time-frequency localization) and Koopman operator theory (for linearizing nonlinear dynamics) have been explored separately, their integration within Transformer architectures for ECG classification has not been systematically investigated.
• Objective: The authors aim to evaluate the efficacy of Koopman-based features (derived via Extended Dynamic Mode Decomposition, EDMD) combined with Transformers for ECG classification, comparing them against Wavelet-based approaches and hybrid models.

2. Methodology

Dataset and Preprocessing

• Data Source: MIMIC-IV-ECG database (large-scale, clinical ECG waveforms).
• Tasks:
  1. Binary Classification: Normal vs. Non-normal.
  2. Four-Class Classification: Normal, Atrial Fibrillation (AFib), Ventricular Arrhythmia, and Conduction Block.
• Preprocessing: Signals were resampled, z-score normalized, and segmented into fixed-length windows (2 seconds) with 50% overlap.

Feature Extraction Strategies

The study compares four distinct pipelines feeding into a Transformer encoder:

1. Wavelet + Transformer: Uses Discrete Wavelet Transform (DWT) to generate multiresolution time-frequency coefficients as input tokens.
2. Koopman + Transformer: Uses Extended Dynamic Mode Decomposition (EDMD) to approximate the Koopman operator.
   • Dictionary: Radial Basis Function (RBF) dictionary.
   • Features: Real/imaginary components, magnitudes, and growth rates of the Koopman eigenspectrum.
3. Hybrid (Wavelet + Koopman) + Transformer: Concatenates Wavelet and Koopman features before the Transformer.
4. Refined Koopman + Transformer: An ablation study where EDMD hyperparameters (delay embedding, RBF centers, kernel bandwidth, spectral truncation) are systematically tuned to optimize feature quality.

Model Architecture

• Backbone: Transformer Encoder.
• Baseline: A Recurrent Neural Network (RNN) trained on raw ECG waveforms for comparison.
• Reconstruction Analysis: The Koopman framework allows for signal reconstruction from dominant modes, used here to validate that the operator captures essential dynamics (P-QRS-T morphology) and to provide interpretability.

3. Key Contributions

1. Systematic Evaluation: First comprehensive study combining Koopman-based features with Transformers for ECG classification.
2. Benchmarking: Established strong baselines using Wavelet+Transformer and RNNs, clarifying the specific strengths of spectral (Wavelet) vs. dynamical (Koopman) features.
3. Methodological Improvement: Demonstrated that tuning the EDMD dictionary (specifically using RBFs with optimized parameters) significantly outperforms naive feature extraction, surpassing both Wavelet-only and Hybrid systems.
4. Interpretability: Provided a Koopman-based reconstruction analysis, linking learned features to physiological dynamics (e.g., identifying modes corresponding to QRS complexes and T-waves).

4. Experimental Results

Performance Metrics (F1 Score on Test Set):

Method	Binary Classification (F1)	Four-Class Classification (F1)
Wavelet + Transformer	0.750	0.700
Koopman + Transformer (Initial)	0.697	0.771
Hybrid (Wavelet + Koopman)	0.677	0.533
Koopman + Transformer (Refined)	0.786	0.764
RNN (Raw ECG Baseline)	0.782	0.700

Key Findings:

• Binary Task: Wavelet features initially outperformed Koopman features. However, the Refined Koopman approach achieved the highest score (0.786), surpassing the Wavelet baseline and the RNN.
• Multi-Class Task: Koopman features were inherently superior to Wavelets (0.771 vs. 0.700), suggesting that dynamical modes better distinguish between specific arrhythmia subtypes than time-frequency coefficients alone.
• Hybrid Failure: Simply concatenating Wavelet and Koopman features degraded performance in both tasks. The authors attribute this to redundancy or misalignment between the feature spaces, introducing noise rather than complementary information.
• Reconstruction: The EDMD-based reconstruction closely matched original ECG waveforms (including P-QRS-T morphology), with errors primarily occurring at sharp transitions (QRS peaks), validating the dynamical fidelity of the features.
• Efficiency: Feature-based methods (Wavelet/Koopman) were significantly more computationally efficient and scalable than the raw RNN baseline.

5. Significance and Conclusion

• Dynamical Systems in AI: The paper successfully bridges dynamical systems theory (Koopman operator) with modern deep learning (Transformers), showing that linearizing nonlinear physiological data in a high-dimensional feature space is highly effective for classification.
• Importance of Feature Engineering: The results highlight that the success of Koopman methods is highly sensitive to the choice of basis functions (dictionary) and hyperparameters. A "black box" application of EDMD is insufficient; principled tuning is required.
• Future Directions: The study suggests that naive feature concatenation is insufficient for hybrid models; future work should focus on learned fusion mechanisms (e.g., attention-based gating). The framework is also applicable to other biomedical signals like EEG and PPG.

In summary, this work demonstrates that Koopman-based feature learning, when optimized via EDMD, offers a superior and interpretable alternative to traditional spectral methods for complex, multi-class ECG classification tasks.