Technical Summary: DeepArrhythmia

Problem Definition

Beat-level Electrocardiography (ECG) arrhythmia detection aims to assign an arrhythmia class (Normal, Supraventricular Ectopic, Ventricular Ectopic, or Fusion) to each individual heartbeat within a recording. While existing automated systems have made substantial progress, they frequently treat beats as isolated local instances, utilizing short, beat-centered windows or isolated beat crops as input. This approach is fundamentally limited because beat-level classification is not purely local; the evidence required to classify a beat often extends beyond the target complex to the surrounding rhythm context.

For instance, distinguishing between Ventricular Ectopic Beats (VEBs) and Supraventricular Ectopic Beats (SVEBs) often requires analyzing timing (premature occurrence, compensatory pauses) and morphological consistency relative to neighboring beats, rather than just the local shape of the QRS complex. Current end-to-end models that process longer contexts often fail to explicitly ground their predictions in physiological anchors (e.g., R-peak locations, specific interval measurements), making it difficult to verify if a decision is supported by recognizable ECG cues. Furthermore, indiscriminately acquiring rich physiological evidence for all segments is inefficient; for easy segments, additional measurements add computational cost without improving accuracy, while for noisy signals, auxiliary evidence may introduce noise.

Methodology: DeepArrhythmia

DeepArrhythmia is a tool-grounded multimodal framework designed for segment-contextualized beat-level ECG classification. It operates on 10-second multi-beat ECG segments and integrates raw ECG signals, rendered waveform images, and structured physiological evidence through a central multimodal agent.

Core Architecture

The framework decouples physiological measurement from evidence integration using specialized tools:

1. Peak Detector: A 1D-UNet++ model localizes R-peaks to identify beat instances. These peaks serve as temporal anchors for aligning numerical features, morphology analysis, and the final structured output.
2. Feature Extractor: Generates explicit numerical rhythm and morphology measurements (e.g., RR intervals, normalized RR, R-peak amplitude, higher-order statistics, and numerical morphology descriptors) for each beat.
3. Morphology Analyzer: A vision-language model trained via knowledge distillation (using a teacher LLM) to generate textual morphology rationales based on beat-centered image crops and numerical features. This provides interpretable, morphology-focused textual evidence.
4. Central Multimodal Agent: A large language model (LLM) backbone (Qwen3.5-4B) that consumes signal tokens, image tokens, and serialized tool outputs to produce beat-level predictions.

Confidence-Guided Selective Evidence Acquisition

A key innovation is the Confidence Calculator, which acts as an evidence-gating tool. The system operates in two evidence states:

• Minimal Evidence State ($E_{min}$): Contains the signal, image, and beat anchors.
• Rich Evidence State ($E_{rich}$): Augments $E_{min}$ with numerical features and textual morphology analysis.

The system first generates an initial prediction under the minimal state. It then calculates a segment-level confidence score ($C_{seg}$) by averaging the maximum class probabilities of all beats in the segment.

• If $C_{seg} \geq \tau$ (a dataset-specific threshold), the system returns the minimal-evidence prediction.
• If $C_{seg} < \tau$, the system selectively invokes the Feature Extractor and Morphology Analyzer to acquire rich evidence before producing a refined prediction.

This routing policy balances the trade-off between under-evidenced decisions on difficult, rhythm-dependent segments and over-evidenced decisions where additional tools add cost or noise.

Training Strategy

DeepArrhythmia employs a two-stage masked supervised fine-tuning (SFT) process:

1. Stage 1 (Specialization): Two branch-specific agents are trained. One learns to predict under minimal evidence, and the other under rich evidence. This establishes the output syntax and alignment between beat anchors and class labels.
2. Stage 2 (Routing): A routed agent is trained on a held-out split to learn the confidence-gated acquisition policy. The model is trained to decide whether to stop after minimal evidence or to invoke additional tools based on the calculated confidence score.

Key Contributions

1. Segment-Contextualized Prediction: The authors reformulate beat-level classification as peak-aligned prediction within multi-beat segments, allowing decisions to combine local morphology with surrounding rhythm context (timing, compensatory pauses, cross-beat consistency).
2. Tool-Grounded Multimodal Agent: The framework separates physiological measurement from evidence integration. Specialized tools provide structured intermediate evidence (numerical measurements and textual descriptions) that serves both as input for prediction and as auditable evidence for interpretation.
3. Confidence-Guided Selective Acquisition: The paper proposes using model confidence as an evidence gate to selectively acquire richer physiological evidence only when necessary, addressing the non-uniform marginal value of such evidence across different segments.

Experimental Results

The framework was evaluated on four public datasets: MIT-BIH Arrhythmia, MIT-BIH Supraventricular, INCART, and VitalDB Arrhythmia, using subject-disjoint splits to ensure generalization.

• Performance: DeepArrhythmia variants achieved the best or tied-best performance across all datasets and metrics (Macro-F1 and Micro-F1) compared to conventional machine learning (SVM), deep learning (CNNs, Transformers, TCNs), and existing Vision-Language Models (VLMs).
• Contextual Value: The results demonstrate that segment-level context and structured physiological grounding significantly improve beat-level classification. The framework outperformed non-agentic fusion baselines, indicating that the model architecture plays a central role in translating heterogeneous evidence into accurate predictions.
• Selective Acquisition: The threshold-induced routed model consistently matched or exceeded the performance of a simple routing baseline while reducing reliance on uninformative evidence. Analysis showed that richer evidence primarily improved performance on low-confidence (difficult) segments, while high-confidence segments benefited less, validating the selective acquisition strategy.
• Interpretability: Human expert evaluations of the generated morphology analyses showed high scores for physiological correctness and clinical utility. The generated explanations were found to be faithful to the evidence used, as evidenced by targeted perturbation tests where disrupting cited cues (e.g., QRS width, RR intervals) significantly degraded accuracy.

Significance and Claims

The paper claims that DeepArrhythmia offers a practical alternative to always invoking all tools, achieving a favorable accuracy-efficiency trade-off. By decoupling measurement from integration, the framework produces predictions that are both more accurate and more interpretable, retaining intermediate evidence for error analysis and expert review.

The authors position DeepArrhythmia not as an autonomous diagnostic system, but as an assistive decision-support framework. The significance lies in operationalizing decades of progress in physiological signal processing (detectors, feature extractors) by integrating them into a unified reasoning loop orchestrated by a VLM. This approach allows for extensible domain knowledge integration while maintaining cross-modal coordination.

The paper acknowledges limitations, including:

• Performance heterogeneity across classes, particularly for rare fusion beats.
• Sensitivity to incomplete temporal context and low signal quality.
• Limited cross-dataset generalization, suggesting a need for robust uncertainty-aware routing in future work.
• The retrospective nature of the study, meaning prospective utility across diverse institutions and devices remains to be established.

The authors conclude that the framework should be used to assist, not replace, clinical judgment, with future work focusing on multicenter validation and prospective workflow evaluation.