Towards Dys-XAI: Influence-Based Explanations for Dysarthria Severity Assessment

Technical Summary: Towards Dys-XAI: Influence-Based Explanations for Dysarthria Severity Assessment

Problem Statement
Dysarthria severity assessment is critical for clinical therapy planning and longitudinal monitoring but currently relies on manual perceptual rating, which is time-consuming and suffers from inter-rater variability. While deep learning models have achieved performance comparable to expert ratings on public datasets, their "black-box" nature hinders clinical adoption. Existing Explainable AI (XAI) methods for speech, such as feature attribution (e.g., SHAP, LIME) or attention visualizations, typically generate acoustic feature importance scores or spectrogram heatmaps. The authors argue these are insufficient for clinical utility because:

1. They are local and utterance-level, failing to capture the ordinal nature of severity labels (where adjacent categories represent incremental differences rather than nominal classes).
2. They rely on engineered acoustic descriptors or time-frequency regions that are difficult for clinicians to translate into perceptual constructs (e.g., consonant precision, breathiness) used in clinical reasoning.

Methodology
The paper proposes Dys-XAI, an influence-based, instance-level explainability framework that shifts the explanation perspective from acoustic features to training instances. Instead of asking which features were important, the framework asks which training samples exerted supportive or opposing influence on a specific prediction.

• Task Formulation: The task is defined as a 4-class ordinal classification (Typical, Mild, Moderate, Severe) using the TORGO dataset.
• Instance-Level Influence Computation: The authors employ a gradient-based influence estimation method. By tracking checkpoints $\{\theta_1, \dots, \theta_M\}$ saved during training, they compute the influence of a training sample $x_i$ on a test sample $x_t$ as the cumulative inner product of their gradients:
  $$I_{i \to t} = \sum_{m=1}^{M} \nabla_\theta \ell(f_{\theta_m}(x_t), y_t)^\top \nabla_\theta \ell(f_{\theta_m}(x_i), y_i)$$
  A positive inner product indicates supportive influence (training on $x_i$ reduces loss on $x_t$), while a negative product indicates counteracting influence.
• Aggregation Strategies:
  1. Class-Level Aggregation: Influence scores are aggregated by the true severity labels of test samples to quantify supportive vs. competing contributions across the ordinal scale. This produces a $4 \times 4$ influence matrix ($S_{c \leftarrow c'}$) showing how training samples of severity $c'$ influence predictions for severity $c$.
  2. Model-Level Ordinal Sensitivity: Influence is aggregated by ordinal distance ($d = |c - c'|$) to assess whether the model respects the ordinal structure (i.e., if influence decays as the distance between severity levels increases).
• Validation: The framework is validated via controlled deletion experiments. The authors selectively remove high-influence or low-influence training samples (5–20%) and retrain the model to observe systematic shifts in predictions.

Key Results

• Validation of Influence Scores: Controlled deletion experiments confirmed the faithfulness of the influence scores.
  • High-Influence Removal: Removing the top 20% of highly influential samples caused severe performance degradation (e.g., accuracy for Moderate severity dropped from 36.8% to 0.3%; Severe dropped from 72.3% to 43.1%).
  • Low-Influence Removal: Removing the bottom 20% of samples (low influence) improved performance, particularly for ambiguous classes (Mild accuracy tripled from 5.5% to 17.6%), suggesting these samples contained noise or mislabeled data.
• Cross-Severity Influence Patterns: The class-level influence matrix revealed a two-tier structure:
  • Binary Distinction: Typical speech is isolated; it receives positive influence only from other typical samples and negative influence from all dysarthric levels.
  • Ordinal Gradation: Dysarthric levels (Mild, Moderate, Severe) exhibit mutual positive influence, indicating the model recognizes shared acoustic features across the dysarthric continuum.
• Ordinal Sensitivity: The average influence $\bar{S}(d)$ decreased monotonically with ordinal distance ($d$). Same-level influence was 13 times stronger than adjacent-level influence, and influence became negative for distant levels ($d \ge 2$), confirming the model encodes severity as an ordered continuum rather than nominal categories.
• Case Study & Dataset Auditing: In a qualitative comparison, the proposed method provided ranked audio references, whereas SHAP provided uninterpretable acoustic descriptor scores. Notably, the influence ranking identified that "near-silent" recordings from severe speakers were among the most influential samples for severe predictions. This revealed a spurious association learned by the model (silence $\to$ severe), demonstrating the framework's utility for dataset auditing.

Significance and Claims
The paper claims that Dys-XAI provides auditable explanations by linking model decisions to perceptible reference cases (audio recordings) rather than abstract feature scores. This aligns with clinical reasoning, where judgments are often justified by comparison with prototypical examples.

The authors emphasize that this approach:

1. Explicitly handles ordinality: By aggregating influence by severity label, the framework exposes relational evidence between adjacent and distant severity levels.
2. Enables Perceptual Verification: Clinicians can directly listen to supportive and competing training samples to verify if the model's decision aligns with clinical markers.
3. Identifies Data Issues: The framework serves as a tool for dataset auditing, capable of surfacing spurious correlations (e.g., silence associated with severity) that feature-based methods might miss.

The work is presented as a step toward bridging the gap between high-performance deep learning models and the transparency required for clinical deployment in dysarthria assessment.