Cross-Attention is Half Explanation in Speech-to-Text Models

Technical Summary: Cross-Attention is Half Explanation in Speech-to-Text Models

Problem Statement
Cross-attention mechanisms are foundational to encoder-decoder Transformer architectures used in Speech-to-Text (S2T) systems, including Automatic Speech Recognition (ASR) and Speech Translation (ST). These mechanisms are frequently exploited for downstream applications such as timestamp prediction, speech-text alignment, and speaker identification, operating under the implicit assumption that attention scores reliably reflect the model's focus on specific input features during output generation. While the reliability of attention as an explanation has been extensively debated in Natural Language Processing (NLP), this assumption remains largely unverified in the speech domain. A primary concern is "context mixing," where the encoder's output sequence reorganizes or mixes raw audio information with contextual data, potentially obscuring the direct alignment between cross-attention weights and the original input signal. Existing S2T explainability work has focused primarily on self-attention or the effects of context mixing, without systematically assessing the explanatory power of cross-attention itself.

Methodology
The authors present the first systematic analysis of cross-attention as a proxy for input-output dependencies in S2T models. The study compares cross-attention (CA) scores against input saliency maps derived from SPES, a state-of-the-art feature-attribution method for S2T. The methodology involves three primary comparisons:

1. CA vs. Input Saliency ($SM_X$): To assess how well CA reflects the relevance of raw input features (mel-spectrogram) to token predictions.
2. CA vs. Encoder Output Saliency ($SM_H$): To evaluate how well CA explains the decoder's use of encoded representations, thereby isolating the impact of context mixing.
3. Aggregation Strategies: The study evaluates different aggregation methods (2D average, 2D max, and 2-step pooling) over time and frequency dimensions to determine the optimal way to compress 3D saliency maps into a 1D temporal representation compatible with CA granularity.

The analysis spans monolingual and multilingual settings, single-task (ASR) and multi-task (ASR+ST) configurations, and models of varying scales (from 125M to 878M parameters). The authors utilize Pearson correlation to quantify the linear relationship between CA scores and saliency maps, as well as deletion metrics to evaluate the faithfulness of the explanations.

Key Contributions

• Systematic Empirical Assessment: The paper provides the first large-scale empirical evaluation of cross-attention as an explanatory tool in S2T, covering multiple languages (English, Italian), tasks, and model architectures (Conformer-Transformer).
• Quantification of Explanatory Limits: The study quantifies the extent to which cross-attention captures input relevance, moving beyond qualitative observations to precise statistical bounds.
• Context Mixing Analysis: By comparing CA against both raw input saliency and encoder-output saliency, the work isolates the specific contribution of context mixing to the discrepancy between attention and true input relevance.
• Aggregation Insights: The authors demonstrate that 2D maximum pooling yields the most faithful input explanations, preserving localized salient regions in the frequency domain that average pooling tends to blur.

Results
The empirical findings reveal that cross-attention offers useful but incomplete explanatory cues:

• Moderate Alignment: There is a moderate to strong correlation between cross-attention and input saliency maps, particularly when scores are aggregated across attention heads and layers. The best correlations reach approximately 0.588.
• Limited Coverage: Cross-attention captures only about 50% of input relevance. Even when comparing CA against encoder output saliency (where context mixing is removed), CA explains only 52–75% of the encoder's output saliency.
• Layer and Head Dynamics: Individual attention heads often exhibit noisy or weak correlations. However, averaging across heads consistently improves alignment, and the final decoder layers generally show the strongest correlation with saliency maps.
• Context Mixing Impact: The correlation between CA and encoder outputs is consistently higher than with raw inputs, with absolute differences ranging from 0.03 to 0.18. This quantifies the influence of context mixing, yet the remaining gap confirms that CA is not a faithful proxy even at the encoder-output level.
• Deletion Metrics: Direct evaluation using deletion metrics shows that CA-based explanations (score ~41.2) are significantly less faithful than feature-attribution methods (score ~52.9 for aggregated, ~91.3 for full-resolution), indicating that CA discards fine-grained time-frequency cues.

Significance and Claims
The paper concludes that cross-attention should not be treated as a standalone explainable AI (XAI) tool or a faithful proxy for model behavior in S2T systems. While it provides informative cues and is computationally lightweight, it offers only a partial view of the factors driving predictions. The authors argue that cross-attention is "half explanation," capturing roughly half of the relevant signal. Consequently, the paper advocates for interpreting cross-attention with caution in downstream applications and suggests that future work should explore hybrid approaches or training-time strategies (e.g., attention regularization) to better align attention mechanisms with attribution signals, rather than relying on attention scores as a substitute for formal explainability methods.