Technical Summary: SpurAudio - A Benchmark for Studying Shortcut Learning in Few-Shot Audio Classification

Problem Statement

Few-shot classification (FSC) aims to recognize novel classes from limited labeled examples, a capability critical for real-world audio applications such as bioacoustic monitoring, industrial fault diagnosis, and healthcare analysis. However, a defining challenge in audio is its additive nature: foreground events are superimposed on background sounds in the time–frequency domain. Unlike images, where objects are often spatially separable, audio foregrounds and backgrounds are inseparably mixed.

This mixing creates a vulnerability where models may exploit spurious correlations between class labels and background cues (contextual shortcuts) rather than learning semantic foreground content. While such effects are known in vision, their role in few-shot audio classification remains largely unexplored. Existing audio benchmarks often fail to control or disentangle foreground-background correlations, potentially overestimating model robustness by rewarding reliance on contextual shortcuts. Consequently, it remains unclear how much current few-shot audio methods rely on these shortcuts and how fragile their performance is when background contexts shift (Out-of-Distribution or OOD).

Methodology: The SpurAudio Benchmark

To address this gap, the authors introduce SpurAudio, a controlled benchmark designed to systematically isolate and evaluate spurious correlations in few-shot audio classification.

Dataset Construction

SpurAudio is synthesized by mixing foreground events from five diverse real-world datasets (ESC-50, UrbanSound8K, VocalSound, WILD DESED, and USM) with semantically unrelated background textures.

• Foreground (FG): The target event class to be recognized.
• Background (BG): A confounding context sampled from a semantically unrelated class.
• Mixing Process: Foreground and background signals are resampled to 16 kHz, trimmed/padded to 5 seconds, and mixed using the EBU R128 standard. The background is scaled to a fixed perceptual margin of 8 dB below the foreground ($\gamma = 8$) to ensure it remains audible without overwhelming the signal.
• Quality Control: Manual annotation ensures semantic independence and perceptual separation, resulting in a curated subset of 16,378 sound events.

Evaluation Protocol

The benchmark evaluates FSC methods under two distinct task settings:

1. In-Distribution (IID): Support and query examples share the same background pairings for each class (e.g., Class A is always paired with Background X). This allows models to exploit background shortcuts.
2. Out-of-Distribution (OOD): Background pairings are altered to break spurious correlations (e.g., Class A is paired with Background Y in the query set, while the support set uses Background X). This forces models to rely on foreground content.

The performance gap, defined as $\Delta = \text{Acc}_{\text{IID}} - \text{Acc}_{\text{OOD}}$, serves as the primary metric for quantifying reliance on spurious correlations.

Experimental Scope

The authors evaluate a comprehensive range of FSC methodologies across multiple backbone architectures:

• Method Families: Fine-tuning-based, Meta-learning-based, Metric-based, Contrastive-based, and Transductive methods.
• Backbones: From shallow CNNs (Conv64, ResNet12) to large pretrained audio foundation models (CLAP, AST, AudioMAE, Qwen2-Audio-7B).
• Settings: 1-shot and 5-shot classification tasks with 10 query samples per class.

Key Results

1. Universal Vulnerability to Shortcut Learning

The study reveals that every FSC method family suffers a significant performance degradation when background correlations are disrupted, regardless of the backbone size or the number of shots.

• Magnitude of Gap: Even state-of-the-art methods achieving high accuracy under standard (IID) protocols show severe drops in OOD settings. For instance, on ResNet12, the gap ($\Delta$) for methods like Baseline++ and ProtoNet exceeds 10% in 5-shot settings.
• Foundation Models: Crucially, this vulnerability persists even in large pretrained audio foundation models (e.g., CLAP, Qwen2-Audio-7B) when paired with standard few-shot heads. Large models achieve ~96% IID accuracy but still suffer double-digit gaps under OOD conditions, ruling out limited backbone capacity as the sole explanation.

2. Geometric Analysis of Failure

Through geometric decomposition of feature embeddings, the authors identify the mechanism behind these failures:

• Magnitude Contraction: Background noise consistently reduces the magnitude (L2 norm) of feature vectors while preserving their angular direction (semantic identity).
• Directional Stability: The cosine similarity between mixed samples and clean prototypes remains high, indicating that the semantic direction is intact.
• Metric Sensitivity: Methods relying on Euclidean distance or unnormalized dot products (e.g., ProtoNet, Baseline) are highly sensitive to magnitude changes, interpreting the "signal dampening" caused by background noise as dissimilarity. Conversely, methods using cosine similarity (e.g., Baseline++, Meta-Baseline) or transductive approaches (e.g., Proto-LP, BD-CSPN) that normalize features or leverage query-set statistics show significantly smaller gaps, often reducing $\Delta$ by an order of magnitude.

3. Head vs. Backbone Dependency

The "inference head" plays a more critical role than the encoder in determining robustness.

• Head-Backbone Swaps: Replacing magnitude-sensitive heads with cosine-based heads on standard backbones yields the largest robustness gains.
• Transductive Methods: Transductive heads (which process the query set collectively) consistently shrink the IID-OOD gap, particularly when paired with large pretrained encoders that produce well-clustered manifolds.

Key Contributions

1. SpurAudio Benchmark: A controlled dataset enabling the manipulation of foreground-background correlations across multiple audio domains to systematically study shortcut learning.
2. Characterization of OOD Failures: Empirical evidence showing that state-of-the-art few-shot audio methods collapse when background contexts shift, revealing a fundamental reliance on spurious correlations.
3. Comprehensive Benchmarking: An extensive evaluation of metric-based, meta-learning, contrastive, transductive, and fine-tuning approaches across standard and large pretrained backbones, highlighting systematic algorithmic strengths and vulnerabilities.

Significance and Claims

The paper argues that the observed vulnerability to background shifts is a fundamental property of few-shot audio inference rather than a limitation of representational capacity. The findings suggest that current benchmarks often conflate semantic foreground content with background context, leading to an overestimation of model robustness.

The authors conclude that robustness in few-shot audio learning is strictly dependent on the choice of inference mechanisms that ignore feature magnitude (e.g., cosine similarity, transductive propagation). This work highlights the need for benchmarks that explicitly probe context dependence and motivates future research into magnitude-aware objectives and adaptive similarity metrics to ensure reliable deployment in real-world, variable acoustic environments.