Technical Summary: Acoustic Simulation Framework for Multi-Channel Replay Speech Detection

1. Problem Statement

Replay speech attacks, where recorded audio is played back to deceive Automatic Speaker Verification (ASV) systems, pose a significant threat to voice-controlled environments. While multi-channel audio offers spatial cues that could enhance detection robustness, existing research and datasets predominantly rely on single-channel recordings. Collecting real-world multi-channel replay data is prohibitively difficult due to hardware costs, synchronization requirements, and the labor-intensive nature of obtaining precise ground-truth spatial labels across diverse environments.

Consequently, current replay detectors struggle to generalize to unseen environments. The only existing multi-channel dataset, ReMASC, is limited in size and diversity. This scarcity hinders the development of models that can effectively exploit spatial information (such as inter-channel phase differences) to distinguish between live speech and replayed audio, which undergoes "double reverberation" and distinct acoustic coloration.

2. Methodology

2.1 Acoustic Simulation Framework

The authors propose a simulation framework to generate synthetic multi-channel replay speech recordings using publicly available resources. The framework models the replay attack chain in the frequency domain through three distinct scenarios:

1. Genuine Scenario: The ASV system records speech directly from a talker, modeled as a sum of propagation paths filtered by directional responses and room impulse responses (RIRs).
2. Spoofing Scenario: An attacker records the source signal using a single-channel microphone, applying its own propagation paths and directional responses.
3. Replay Scenario: The attacker replays the spoofed signal through a loudspeaker. The signal propagates through a new set of room paths to the ASV array, introducing a second acoustic convolution.

The simulation utilizes:

• Source Data: The EARS dataset for anechoic speech and measured talker directivities.
• Loudspeaker Modeling: Measured loudspeaker directivity impulse responses (SOFA files) from Gallien et al.
• Noise Modeling: The WHAMR! dataset for diffuse background noise, rendered spatially consistent across channels.
• Room Acoustics: Pyroomacoustics to generate spatial RIRs, simulating variable room dimensions, materials, and source-microphone geometries.

The framework generates a balanced dataset of 4,280 recordings (training, validation, and testing) derived from 107 talkers, simulating four specific microphone array configurations found in the ReMASC dataset.

2.2 Detection Model and Feature Engineering

The study employs M-ALRAD, a Convolutional Recurrent Neural Network (CRNN) baseline, as the detection architecture. The authors extend this model to better exploit spatial information:

• Beamforming: An adaptive complex beamformer processes multi-channel Short-Time Fourier Transforms (STFTs) to produce a beamformed spectrogram, extracting log-power and phase (sine/cosine) features.
• Inter-Channel Phase Difference (IPD): To prevent the aggressive collapse of inter-channel cues during beamforming, the authors augment the model with IPD features computed for adjacent microphone pairs. These features capture directional cues that are sensitive to the double-reverberation artifacts of replay attacks.
• Per-Channel Log-Power: The model is also tested with per-channel log-power features to evaluate the contribution of magnitude information.

The input to the CRNN consists of the beamformed features concatenated with IPD maps and, in some configurations, per-channel log-power features.

3. Key Contributions

1. Acoustic Simulation Framework: The introduction of a framework capable of generating multi-channel replay speech configurations using open-source resources, enabling controlled experiments without the need for extensive real-world data collection.
2. Model Extension: The augmentation of the M-ALRAD detector with IPD features computed for adjacent microphone pairs. This approach preserves spatial cues that are often lost in standard beamforming, aiming to improve generalization from synthetic to real data.
3. Environment-Independent Evaluation: A comprehensive evaluation on the ReMASC corpus where models are trained exclusively on synthetic data. This includes a full ablation study analyzing the beamformer, per-channel log-power, and IPD branches to determine which features generalize best to unseen real environments.

4. Results

The models were evaluated using the Equal Error Rate (EER) metric across the four environments of the ReMASC dataset (Outdoor, Quiet Indoor, Noisy Indoor, and Moving Vehicle).

• Synthetic Performance: Both extended models (M-ALRAD + IPD and M-ALRAD + per-ch + IPD) outperformed the baseline on the synthetic test set, with EER reductions ranging from 0.7 to 2.5 percentage points. The performance improved monotonically with the number of microphone channels when IPD features were included.
• Generalization to Real Data:
  • Outdoor (EnvA) and Noisy Indoor (EnvC): The proposed models showed significant improvements over the baseline. Specifically, M-ALRAD + IPD reduced the EER in the outdoor environment from 53.1% to 39.9% (for the 6-mic array) and achieved the best results in the noisy indoor environment (19.2% EER).
  • Quiet Indoor (EnvB): All models performed at or near chance levels (EER ≥ 50%). The authors attribute this to a distribution mismatch in source-array geometry; the training data restricted talkers to a near-frontal cone, whereas EnvB featured widely varying positions.
  • Moving Vehicle (EnvD): Modest improvements were observed for larger arrays, though performance remained challenging due to the complex, unmodeled acoustic conditions of a vehicle.
• Ablation Study:
  • Beamformer vs. IPD: The beamformer acts as a necessary spatial pre-filter. Removing it degraded performance significantly, even when IPD features were present.
  • IPD vs. Per-Channel Power: Adding IPD to the beamformer provided the largest generalization gain (e.g., a 7.5 percentage point improvement in the outdoor environment). In contrast, adding per-channel log-power features yielded marginal synthetic improvements and often degraded performance on real data, suggesting that simulation-specific spectral textures do not transfer well.

5. Significance and Claims

The paper claims that the proposed acoustic simulation framework successfully bridges the gap between single-channel and multi-channel replay detection research by providing a scalable method for data generation. The primary significance lies in demonstrating that Inter-Channel Phase Difference (IPD) features, when combined with an adaptive beamformer, offer a robust mechanism for detecting replay attacks in unseen environments without requiring real-world training data.

The authors modestly conclude that while the framework improves generalization, particularly in outdoor and noisy indoor settings, challenges remain in environments with highly variable source geometries (like the quiet indoor room). They assert that the combination of the adaptive beamformer and IPD features is complementary and necessary, whereas per-channel magnitude features tend to overfit to simulation-specific spectral characteristics. Future work is identified as broadening the simulated source-array geometries and incorporating more diverse loudspeaker and room configurations to further enhance generalization.