Multi-Channel Replay Speech Detection using Acoustic Maps

Technical Summary: Multi-Channel Replay Speech Detection using Acoustic Maps

Problem Statement
Automatic Speaker Verification (ASV) systems, widely used in voice assistants and IoT devices, remain vulnerable to physical access (PA) attacks, specifically replay attacks. In these scenarios, an adversary plays back a previously captured recording through a loudspeaker to deceive the system. While logical access attacks (e.g., deepfakes) manipulate content or identity, replay attacks exploit the fundamental physical differences between sound production mechanisms: genuine speech originates from a human vocal tract, whereas replayed speech is emitted by an electro-acoustic transducer (loudspeaker) with distinct frequency responses, directivity, and radiation characteristics. Existing ASV systems often struggle to reliably discriminate between genuine and replayed speech, even when using commodity devices. Furthermore, while multi-channel recordings offer spatial cues that are difficult to counterfeit, progress in this area has been limited by a lack of suitable datasets and models designed to exploit spatial information effectively.

Methodology
The authors propose a novel approach utilizing acoustic maps as a spatial feature representation for replay detection from multi-channel recordings. The methodology consists of two primary stages:

1. Acoustic Map Generation:

   • Multi-channel audio is pre-processed using the Short-Time Fourier Transform (STFT).
   • A discrete spatial grid is defined over azimuth ($[-90^\circ, 90^\circ]$) and elevation ($[-50^\circ, 50^\circ]$) angles.
   • Using a delay-and-sum beamformer, the system computes a narrowband pseudo-power response for each grid point. This process encodes the directional energy distribution of the sound field.
   • The resulting maps are time-averaged and frequency bins are grouped into four disjoint bands (Low, Mid, High, Super-High) to create a compact 3-D tensor ($K \times A \times E$). This representation captures array-manifold structure, direct-path cues, and early reflections.
   • The authors note that while delay-and-sum is the primary method used, the framework is compatible with other spatial processing techniques like MVDR or SRP-PHAT.

2. Classification Network:

   • A lightweight Convolutional Neural Network (CNN) is designed specifically to operate on these acoustic maps.
   • The architecture employs repeated blocks of depthwise separable 2D convolutions (operating over azimuth and elevation dimensions), followed by batch normalization, ELU activation, and max pooling.
   • The network is shallow, utilizing only approximately 6,000 trainable parameters. It concludes with a shallow fully connected head (MLP) for binary classification (genuine vs. replay).
   • Training utilizes categorical cross-entropy loss with MixUp augmentation.

Key Contributions

• Novel Feature Representation: The introduction of acoustic maps derived from classical beamforming to explicitly encode directional energy distributions, reflecting the physical differences between human speech radiation and loudspeaker replay.
• Efficient Model Design: The development of a compact CNN tailored to acoustic maps that achieves competitive performance with a significantly reduced parameter count (~6k) compared to state-of-the-art (SOTA) baselines.
• Comprehensive Evaluation: An extensive evaluation on the ReMASC dataset (the only public multi-channel replay dataset), analyzing performance across different microphone arrays (2 to 7 microphones), beamforming methods, and acoustic environments (both environment-dependent and environment-independent).

Experimental Results
The approach was evaluated on the ReMASC dataset using the Equal Error Rate (EER) metric:

• Performance vs. Array Geometry: Detection accuracy correlates with the number of microphones. Arrays with more microphones (D3: 6 mics, D4: 7 mics) yielded lower EERs (e.g., 10.1% for D3) compared to smaller arrays (D1: 2 mics, 21.6%). This confirms that richer spatial sampling improves the reliability of directional energy cues.
• Comparison with SOTA: While the proposed method did not uniformly outperform all SOTA learning-based baselines (such as M-ALRAD or CRNN-based detectors) in terms of raw accuracy, it offered a superior trade-off between performance and computational cost. For instance, on the D3 array, the proposed method achieved 10.1% EER, comparable to M-ALRAD (10.4%), but with ~6k parameters versus ~300k.
• Beamformer Choice: Delay-and-sum provided a stable baseline. MVDR showed comparable or slightly better results in some cases but was more sensitive to estimation errors. SRP-PHAT generally exhibited higher EERs, likely due to sensitivity to reverberation.
• Generalization: In environment-independent scenarios (training on three environments, testing on one unseen), performance degraded significantly for all methods, including the proposed approach. The EER for the proposed method rose substantially (e.g., to 43.2% on Env-D), highlighting that fixed frequency bands and static spatial processing limit adaptability to unseen acoustic conditions.

Significance and Claims
The paper claims that acoustic maps provide a compact and physically interpretable feature space for replay attack detection. The significance of this work lies in demonstrating that classical spatial processing, when combined with a lightweight neural network, can effectively leverage multi-channel data to distinguish between genuine and replayed speech. The authors emphasize that while the current formulation has limitations regarding generalization to unseen environments (due to fixed spatial and frequency representations), the approach offers a viable, resource-efficient alternative to deeper, more complex models. Future work is suggested to focus on learning-based frequency band selection to better capture directivity cues in varying acoustic conditions.