How Fast, How Sure, and Where It Breaks: A Streaming Sequential-Decision Layer for Voice-Deepfake Detection under Real Telephony Codecs

Technical Summary: A Streaming Sequential-Decision Layer for Voice-Deepfake Detection under Real Telephony Codecs

Problem Statement
Voice-cloning attacks increasingly target telephone channels, where audio is compressed by speech codecs and potentially degraded by packet loss. Existing deepfake detectors are typically trained and evaluated on fixed-length, pre-segmented clips, often assuming clean audio or specific degradation conditions. In a live telephony scenario, audio arrives as a stream, and a decision must be made based on whatever has arrived so far, without waiting for the full utterance. Furthermore, detectors often lack calibrated confidence scores, making it difficult to determine when to commit to a decision versus when to abstain (defer) due to low confidence. The core challenge is to adapt a frozen detector to a streaming, sequential-decision context under real-world telephony degradation (codecs and packet loss) while quantifying the trade-offs between latency, confidence, and error rates.

Methodology
The study employs a "frozen" detector approach, wrapping the AASIST backbone (a raw-waveform graph-attention network) without retraining. The methodology introduces a streaming sequential-decision layer that processes audio as a growing prefix:

1. Growing-Prefix Inference: Audio is processed in increments (0.25s to 4.04s). Shorter prefixes are tile-padded (repeated) to fill the detector's fixed input window, mimicking standard evaluation practices.
2. Per-Duration Calibration: Since raw scores from different prefix lengths are not directly comparable, a separate Platt (logistic) calibrator is fitted for each prefix length on held-out development data. This maps raw scores to a calibrated probability $P(\text{spoof})$ for each time step.
3. Sequential Stopping Rule: A two-sided, symmetric stopping rule is applied. The system commits to a decision (spoof or bona fide) only when the calibrated confidence exceeds a threshold $\theta$ (e.g., $\ge \theta$ for spoof, $\le 1-\theta$ for bona fide) for $k$ consecutive steps (persistence guard, $k=2$).
4. Abstention and Forcing: If the confidence threshold is never reached by the end of the audio, the system is "forced" to make a decision based on the full-length score. This forced fraction represents calls deferred in a real deployment.
5. Channel Characterization: Experiments utilize the ASVspoof 2019 LA dataset passed through four real wideband codecs (Opus, Speex-WB, EVS, AMR-WB) at various bitrates and packet loss rates (0%, 10%, 20%).

Key Contributions

1. Streaming Decision Layer: A novel framework that transforms a static classifier into a streaming policy with explicit latency-confidence-abstention trade-offs, applicable to any frozen raw-waveform backbone (demonstrated on AASIST and RawNet2).
2. Channel Characterization: A comprehensive benchmark showing that the same frozen detector's performance varies drastically (EER from ~0.8% to ~29%) solely based on the telephony codec and bitrate, emphasizing that "robustness" is channel-dependent.
3. Latency vs. Accuracy Decomposition: A rigorous analysis separating genuine early-decision gains from artifacts. The author demonstrates that apparent accuracy improvements in harsh channels are largely driven by a tile-padding artifact where the system exploits unreliable, very short prefixes (<1.5s) that are padded to fill the model window.
4. Calibration Transfer Analysis: A critical finding that calibration does not transfer well from clean to harsh channels. A calibrator fitted on clean audio becomes severely overconfident on low-bitrate channels, leading to dangerous error rates.

Results

• Channel Dominance: The Equal Error Rate (EER) of the frozen detector spans from 0.83% (clean) to 29.45% (Speex-WB at 10 kbps). EVS is the most transparent codec, while Speex-WB is the harshest. Packet loss further compounds degradation.
• Latency Reduction: The streaming layer reduces mean decision latency from 3.26 s (full utterance) to approximately 1.6–2.1 s (forced-inclusive), roughly halving the wait time.
• Error vs. Coverage: Under harsh conditions (e.g., Speex-WB q4 + 20% loss), the streaming layer at $\theta=0.9$ achieves an error rate of 2.9% on the subset of calls it decides (covering 53% of calls), compared to **13%** for a full-utterance detector at the same coverage, and ~24% if forced to decide on all calls.
• The "Tile-Padding" Artifact: When restricting the analysis to prefixes $\ge 1.5$ s (removing the unreliable short-window padding), the apparent advantage of the streaming selector collapses. The "temporal selection" advantage drops from ~82% to ~13%, indicating that most of the early gain was due to the system making decisions on padded, low-quality short windows rather than genuine early detection capability.
• Calibration Failure: A calibrator trained on clean audio yields 3–3.5× higher error on harsh low-bitrate channels due to overconfidence. Conversely, a harsh-channel calibrator on clean audio leads to over-caution (more abstentions) but fewer false commits.

Significance and Claims
The paper claims that the primary value of this approach is not a universal increase in accuracy, but rather the ability to reduce latency and provide a calibrated deferral signal. The system allows operators to trade off coverage for confidence, abstaining on roughly 47% of calls in harsh conditions to maintain low error rates on the decided subset.

Crucially, the author modestly frames the results:

• Not a New Detector: The work does not propose a new deepfake detector but rather a decision policy and channel characterization for existing frozen detectors.
• Conditional Validity: The results are conditional on the specific frozen backbone (AASIST/RawNet2) and the specific dataset (ASVspoof 2019 LA codec-augmented). The paper explicitly states it does not claim robustness to unseen, modern neural-codec or diffusion-based attacks.
• Actionable Insight: The most actionable result is the calibration transfer rule: for deployment in telephony, calibration must be fitted on a representative harsh condition (e.g., low bitrate) rather than clean audio, as clean-trained calibrators are dangerously overconfident in degraded channels.
• Artifact Awareness: The study highlights that "early decision" gains in harsh channels are often illusory, driven by tile-padding artifacts, and that genuine early classification is unreliable below ~1.5 seconds even on clean audio.

In summary, the paper provides a framework for operating voice deepfake detectors in real-time telephony, demonstrating that while channel degradation severely impacts raw accuracy, a sequential decision layer can effectively manage latency and risk through calibrated abstention, provided the calibration is matched to the channel conditions.