Technical Summary: Window Size Versus Accuracy Experiments in Voice Activity Detectors

Problem Statement

Voice Activity Detection (VAD) is a foundational component for speech applications, serving critical roles in gating storage and reducing computational load for downstream processing. While VADs have evolved from early energy-based methods to modern neural approaches, there is a need to understand how analysis window size impacts accuracy across different algorithmic families. Specifically, this work investigates the performance trade-offs when extending time windows from milliseconds to seconds, a range not exhaustively explored in prior literature which often focuses on standard fixed windows.

Methodology

The study evaluates three distinct VAD algorithms across a diverse set of real-world digital audio streams:

1. Root Mean Square (RMS): A simple, energy-based baseline that maps signal levels to a probability score without signal-type awareness.
2. WebRTC: A widely deployed, open-source VAD with a fixed native window size of 10ms.
3. Silero: A modern, neural-network-based VAD known for robustness, with a fixed native window size of 16ms.

Experimental Setup:

• Dataset: A 1.1-hour internal test set comprising 21 diverse files (podcasts, movies, music, documentaries) in English, 16-bit PCM, 16kHz, single-channel.
• Window Size Manipulation: For WebRTC and Silero, larger effective windows (ranging from 10ms up to 10 seconds) were constructed by subdividing the target window into model-compatible sub-windows and averaging their predictions.
• Post-Processing: Each VAD was evaluated both in its raw form and with a hysteresis state machine (using distinct high/low thresholds for turning on/off) tuned via grid search.
• Metrics: Performance was assessed using True Positive Rate (TPR), False Positive Rate (FPR), Receiver Operating Characteristic (ROC) Area Under Curve (AUC), Precision-Recall (PR) Average Precision (AP), and the Matthews Correlation Coefficient (MCC). MCC was prioritized for its robustness against class imbalance.

Key Results

1. Algorithmic Performance Hierarchy

Across all tested window sizes and metrics, a clear performance hierarchy emerged:

• Silero significantly outperformed both WebRTC and RMS.
• WebRTC performed significantly better than the simple RMS baseline.
• RMS demonstrated a narrow "sweet spot" and, for most of its operating range, underperformed random guessing due to its lack of signal-type discrimination.

2. Impact of Window Size

• General Trend: Increasing the window size by averaging sub-windows generally led to a degradation in performance as measured by ROC AUC and Average Precision.
• High-Recall Anomaly: In the 0.8 to 1.0 recall range, larger windows occasionally outperformed smaller ones. The authors attribute this to the ground truth labeling methodology, where any presence of speech in a window labels the entire window as "speech." Consequently, larger windows are less likely to miss speech segments that fall between words, whereas smaller windows might classify inter-word gaps as non-speech, penalizing the model despite the continuous nature of the speech chunk.

3. Impact of Hysteresis

The application of hysteresis post-processing yielded mixed results depending on the underlying model:

• WebRTC: Hysteresis provided a measurable benefit, increasing the peak MCC score from 0.41 to 0.47.
• Silero & RMS: Hysteresis offered no significant benefit. Silero's MCC slightly decreased (0.72 to 0.71), and RMS remained largely unchanged (0.11 to 0.10).

Significance and Contributions

This paper offers practical references for optimizing VAD systems by quantifying the relationship between analysis window size and classification accuracy. The primary contributions include:

• Empirical Validation of Modern VADs: Confirming that neural-based approaches (Silero) consistently and significantly outperform traditional energy-based (RMS) and widely used open-source (WebRTC) baselines in diverse real-world scenarios.
• Window Size Analysis: Demonstrating that while larger windows are sometimes necessary for specific downstream use cases, they generally degrade standard accuracy metrics when constructed via averaging.
• Post-Processing Insights: Establishing that hysteresis is a beneficial post-processing step specifically for WebRTC, but not necessarily for more advanced models like Silero or simple baselines like RMS.

Limitations

The authors acknowledge several constraints in their study:

• The dataset, while diverse, is relatively small (1.1 hours) and not exhaustive.
• The method of averaging sub-windows to create larger windows introduces inherent smoothing, which may have masked potential benefits of hysteresis that could be observed with raw, non-averaged windows.
• The ground truth labeling treats entire phrases as single speech units, which may overestimate the utility of larger windows by incorrectly penalizing the classification of gaps between words as non-speech.