Assessing the Energy and Carbon Emissions of Neural Speaker Verification Model in Training and Inference

Technical Summary: Assessing the Energy and Carbon Emissions of Neural Speaker Verification Models in Training and Inference

Problem Statement
Deep-learning speaker verification (SV) systems increasingly rely on deep neural network backbones, such as ResNet, to extract discriminative speaker embeddings. While architectural depth has increased from 9 to over 290 layers to improve performance, this trend introduces significant trade-offs regarding memory requirements, runtime, and environmental impact. Despite the known financial and ecological costs of training large models, there is a lack of published studies specifically analyzing the trade-off between speaker verification performance, energy consumption, and carbon footprint. Existing literature has addressed these factors in Automatic Speech Recognition (ASR) and spoken language understanding, but SV remains under-explored in this context.

Methodology
The authors conducted a systematic evaluation of ResNet-based SV architectures trained on the VoxCeleb2 dataset. The study varied three key structural dimensions:

1. Depth: Scaling the number of layers from 18 to 419 (ResNet-X-D).
2. Width: Adjusting the number of feature maps via multipliers (ResNet-50-Wx).
3. Stage Distribution: Redistributing residual blocks across the four stages of the network while keeping total depth constant (ResNet-50-D[a,b,c,d]).

Environmental Measurement:
Energy consumption and carbon emissions were measured using the Compute Energy and Emissions Monitoring Stack (CEEMS). This tool gathers fine-grained data from node-level hardware sensors, specifically Intel's Running Average Power Limit (RAPL) for CPU/DRAM and Baseboard Management Controllers (via IPMI-DCMI) for broader system components. Carbon footprint was estimated by converting energy usage (kWh) into CO2 equivalents (CO2eq) using real-time emission factors from the French electricity grid (RTE eCO2), which has a low carbon intensity (~50 gCO2/kWh).

Experimental Setup:
Models were trained using the Kiwano toolkit with standard data augmentation (MUSAN, RIRs, SpecAugment) and AAM loss. Evaluation was performed on in-domain datasets (VoxCeleb1-O, E, H) and out-of-domain datasets (CommonBench, CN-Celeb) using Equal Error Rate (EER) and minimum Detection Cost Function (minDCF) as performance metrics. Inference costs were also measured on the CommonBench dataset under both FP32 and FP16 precision.

Key Results

• Diminishing Returns on Depth: Increasing network depth improves performance, but with rapidly diminishing returns. Moving from ResNet-18 to ResNet-101/200 yields significant gains, but the transition from ResNet-200 to ResNet-419 results in marginal accuracy improvements (e.g., average EER dropping only from 3.44% to 3.35%) while energy consumption quadruples (from ~222 kWh to ~896 kWh).
• Width Scaling: Moderate width expansion (e.g., ResNet-50-W2) offers slight in-domain gains but fails to generalize consistently to out-of-domain data. Extreme widening (ResNet-50-W4) provides negligible performance benefits over the baseline while drastically increasing energy use and carbon emissions. Conversely, narrower variants (e.g., ResNet-50-W0.5) reduce energy consumption significantly while maintaining competitive accuracy.
• Stage Distribution: Concentrating residual blocks in the intermediate stages (stages 2 and 3) improves the performance-efficiency trade-off. Variants like ResNet-50-D[2,8,2,2] and ResNet-50-D[2,6,6,2] achieve lower EERs than the baseline ResNet-50-D with fewer parameters and slightly reduced energy consumption. Conversely, concentrating blocks in the first or last stages degrades performance.
• Inference Efficiency: Inference costs are significantly lower than training but remain non-negligible at scale. Switching to FP16 precision yields 25–35% energy savings and reduced runtime without performance loss. The largest model (ResNet-419) emits nearly six times more CO2 than the smallest (ResNet-18) during inference on large datasets.

Key Contributions

1. Comprehensive Comparison: The paper provides a detailed comparison of ResNet variants by systematically varying depth, width, and block distribution to characterize the trade-offs between SV performance and environmental impact.
2. Cross-Domain Evaluation: Models were evaluated on both in-domain and out-of-domain test sets, linking performance metrics directly to measured energy consumption and carbon footprint.
3. Actionable Guidelines: The study identifies specific configurations that offer the best balance between accuracy, energy, and emissions, providing concrete recommendations for designing sustainable SV systems.

Significance and Recommendations
The paper concludes that evaluating SV systems solely on performance metrics (EER, minDCF) is insufficient. The authors argue for a shift toward energy-aware design. Their findings suggest:

• Optimal Architectures: Balanced, mid-sized models like ResNet-34-D and ResNet-50-D represent the "sweet spot," offering strong accuracy with energy consumption an order of magnitude lower than very deep models.
• Resource-Constrained Scenarios: Narrower variants (e.g., ResNet-50-W0.5) are recommended for scenarios where energy and carbon footprint are primary constraints.
• Design Strategy: Reallocating residual blocks toward intermediate stages is a more effective strategy for efficiency than concentrating capacity in the earliest or latest stages.
• Avoidance of Excess: Very deep models (beyond ResNet-200) and extremely wide variants should be avoided unless their marginal performance gains clearly justify the disproportionate environmental cost.

The study emphasizes that while deeper networks capture more discriminative information, the environmental cost of training and inference often outweighs the marginal gains in verification accuracy, necessitating a more sustainable approach to model selection and deployment.