💬 NLP

"OK Aura, Be Fair With Me": Demographics-Agnostic Training for Bias Mitigation in Wake-up Word Detection

This study demonstrates that demographics-agnostic training techniques, specifically data augmentation and knowledge distillation, significantly reduce performance disparities in Wake-up Word detection across diverse speaker groups by excluding demographic labels during the training phase.

Original authors: Fernando López, Paula Delgado-Santos, Pablo Gómez, David Solans, Jordi Luque

Published 2026-04-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Fernando López, Paula Delgado-Santos, Pablo Gómez, David Solans, Jordi Luque

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a smart speaker in your living room. You say, "OK Aura," and it wakes up to listen to your commands. It's supposed to work for everyone, no matter who they are. But in reality, these devices often act like picky bouncers at a club: they let some people in easily but make others jump through hoops, repeat their names, or change how they speak just to get noticed.

This paper is about how the researchers at Telefónica tried to fix this "picky bouncer" problem without needing to know exactly who the people are.

The Problem: The "One-Size-Fits-None" Model

Voice assistants are trained on huge piles of recorded voices. But often, these piles are unbalanced. Imagine a cooking class where 80% of the students are tall men from Madrid, 15% are women, and only 5% are children or people with regional accents.

If the teacher (the AI) only learns from that specific group, they become great at recognizing tall men from Madrid but terrible at understanding children or people with different accents. The result? The device works perfectly for the majority but fails the minority, creating an unfair experience.

The Challenge: The "Privacy Wall"

Usually, to fix this, you'd say, "Okay, let's look at the data and make sure we have enough women, kids, and people with accents." But there's a catch: Privacy.
In the real world, companies often can't collect or store labels like "Age: 65" or "Accent: Caribbean" because it's sensitive personal data. They need a way to make the AI fair without ever knowing who the user is.

The Solution: "Blind" Training

The researchers came up with a clever strategy called Demographics-Agnostic Training. Think of it as training a chef to cook a dish that tastes good to everyone, without ever asking the diners their age, gender, or where they are from. They used two main tricks:

Trick 1: The "Static Radio" Game (Data Augmentation)

Imagine you are trying to learn a song, but every time you practice, someone turns the radio dial slightly or adds a little bit of static.

The Idea: The researchers took the audio data and artificially added "static" or changed the pitch and tone slightly.
The Analogy: If the AI learns to recognize "OK Aura" even when the voice sounds a bit like it's coming through a bad connection, or when the pitch is slightly higher or lower, it stops relying on specific "tells" (like a deep male voice or a specific accent) to make a decision. It learns the core of the phrase instead of the flavor of the speaker.
The Result: One specific technique, called Frequency Masking (hiding parts of the sound spectrum), was the MVP. It forced the AI to look at the whole picture rather than just one detail, reducing unfairness by huge margins (up to 83% better for age groups!).

Trick 2: The "Master Chef" and the "Apprentice" (Knowledge Distillation)

The Master: They used a massive, super-smart AI model (trained on millions of hours of diverse audio) as a "Teacher." This teacher is so smart it has already learned to ignore differences between people and focus on the words.
The Apprentice: They trained a tiny, lightweight model (the "Student") that has to fit on a phone or speaker.
The Analogy: Instead of the student learning from scratch, they let the student watch the Master Chef cook. The student tries to mimic the Master's decisions. Since the Master is already fair and robust, the student learns to be fair too, without needing to be told why a decision was made.

The Results: A Fairer Club

When they tested these methods:

The "Picky Bouncer" became a "Welcoming Host." The gap in performance between different groups (men vs. women, young vs. old, different accents) shrank dramatically.
Privacy was preserved. They didn't need to know who the speakers were to make the system fair.
Performance stayed high. The system didn't get worse at recognizing the wake-up word; it just got better at recognizing it for everyone.

The Bottom Line

This paper proves that you don't need to spy on people's demographics to make technology fair. By teaching the AI to be robust against "noise" and by letting it learn from a super-smart, pre-trained mentor, we can build voice assistants that treat a 70-year-old grandmother with a regional accent just as fairly as a 25-year-old tech enthusiast.

It's like teaching a security guard to recognize a friend by their walk and spirit, rather than by their height or the color of their shirt.

1. Problem Statement

Voice-based interfaces rely on Wake-up Word (WuW) detection to activate systems. However, these systems often exhibit significant performance disparities across demographic groups (sex, age, and accent), leading to unfair user experiences where certain groups must repeat commands more frequently.

The Challenge: Traditional bias mitigation strategies often require explicit demographic labels during training to condition models or rebalance data. In real-world deployments, these labels are often unavailable, privacy-sensitive, or incomplete.
The Gap: There is a need for effective mitigation strategies that do not rely on demographic labels during the training phase (demographics-agnostic) while maintaining high detection accuracy on resource-constrained, always-on devices.

2. Methodology

The authors propose a demographics-agnostic training pipeline using the proprietary "OK Aura" dataset. The core philosophy is to reserve demographic labels strictly for post-hoc evaluation, ensuring the training process remains blind to sensitive attributes.

A. Dataset

Primary Data: OK Aura, a Spanish WuW dataset containing ~5.8k samples from 546 speakers. It includes diverse acoustic environments and negative samples (distractors).
Augmentation Data: Public corpora (Common Voice, M-AILabs) and noise/impulse response datasets (SLR28, DEMAND, MicIRP) are used for training robustness but excluded from bias evaluation due to lack of granular metadata.
Demographic Attributes: Sex (Male/Female), Age (grouped), and Spanish Accent (14 regional/national varieties).

B. Model Architecture

Student Model (Target): A lightweight Gated Recurrent Unit (GRU) classifier (device-sgru) designed for on-device inference. It uses 13 MFCC features and has ~145k parameters, capable of ~25ms inference on a Pixel XL.
Teacher Model (Distillation Source): A high-capacity w2v-BERT 2.0 based model (w2v-BERT2-kws). It utilizes a frozen Conformer encoder trained on 4.5M hours of unlabeled audio, with a learnable weighted sum of layer outputs and Multi-Head Factorized Attention (MHFA).

C. Mitigation Strategies

The study investigates two families of label-free techniques:

Data Augmentation (Spectral Perturbation):
The hypothesis is that disrupting demographic-correlated acoustic cues (e.g., Fundamental Frequency $F_0$ for sex/age, formants for accent) forces the model to learn invariant representations.
- FreqMixStyle: Mixes frequency-wise statistics (mean/variance) between samples to promote domain invariance.
- FilterAugment: Applies smooth, frequency-dependent gains to simulate acoustic filtering without masking entire bands.
- Frequency Masking (FreqMask): Randomly zeros out frequency bands (based on SpecAugment) to force the model to rely on distributed spectral evidence rather than specific cues.
- Device Impulse Responses (DIR): Convolves signals with device impulse responses to simulate hardware variations.
Knowledge Distillation (KD):
The large, pre-trained SSL teacher model (which naturally suppresses speaker identity in upper layers) is used to train the compact student model.
- Process: The student minimizes a weighted combination of Cross-Entropy (ground truth) and Kullback-Leibler (KL) divergence (matching the teacher's soft predictions).
- Goal: Transfer the "demographically robust" representations from the large-scale teacher to the efficient student.

3. Key Contributions

Label-Free Framework: Demonstrates a practical pipeline for bias mitigation in WuW detection that does not require demographic labels during training, addressing privacy and data scarcity issues.
Systematic Evaluation: Provides a rigorous quantification of bias using Predictive Disparity (PD) and Disparate Impact (DI) across sex, age, and accent on a real-world dataset.
Augmentation Analysis: Identifies that not all augmentation techniques are universally beneficial; some (like FreqMixStyle) can degrade fairness for specific groups while helping others.
Frequency Masking Efficacy: Establishes Frequency Masking as the most robust single technique for reducing bias across all attributes without sacrificing overall accuracy.

4. Experimental Results

The study compares a baseline model against models trained with various mitigation techniques.

Baseline Bias: The baseline model showed measurable disparities:
- Sex: Male speakers performed slightly better (F1: 0.9863 vs. 0.9825).
- Age: The 21–30 group performed best (0.9956), while 41–50 performed worst (0.9827).
- Accent: Central Southern Spain outperformed Northern Spain.
Mitigation Performance (Relative Reduction of Predictive Disparity - RRPD):
- Frequency Masking: Achieved the most consistent improvements:
  - Sex: 39.94% reduction in disparity.
  - Age: 83.65% reduction in disparity.
  - Accent: 40.48% reduction in disparity.
- Knowledge Distillation: Significantly reduced Sex (79.64% RRPD) and Age (85.35% RRPD) disparities but showed limited impact on Accent.
- FilterAugment: Highly effective for Sex (88.26% RRPD) but increased accent disparity (-40.19%).
- FreqMixStyle: Improved Age and Accent but degraded Sex fairness (-21.42% RRPD).
Overall Accuracy: The mitigation techniques maintained competitive overall F1-scores, proving that fairness can be improved without sacrificing detection performance.

5. Significance and Conclusion

Practical Deployment: The proposed methods are highly relevant for commercial voice assistants where collecting demographic labels is legally or ethically restricted.
Mechanism Insight: The results suggest that Frequency Masking is particularly effective because it prevents the model from overfitting to demographic-specific spectral bands (like $F_0$ ), forcing it to distribute attention across the broader spectrum.
Limitations: The study notes that univariate analysis misses intersectional biases (e.g., older female speakers with specific accents) and that some groups were too small in the test set for robust statistical guarantees.
Future Work: The authors plan to explore intersectional fairness and multi-objective optimization frameworks that balance overall accuracy with per-group fairness.

In summary, the paper proves that demographics-agnostic training, specifically through Frequency Masking and Knowledge Distillation, is a viable and effective strategy for creating fairer Wake-up Word detection systems in real-world, privacy-constrained environments.