🤖 machine learning

BioKD: Selective Physiology-to-Video Knowledge Distillation via Reliability Gate for Emotion Recognition

This paper proposes BioKD, a reliability-aware knowledge distillation framework that leverages noisy physiological signals as privileged training information to guide a video-only student model for robust emotion recognition, effectively mitigating negative transfer through adaptive gating while eliminating the need for physiological sensors during inference.

Original authors: Bojing Hou, Ruohao Li, Yitong Zhu, Hongjun Liu, Luwen Yu, Yuyang Wang

Published 2026-08-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Bojing Hou, Ruohao Li, Yitong Zhu, Hongjun Liu, Luwen Yu, Yuyang Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess how someone is feeling just by watching them. You might look at their smile, their frown, or how they move their hands. This is the world of affective computing, a branch of science dedicated to teaching computers to understand human emotions. For a long time, scientists have relied on these visible "behavioral cues" because cameras are easy to use and don't bother people. However, there's a catch: people are good at hiding their true feelings. They might smile when they are sad or stay still when they are furious. It's like trying to guess the weather by only looking at a person's umbrella; sometimes they carry one even when it's sunny, or forget it when it's pouring.

To get a better read on the "real" weather inside a person, scientists also look at physiological signals. These are the body's internal alarms, like heart rate, skin sweat, or brain waves. Think of these as the body's honest, involuntary whispers that can't be faked as easily as a smile. The problem is, these whispers are messy. Sensors can slip, wires can wiggle, and every person's body reacts differently. It's like trying to listen to a radio station that has a lot of static and changes frequency randomly. So, the big question becomes: How can we use these noisy, internal whispers to teach a computer to read emotions, without forcing the computer to wear messy sensors every time it tries to guess a feeling?

This is where a new study called BioKD steps in with a clever solution. The researchers realized that while physiological signals are too messy to use as a permanent tool for reading emotions, they are incredibly useful for training a computer. They proposed a system that acts like a strict but helpful tutor. Imagine a student (a computer program that only watches videos) trying to learn how to spot emotions. Usually, this student has to learn from scratch, guessing based on what it sees. But in this new method, the student gets to sit next to a "teacher" who has access to those messy internal whispers (the physiological signals) only during the training sessions.

The teacher knows the truth because it can hear the body's whispers, but it also knows those whispers are sometimes unreliable. If the teacher is confident but the signal is actually noisy, it might give the student bad advice. To fix this, BioKD uses a special "reliability gate." Think of this gate as a bouncer at a club. When the teacher tries to pass information to the student, the bouncer checks: "Is the teacher's signal clear and steady right now?" If the answer is yes, the teacher gets to whisper the secret to the student. If the answer is no (because the signal is shaky or the teacher is confidently wrong), the bouncer blocks the information, protecting the student from learning bad habits.

The researchers tested this idea on two big datasets of people watching videos while their brain waves and heart rates were recorded. They found that BioKD was much better at teaching the video-only student than other methods. For example, when testing on the DEAP dataset, the BioKD student achieved an accuracy of 68.01% in recognizing "arousal" (how excited or calm someone is) when looking at new people it had never seen before. This was a significant jump compared to other methods that tried to just copy the teacher without checking if the teacher was reliable.

One of the most important things the paper found is that you can't just trust a teacher because they sound confident. The study showed that the physiological teacher often made "overconfident errors"—it would be very sure of its answer, but actually be wrong. If a student blindly copied this confident teacher, it would learn to make the same mistakes. BioKD's "bouncer" successfully spotted these moments and stopped the bad advice from getting through. In fact, when the teacher was confidently wrong, BioKD corrected the error in 39.47% of cases, whereas a standard method only fixed 18.42% of them.

The beauty of this system is that once the student is trained, the teacher and the messy sensors disappear. The final computer only needs a video camera to work, making it easy to use in real life without wires or sticky sensors. The researchers suggest that this approach of using noisy internal signals as a temporary, supervised guide could be a game-changer for making emotion-recognition technology that is both smart and practical. They didn't just show that it works; they proved that checking the reliability of the teacher is just as important as having a teacher in the first place.

Technical Summary: BioKD – Selective Physiology-to-Video Knowledge Distillation via Reliability Gate for Emotion Recognition

1. Problem Statement

Emotion recognition systems relying solely on video (facial expressions, body movements) often struggle when external behaviors are ambiguous, socially masked, or regulated, leading to discrepancies between observable cues and internal affective states. While physiological signals (e.g., EEG, EDA, BVP) provide direct access to internal states, they are inherently noisy, suffer from inter-subject variability, and are intrusive or costly to acquire, making them unsuitable for real-world inference-time deployment.

Current multimodal fusion approaches attempt to use both modalities at inference, inheriting the deployment limitations of physiological sensors. Conversely, using physiological signals solely as training supervision via standard Knowledge Distillation (KD) is problematic. Because physiological teachers are built on noisy data, their predictions can be unstable or contain "overconfident errors" (high confidence but incorrect labels). Applying uniform KD in this setting risks negative transfer, where the student model learns from misleading physiological supervision, degrading performance rather than improving it.

The core challenge addressed is: How to safely distill affective knowledge from unreliable physiological teachers into deployable video-based models without introducing negative transfer caused by noisy supervision.

2. Methodology: The BioKD Framework

The authors propose BioKD, a reliability-aware, progressive physiology-to-video knowledge distillation framework. The framework treats physiological signals as "privileged information" available only during training, while the deployed model relies exclusively on video.

A. Selective Reliability Gate

The core innovation is a sample-wise gating mechanism that dynamically regulates the strength of knowledge transfer based on the estimated reliability of the physiological teacher for each specific sample.

Instantaneous Metrics: The system computes three cues at each iteration:
- Feature Agreement ( $a_i$ ): Semantic alignment between the student's and teacher's projected representations in a shared space.
- Prediction Disagreement ( $d_i$ ): The absolute difference between the student's and teacher's predicted probabilities.
- Teacher Confidence ( $conf_i$ ): The teacher's own prediction confidence.
Temporal Consistency Memory: To handle non-stationary noise and short-term fluctuations, the system employs a Consistency Memory Bank. It uses exponential smoothing (momentum) to accumulate historical estimates of feature agreement and prediction disagreement, while treating teacher confidence as an instantaneous saliency cue.
Monotonic Gating: A gating function combines these cues to generate a reliability weight ( $w_i$ $w_{i}$ ).
- High historical agreement and high teacher confidence increase the weight.
- Persistent disagreement decreases the weight.
- The gate enforces a monotonic constraint: disagreement always suppresses transfer, preventing the propagation of high-confidence but incorrect teacher signals.
- A mean-anchoring regularizer prevents the gate from collapsing to zero or one globally.

B. Progressive Distillation Strategy

To stabilize training, BioKD introduces teacher supervision in three stages:

Stage 0 (Warm-up): The student is trained only on ground-truth labels to establish a stable visual baseline.
Stage 1 (Logit Alignment): Logit-level distillation is introduced to guide the student on inter-class semantic relations. The reliability gate begins tracking consistency patterns in the background.
Stage 2 (Feature Refinement): Feature-level distillation is activated, controlled by the reliability gate. Deep feature alignment occurs only when the teacher is deemed reliable for that specific sample.

3. Key Contributions

BioKD Framework: A novel framework that repositions physiological signals as training-only knowledge sources, improving video-based emotion recognition without modifying inference-time inputs or requiring physiological sensors during deployment.
Reliability-Aware Gating: A mechanism that explicitly models sample-wise reliability variations in physiological supervision. It selectively transfers knowledge, mitigating negative transfer caused by noisy, unstable, or overconfident physiological signals.
Progressive Distillation: The integration of a multi-stage training schedule that gradually introduces teacher supervision, ensuring stability when distilling from unreliable physiological teachers.

4. Experimental Results

Experiments were conducted on the DEAP and AMIGOS datasets under both trial-wise and subject-wise (subject-independent) evaluation protocols for Valence and Arousal recognition.

Performance: BioKD consistently outperformed representative baselines (including standard KD, FitNets, VID, RKD, and other cross-modal distillation methods).
- On DEAP (Subject-wise, Arousal), BioKD achieved 65.29% accuracy, significantly outperforming the best baseline (EmotionKD at 61.07%) and the video-only student (56.43%).
- On AMIGOS, BioKD also showed consistent gains over baselines across all tasks.
Negative Transfer Mitigation: Ablation studies confirmed that naïve distillation (without the gate) often leads to performance degradation compared to the video-only baseline, validating the risk of negative transfer. The full BioKD model (with gate and memory) was essential for achieving gains.
Correction of Overconfident Errors: BioKD successfully corrected 39.47% of cases where the physiological teacher made overconfident wrong predictions, outperforming standard KD (18.42%) and the video-only student (25.00%).
Calibration: BioKD significantly improved model calibration, achieving an Expected Calibration Error (ECE) of 0.089, compared to 0.234 for the video-only student and 0.190 for the best competing baseline.
Generalization: The method proved effective across different student backbones (MobileNetV2-GRU and VideoMAE) and various teacher modality configurations (EEG-only vs. full multimodal).
Deployment: The deployed model incurs no additional inference-time overhead relative to the video student, as the teacher, gate, and memory are discarded after training.

5. Significance and Claims

The paper claims that the primary limitation in physiology-to-video distillation is not the modality gap, but the unreliability of the teacher supervision at the sample level. The authors argue that:

Average teacher performance is insufficient: A teacher with high average accuracy can still provide misleading, high-confidence supervision for specific noisy samples.
Explicit reliability modeling is necessary: Unlike previous methods that assume uniform reliability or focus solely on alignment, BioKD demonstrates that explicitly modeling and filtering supervision reliability is critical for stable cross-modal learning.
Privileged Information Paradigm: Physiological signals are best utilized as "privileged information" during training to complement video, rather than as mandatory inference-time inputs. This approach bridges the gap between internal affective states and external behavioral cues while maintaining the deployability of video-only systems.

The work concludes that BioKD offers a robust solution for leveraging rich but noisy physiological data to enhance video-based emotion recognition, provided that the distillation process is carefully regulated to prevent the propagation of unreliable signals.