GenLie: A Global-Enhanced Lie Detection Network under Sparsity and Semantic Interference

1. Problem Statement

Video-based lie detection aims to identify deceptive behaviors from visual cues. However, the field faces a fundamental challenge: learning sparse yet discriminative representations.

• Sparsity: Deceptive signals (e.g., micro-expressions) are subtle, short-lived, and scattered across temporal segments. They are easily overwhelmed by redundant information in the video.
• Semantic Interference: Individual differences (identity) and contextual variations introduce strong noise, making it difficult to distinguish between a specific person's baseline behavior and actual deception.
• Limitations of Existing Methods:
  • Explicit methods rely on predefined cues (micro-expressions, action units) but miss undefined behaviors.
  • Implicit methods (CNNs/Transformers) learn end-to-end but struggle with sparse signals and identity noise.
  • Hybrid methods often require complex, non-scalable multi-branch architectures.

2. Methodology: GenLie Framework

GenLie is an end-to-end framework designed to perform local feature modeling under global supervision. It adopts a local–global joint modeling strategy consisting of three main components:

A. Preprocessor (Local Modeling)

The goal is to isolate sparse cues and suppress redundancy.

• Segmentation: Videos are divided into $N$ equal-length segments.
• Redundancy-Aware Frame Selection: From each segment, the top-$K$ informative frames are selected to form a compact set ($N \times K = 128$). The paper evaluates six strategies (AU-based, Micro-expression, Gaze, Posture, Multi-cue Fusion, and Uniform).
• Task-Driven Re-embedding: The selected frames are processed by a frozen VideoMAEv2 encoder to extract segment-level features. These features are then refined through a two-layer MLP (re-embedding module) to highlight latent deceptive cues and unify sparse signals.

B. Aligner (Global Representation)

• The sequence of segment features is aggregated using mean pooling and passed through an MLP to generate a final global video embedding ($z \in \mathbb{R}^{768}$). This embedding represents the video's semantic content while attempting to filter out noise.

C. Discriminative Head (Global Supervision)

To ensure the learned representations are robust against identity bias and highly discriminative, two auxiliary objectives are introduced alongside the main classification loss:

1. Speaker-Decorrelation (Adversarial Learning):
   • An auxiliary speaker classifier is attached to the global embedding via a Gradient Reversal Layer (GRL).
   • During backpropagation, gradients from the speaker identity prediction are reversed, forcing the main network to learn features that are identity-invariant (i.e., unable to predict who the speaker is, thereby removing speaker-specific noise).
2. Triplet Loss:
   • A video-level triplet loss is applied to enhance the discriminative structure of the embedding space.
   • It minimizes the distance between embeddings of the same class (anchor and positive) and maximizes the distance to embeddings of different classes (negative), even when sampled from different speakers.

Total Loss Function:
$$L_{total} = L_{cls} + \alpha \cdot L_{id} + \beta \cdot L_{tri}$$
Where $L_{cls}$ is the classification loss, $L_{id}$ is the adversarial speaker loss, and $L_{tri}$ is the triplet loss.

3. Key Contributions

1. Novel Framework: Proposes GenLie, an end-to-end solution specifically targeting the sparsity and semantic interference challenges in lie detection.
2. Local-Global Strategy: Introduces a unique architecture combining redundancy-aware local frame selection, task-driven feature re-embedding, and global adversarial optimization to produce identity-invariant embeddings.
3. State-of-the-Art Performance: Demonstrates superior performance across diverse scenarios (low-stakes and high-stakes) compared to explicit, implicit, and hybrid baselines.
4. Insightful Analysis: Reveals that uniform sampling is often more robust than heuristic cue-based sampling (like micro-expression or gaze), and that speaker-decorrelation is critical for generalization.

4. Experimental Results

The model was evaluated on three public datasets covering different stakes:

• MDPE: Large-scale Chinese corpus (low-stakes).
• Real-Life Trial: Courtroom videos (high-stakes).
• SEUMLD: High-stakes conversational segments.

Performance Highlights:

• Overall: GenLie consistently outperformed all baselines (including MBF, MAD-Net, FFCSN, FacialCueNet, and AFFAKT) in F1-score, Accuracy (ACC), and AUC.
• Specific Achievements:
  • Real-Life Trial: Achieved the best ACC (93.39%) and AUC (98.96%).
  • MDPE: Achieved the highest F1 (38.97%) and AUC (59.91%), handling class imbalance effectively.
  • SEUMLD: Top performance in F1 (42.39%) and AUC (59.15%) under high-stress conditions.

Ablation Studies:

• Re-embedding (FSR): Proven most crucial for performance, particularly on MDPE and SEUMLD.
• Speaker-Decorrelation ($L_{id}$): Essential for the Real-Life Trial dataset to mitigate speaker bias.
• Frame Selection: Contrary to intuition, Uniform sampling yielded the best results across datasets, suggesting that heuristic cue selection may introduce bias or miss subtle, non-facial cues.

5. Significance

• Robustness: GenLie addresses the core limitation of current lie detection systems by explicitly decoupling identity noise from deceptive signals through adversarial training.
• Simplicity and Scalability: Unlike complex hybrid models, GenLie integrates with frozen video encoders (VideoMAEv2) and requires minimal tuning, making it reproducible and scalable.
• Generalization: The framework demonstrates strong generalization capabilities across different cultural contexts, stakes, and recording environments, offering a practical solution for real-world deployment in security, legal, and financial domains.
• Paradigm Shift: The finding that uniform sampling outperforms cue-specific sampling challenges the prevailing assumption that focusing on specific facial micro-cues is always superior, suggesting that global temporal context is equally vital.