See Through the Noise: Improving Domain Generalization in Gaze Estimation

1. Problem Statement

Context: Gaze estimation is critical for applications like human-computer interaction and VR. While deep learning has improved performance, models often fail when deployed in unseen domains (Domain Generalization).
Core Issue: Existing domain generalization methods largely ignore label noise. In gaze estimation, acquiring precise ground-truth annotations is inherently difficult due to unstable human attention, lighting, and occlusion. This leads to significant annotation noise in large-scale datasets (e.g., Gaze360).
Challenges:

1. Regression vs. Classification: Most noisy label learning (NLL) methods are designed for classification. Gaze estimation is a regression task where labels are continuous angles, making it difficult to define clear boundaries between "clean" and "noisy" samples or use standard heuristics like "small-loss selection."
2. Cross-Domain Complexity: Simply correcting noisy labels within the source domain does not guarantee better generalization to target domains, as it fails to bridge the distribution gap between domains.
3. Overfitting Noise: Models trained on noisy data tend to overfit to incorrect labels, learning gaze-irrelevant features (e.g., head pose or background) rather than true gaze direction, degrading cross-domain performance.

2. Methodology: The SeeTN Framework

The authors propose See-Through-Noise (SeeTN), a framework that mitigates label noise to enhance domain generalization. The method operates in three main stages:

A. Semantic Manifold Construction

To address the continuous nature of gaze labels, the authors construct a Semantic Manifold ($M$) to preserve the topological structure between features and labels.

• Prototype Transformation: Features are projected onto a unit hypersphere and mapped to $K$ learnable prototypes ($\mu_k$) via an MLP and Exponential Moving Average (EMA) updates.
• Affinity Alignment: The method calculates an affinity matrix for the features in the manifold ($A^{(m)}$) and the ground-truth gaze labels ($A^{(g)}$) using cosine similarity.
• Objective: The model minimizes the Mean Absolute Error (MAE) between $A^{(m)}$ and $A^{(g)}$. This forces the feature space to maintain a structure consistent with the label space, ensuring that samples with similar gaze angles have similar feature representations.

B. Noise Identification via Affinity Discrepancy

Instead of using loss magnitude (which is unreliable in regression), SeeTN uses the discrepancy between feature affinity and label affinity to detect noise.

• Indicator ($\eta$): For each sample, the method computes a discrepancy indicator $\eta_i$ using a cross-entropy formulation between the label distribution ($\hat{y}^{(g)}$) and the feature distribution ($\hat{y}^{(m)}$) in the manifold.
  $$\eta_i = -\sum_{j=1}^{B} \hat{y}^{(g)}_{i,j} \log \hat{y}^{(m)}_{i,j}$$
• Partitioning: Samples are sorted by $\eta$. The top $t\%$ with the highest discrepancy are classified as the Noisy Set ($D_S^N$), and the rest form the Clean Set ($D_S^C$). This partition is updated dynamically during training to avoid confirmation bias.

C. Noise-Robust Regularization

The framework applies distinct loss functions to the clean and noisy subsets to maximize information utilization while suppressing noise.

• Clean Set ($D_S^C$): Uses standard gaze regression loss ($L_{gaze}$) and an alignment loss ($L_{align}^C$) to strictly enforce the affinity consistency between features and labels.
• Noisy Set ($D_S^N$): Instead of discarding these samples, the method uses a soft alignment loss ($L_{align}^N$). It leverages the affinity matrix derived from clean samples to constrain the feature affinity of noisy samples. This transfers gaze-related information from clean to noisy samples, effectively correcting the noisy labels without relying on the corrupted ground truth.
• Total Loss: $L_{all} = L_{gaze} + L_{align}^C + \lambda L_{align}^N$.

3. Key Contributions

1. Novel Perspective: First work to comprehensively investigate and address domain generalization in gaze estimation specifically through the lens of label noise mitigation.
2. SeeTN Framework: Proposes a prototype-based semantic manifold that bridges continuous gaze labels and feature representations. It introduces a novel affinity-based indicator ($\eta$) to distinguish noisy samples in regression tasks and a noise-robust regularization strategy that utilizes clean samples to supervise noisy ones.
3. State-of-the-Art Performance: Demonstrates that explicitly handling noise leads to superior cross-domain generalization without compromising source-domain accuracy.

4. Experimental Results

The authors evaluated SeeTN on four datasets: ETH-XGaze, Gaze360, MPIIFaceGaze, and EyeDiap, using ResNet-18 and ResNet-50 backbones.

• Quantitative Performance:
  • Cross-Domain: SeeTN significantly outperforms the baseline (ResNet-18) and other state-of-the-art domain generalization methods (e.g., PureGaze, GazeCF, AGG).
    • Example: On the DG $\to$ DM (Gaze360 to MPIIFaceGaze) setting, SeeTN reduced the Mean Angular Error (MAE) from 7.94° (Baseline) to 6.57°, a 17.2% improvement.
  • Within-Domain: Unlike many generalization methods that trade off source accuracy for target performance, SeeTN also improved within-domain accuracy (e.g., reducing error by 3–5% on source domains).
• Comparison with NLL Methods: SeeTN outperformed existing Noisy Label Learning methods adapted for this task (DivideMix, SUGE), which struggled with the regression nature of the problem or within-domain constraints.
• Ablation Studies:
  • Removing the noise-robust regularization ($L_{align}^N$) caused a significant performance drop, proving the value of utilizing noisy samples via clean supervision.
  • The proposed indicator $\eta$ was more effective than standard small-loss criteria ($L_1$) for identifying noise.
• Qualitative Analysis:
  • t-SNE Visualization: Showed that the baseline model clustered features by head pose, whereas SeeTN clustered features by gaze direction, indicating successful learning of domain-invariant semantic information.
  • Noise Detection: Visualizations confirmed that high $\eta$ values correlated with incorrect ground-truth labels, while the model could still predict the correct gaze direction for these samples.

5. Significance

This paper highlights a critical, often overlooked bottleneck in gaze estimation: annotation noise. By shifting the focus from purely learning domain-invariant features to actively mitigating and leveraging label noise, the authors demonstrate that robustness to noise is a prerequisite for effective domain generalization. The SeeTN framework provides a new paradigm for regression-based computer vision tasks where ground truth is inherently noisy, offering a path toward more reliable real-world deployment of gaze tracking systems.