💻 computer science

Domain-Agnostic Feature Modulation for Semi-Supervised Domain Generalization

This paper addresses the challenging domain-label agnostic semi-supervised domain generalization task by proposing a feature modulation strategy to create robust, domain-invariant representations and a dynamic loss-scaling function to mitigate domain noise and improve pseudo-label accuracy, achieving significant performance gains on major benchmarks without relying on domain labels.

Original authors: Venuri Amarasinghe (University of Moratuwa), Kalinga Bandara (University of Moratuwa), Isun Randila (University of Moratuwa), Asini Jayakody (University of Moratuwa), Chamuditha Jayanga Galappaththige

Published 2026-05-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Venuri Amarasinghe (University of Moratuwa), Kalinga Bandara (University of Moratuwa), Isun Randila (University of Moratuwa), Asini Jayakody (University of Moratuwa), Chamuditha Jayanga Galappaththige (Queensland University of Technology), Ranga Rodrigo (University of Moratuwa)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "New City" Challenge

Imagine you are training a robot to recognize animals. You show it thousands of pictures of dogs and cats taken in a sunny park (Source Domain). The robot learns perfectly.

But then, you take the robot to a dark, rainy forest (Target Domain) to test it. The lighting is different, the trees look different, and the animals look different. The robot gets confused. It might think a wet dog is a cat because the "rainy forest" features are messing up its brain.

This is called Domain Shift. In the real world, we often don't have perfect, labeled data for every new environment (like the rainy forest). We have a little bit of labeled data and a lot of unlabeled data (pictures without names).

The goal of this paper is to teach a robot to handle these new, unseen environments using mostly unlabeled data, without needing to know exactly which environment (domain) each picture came from.

The Two Main Hurdles

The authors identified two big problems with current methods:

The "Guessing Game" (Pseudo-Labeling): Since we don't have labels for the new data, the robot has to guess the labels (e.g., "I'm 90% sure this is a dog"). These guesses are called Pseudo-Labels. If the robot is confused by the "rainy forest" background, it makes bad guesses. If you teach the robot using bad guesses, it gets worse.
The "Too Strict" Rule: To avoid bad guesses, current methods are very strict. They only let the robot learn from guesses where it is 95% sure. This means they throw away 90% of the unlabeled data because the robot isn't confident enough. It's like a teacher who only lets students study the easiest questions and ignores the rest.

The Solution: A Two-Part Magic Trick

The authors propose a new method called Domain-Agnostic Feature Modulation. Think of it as a two-step magic trick to fix the robot's brain.

Step 1: The "Blender" (Feature Modulation)

The Analogy: Imagine you are trying to teach someone to recognize a Dog.

The Problem: In the "Photo" world, dogs have fur. In the "Sketch" world, dogs are just lines. If the robot focuses on "fur," it fails in the sketch world. If it focuses on "lines," it fails in the photo world.
The Fix: The authors created a Feature Modulator. Think of this as a blender.
- They take the specific features of a picture (the fur, the lines).
- They mix it with a "Similar Average Representation" (SAR). Imagine the SAR is a "Super-Average Dog" created by blending the best parts of dogs from all different worlds (photos, sketches, cartoons) together.
- The blender dials down the parts that change too much between worlds (like the background or lighting) and dials up the parts that stay the same (the shape of the snout).
The Result: The robot stops looking at the "rainy forest" background and starts focusing on the "dog shape." This makes the robot's guesses (pseudo-labels) much more accurate.

Step 2: The "Smart Volume Knob" (Loss Scaling)

The Analogy: Now that the robot is making better guesses, the authors realized they could be less strict.

The Old Way: "Only listen to guesses where you are 95% sure." (This ignores a lot of data).
The New Way: They introduced a Loss Scaling function. Imagine a volume knob.
- If the robot is very sure (95%), the volume is loud (high weight).
- If the robot is somewhat sure (75%), the volume is lower, but still audible.
- Crucially, they also check for "uncertainty" (how shaky the robot's hand is). If the robot is shaky, they turn the volume down even more to prevent it from learning the wrong thing.
The Result: The robot can now learn from more data (including the "somewhat sure" guesses) without getting confused by the noise.

Why This Matters (The Results)

The authors tested this on four major datasets (like PACS, OfficeHome, etc.), which are like different "universes" of images.

No Domain Labels Needed: Unlike other methods that require a label saying "This is from the Photo World," this method works blindly. It doesn't care where the data comes from.
Better Accuracy: By cleaning up the features (Step 1) and using more data wisely (Step 2), their method beat the previous best methods (like FixMatch) by a significant margin (about 3-6% improvement).
The "Keep Rate" Win: They managed to keep more data in the training loop (higher "Keep Rate") while still keeping the guesses accurate. It's like getting a student to study 20% more questions without them getting confused.

Summary in One Sentence

The paper teaches a computer vision model to ignore the "background noise" of different environments and focus on the core object, allowing it to learn from a much larger pool of unlabeled data without needing to know exactly where that data came from.

Technical Summary: Domain-Agnostic Feature Modulation for Semi-Supervised Domain Generalization

Problem Statement
Semi-Supervised Domain Generalization (SSDG) aims to improve model generalization to unseen target domains by leveraging a small fraction of labeled data alongside abundant unlabeled data. While existing SSDG methods often rely on pseudo-labeling (PL) for unlabeled data, they frequently assume access to domain labels during training—a privilege not always available in real-world scenarios. Furthermore, domain shifts introduce "domain noise," leading to inconsistent pseudo-labels that degrade model performance. Methods derived from semi-supervised learning baselines often suffer from low pseudo-label accuracy, limiting the effective utilization of unlabeled data. A fundamental dilemma exists: lowering the confidence threshold to utilize more unlabeled data increases noise, while maintaining a high threshold discards valuable data. This paper addresses the domain-label agnostic SSDG setting, where domain labels for unlabeled data are unavailable during training.

Methodology
The proposed approach introduces a novel framework comprising four key components: Similar Average Representations (SAR), a Feature Modulator, Pseudo-Labeling using modulated features, and a Loss Scaling function with uncertainty estimation.

Similar Average Representations (SAR):
To mitigate the impact of domain-specific information, the authors define class prototypes ( $P_c$ ) based solely on class labels, making them domain-agnostic. Unlike methods that average features across domains using domain labels, this approach computes SAR ( $R_c$ ) by weighting class prototypes based on their cosine similarity. This creates a representation that emphasizes class-specific signals while suppressing domain-related variations (e.g., background, texture). SARs serve as a robust target for feature modulation.
Feature Modulation:
A learnable feature modulator matrix ( $M$ ) is introduced to blend instance-specific features ( $z$ ) with SARs ( $R$ ). The modulated feature $Z_m$ is calculated as:
$Z_m = M \odot Z + (1 - M) \odot R$
The matrix $M$ is initialized based on the variance of features within each class (features with high variance, likely domain-specific, are down-weighted) and is updated during training via a diagonal loss function. This process shifts features toward their SARs, encouraging the feature extractor to form tightly clustered, domain-invariant representations and forcing the classifier to distinguish between closely related classes by focusing on unique class-specific features rather than shared or domain-specific ones.
Pseudo-Labeling with Modulated Features:
To generate more accurate pseudo-labels, the features of unlabeled samples are modulated toward each class's SAR before passing them through the classifier. The class yielding the highest confidence score on these modulated features is assigned as the pseudo-label. This strategy reduces domain interference during the labeling process, resulting in higher-quality pseudo-labels.
Uncertainty Estimation and Loss Scaling:
To address the trade-off between data utilization and label noise, the authors employ Monte Carlo (MC) dropout to estimate the uncertainty of pseudo-labels. They introduce a dynamic loss scaling function ( $l_{scale}$ ) that adjusts the contribution of each pseudo-label based on its confidence ( $p_{max}$ ) and uncertainty ( $\sigma$ ). This allows the model to incorporate lower-confidence pseudo-labels in a controlled manner, increasing the "keep rate" (the proportion of unlabeled data used) without significantly introducing noise.

Key Contributions

Feature Modulator: A strategy that enhances class-discriminative features and suppresses domain-specific information without relying on domain labels, leading to more accurate pseudo-labels.
Loss Scaling Function: A mechanism that dynamically adjusts the influence of pseudo-labels based on confidence and uncertainty, optimizing the utilization of unlabeled data while mitigating label noise.
Domain-Label Agnostic Setting: The method addresses the SSDG problem in a more challenging setting where domain labels are unavailable, demonstrating effectiveness across four major benchmarks.

Experimental Results
The method was evaluated on four standard Domain Generalization benchmarks: PACS, OfficeHome, VLCS, and DigitsDG, under 5-label and 10-label settings.

Performance: The proposed method (FM) achieved consistent improvements over the strong FixMatch baseline, with average accuracy increases of 3.3% (10-label setting) and 4.3% (5-label setting) across datasets.
Specific Gains: Notable improvements were observed in OfficeHome (57.2% and 61.0% for 5 and 10 labels), VLCS (75.5% and 75.3%), and DigitsDG (73.0% for 10 labels).
Component Analysis: Ablation studies confirmed that the feature modulator improves pseudo-label accuracy, while the loss scaling function significantly increases the keep rate. The combination of both components yielded the best balance between accuracy and data utilization.
Comparison: The method outperformed other SOTA SSDG methods (e.g., StyleMatch, FBCSA, DGWM), many of which rely on domain labels and fail to perform well in the domain-label agnostic setting.

Significance
The paper claims to resolve the fundamental dilemma in SSDG between achieving accurate pseudo-labels and maximizing unlabeled data utilization. By shifting features toward Similar Average Representations, the model learns to distinguish closely clustered classes and form domain-invariant representations without explicit domain supervision. The dynamic loss scaling further enables the effective use of lower-confidence pseudo-labels, which are typically discarded. The authors assert that these innovations allow the model to generalize effectively to unseen domains even in the absence of domain labels, a setting that better reflects real-world constraints.