🤖 machine learning

SloMo-Fast: Slow-Momentum and Fast-Adaptive Teachers for Source-Free Continual Test-Time Adaptation

The paper introduces SloMo-Fast, a source-free continual test-time adaptation framework that employs complementary slow and fast teachers to effectively balance long-term knowledge retention with rapid domain adaptation, thereby outperforming existing methods in handling evolving and recurring domain shifts.

Original authors: Md Akil Raihan Iftee, Mir Sazzat Hossain, Rakibul Hasan Rajib, Tariq Iqbal, Md Mofijul Islam, M Ashraful Amin, Amin Ahsan Ali, AKM Mahbubur Rahman

Published 2026-07-20

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Md Akil Raihan Iftee, Mir Sazzat Hossain, Rakibul Hasan Rajib, Tariq Iqbal, Md Mofijul Islam, M Ashraful Amin, Amin Ahsan Ali, AKM Mahbubur Rahman

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to recognize animals. You show it thousands of pictures of cats and dogs in a sunny park, and it learns perfectly. But then, you take the robot out into the real world. Suddenly, the lighting changes, it starts raining, the camera gets blurry, or the animals are wearing costumes. The robot, trained only on sunny park photos, gets confused and starts making mistakes. This is the problem of "domain shift."

To fix this, scientists use a trick called Test-Time Adaptation (TTA). Instead of retraining the robot from scratch (which takes forever and needs the original training data), the robot learns on the fly while it is working. It looks at the new, confusing pictures, guesses what they are, and tweaks its own brain slightly to get better at that specific moment. However, there's a catch: if the robot keeps changing its mind too fast to适应 (adapt) to a rainy day, it might completely forget how to recognize a dog in the sunny park. This is called "catastrophic forgetting." The big challenge is finding the perfect balance: changing fast enough to handle new weather, but slow enough to remember the old lessons.

This is where a new method called SloMo-Fast comes in. The researchers behind this paper propose a clever team of three "brains" working together to solve this balancing act without needing any of the original training data.

The Problem: The Forgetful Robot

In the real world, environments don't just change once; they change constantly and often repeat. A self-driving car might see rain, then fog, then snow, and then rain again. Existing methods for helping robots adapt usually fail in two ways. Some are too slow to catch up when the weather suddenly changes, leading to a pile-up of mistakes. Others are too fast; they adapt so quickly to the current rain that they wipe their memory of how to handle fog, causing them to crash when the fog returns. Furthermore, most of these methods require access to the original "source" data (the sunny park photos) to help them remember, which is often impossible due to privacy laws or storage limits.

The Solution: A Dual-Teacher Team

The authors of this paper, Md Akil Raihan Iftee and his team, introduce SloMo-Fast (Slow-Momentum and Fast-Adaptive). Imagine a classroom with three students: a Student, a Fast Teacher, and a Slow Teacher.

The Fast Teacher (The Sprinter): This teacher is great at reacting immediately. When the robot sees a new type of corruption (like heavy snow), the Fast Teacher quickly adjusts its understanding to handle it. It's like a sprinter who can instantly change direction. However, sprinters often forget the long-term strategy.
The Slow Teacher (The Marathon Runner): This teacher is very patient. It changes its mind very gradually. It doesn't care about the immediate snow; it cares about remembering the general shape of a "car" or a "dog" across all weather conditions. It acts as a stable anchor, ensuring the robot doesn't forget what it learned yesterday.
The Student: This is the main robot doing the work. It listens to both teachers. It learns from the Fast Teacher to handle the immediate chaos, but it also listens to the Slow Teacher to make sure it doesn't lose its long-term memory.

How They Remember Without a Textbook

Usually, to remember old lessons, you need a textbook (the original source data). SloMo-Fast doesn't have one. Instead, it uses a special trick called Prototype Generation.

Imagine the Fast Teacher is sorting a pile of mixed-up puzzle pieces. It picks out only the pieces that it is very sure about (high confidence) and that stay the same even if you wiggle them a little (stable). It puts these "good" pieces into a special box for each category (a box for "cars," a box for "dogs"). These boxes become the Prototypes.

The Slow Teacher then looks at these boxes. It uses them to learn a "generalized" version of what a car or dog looks like, regardless of the weather. This way, the robot builds its own memory bank from the current data, without ever needing to see the original training photos again.

The Results: Winning the Race

The researchers tested this team on 11 different challenging scenarios, including datasets where images were corrupted with noise, blur, and weather effects (like CIFAR-10-C and ImageNet-C). They also created a new test called Cyclic-TTA, where the weather patterns repeat in cycles (Rain → Fog → Snow → Rain), specifically designed to see if the robot remembers the rain when it comes back.

The results were impressive. SloMo-Fast consistently outperformed the best existing methods.

Across all tests, it reduced the average error rate to 34.7%, beating the previous best methods by at least 1.5%.
In the "Cyclic" test where domains repeat, it showed it could remember past lessons much better than others, avoiding the "catastrophic forgetting" that plagues other systems.
It was also efficient. One version of SloMo-Fast only updated about 4.9% of its parameters (mostly just the "batch normalization" layers), making it very fast and lightweight, while still beating methods that updated 100% of their parameters.

What They Ruled Out

The paper explicitly argues against the idea that you need to store the original source data or prototypes to adapt well. They show that relying on source data is often impractical in real-world privacy-sensitive fields like healthcare or autonomous driving. They also demonstrate that methods which simply reset the model or rely on a single teacher often fail when domains repeat, leading to a loss of performance over time.

The Bottom Line

SloMo-Fast suggests that the key to a robust, adaptable AI isn't just one brain changing fast or slow, but a team where a "Fast" teacher handles the immediate chaos and a "Slow" teacher preserves the long-term wisdom. By generating its own memory from the data it sees right now, this method allows AI to navigate a changing world without forgetting its past, all while respecting privacy and saving storage space. It's a step toward robots that can truly learn and adapt in the messy, unpredictable real world.

Technical Summary: SloMo-Fast for Source-Free Continual Test-Time Adaptation

1. Problem Statement

The paper addresses the challenges of Continual Test-Time Adaptation (CTTA), a setting where models must adapt online to a sequence of evolving, unlabeled target domains without access to source data. While standard Test-Time Adaptation (TTA) handles single domain shifts, CTTA faces three critical hurdles in real-world deployment (e.g., autonomous driving, healthcare):

Source-Free Constraints: Many existing methods rely on source data or prototypes stored in memory banks, which is often infeasible due to privacy concerns, storage limits, or transmission constraints.
Catastrophic Forgetting & Error Accumulation: As models adapt to new domains, they often forget previously learned knowledge. Furthermore, slow adaptation rates during domain transitions can lead to the accumulation of errors, causing model collapse before effective adaptation occurs.
Recurring Domain Shifts: Real-world environments often exhibit cyclic patterns (e.g., day/night cycles, seasonal weather changes). Existing methods struggle to retain long-term knowledge of previously encountered domains when they reappear, often requiring re-adaptation from scratch or suffering from performance degradation.

Current state-of-the-art methods often fail to balance the trade-off between Adaptation Rate (speed of recovery on new domains) and Forgetting Ratio (retention of knowledge on past domains), particularly in cyclic or long-sequence scenarios.

2. Methodology: SloMo-Fast

The authors propose SloMo-Fast, a source-free, dual-teacher framework designed to enhance quick adaptability and generalization without requiring source data. The architecture consists of a Student model ( $S$ ) and two complementary Teacher models: a Fast-Teacher ( $T_1$ ) and a Slow-Teacher ( $T_2$ ).

Core Components

Dual-Teacher Architecture:
- Fast-Teacher ( $T_1$ ): Updated via Exponential Moving Average (EMA) of the Student's weights. It adapts rapidly to the current domain to generate reliable pseudo-labels and features.
- Slow-Teacher ( $T_2$ ): Updated more gradually to ensure robust generalization and long-term knowledge retention. It learns domain-invariant features and provides stable guidance.
- Student ( $S$ ): Updated using symmetric cross-entropy (SCE) loss against predictions from both teachers to leverage their complementary strengths.
Test-Time Reliable Prototype Generation:
- Instead of relying on source prototypes, SloMo-Fast generates class-specific prototypes dynamically from unlabeled target data.
- Dual-Criterion Filtering: Features from $T_1$ $T_{1}$ are stored in class-specific priority queues only if they meet two criteria:
  1. Confidence: Low prediction entropy.
  2. Sensitivity: High prediction stability (measured by Pseudo-Label Probability Difference, PLPD) under data augmentations.
- Prototype Construction: Class prototypes are computed as weighted averages of these reliable features, where weights are inversely proportional to entropy.
Training Objectives for Slow-Teacher ( $T_2$ ):
$T_2$ is optimized using a combination of three losses to learn generalized representations:
1. Contrastive Loss ( $L_{CL}$ ): Aligns features from $T_2$ with the generated class prototypes, separating different classes while pulling same-class features together.
2. Mean Squared Error ( $L_{MSE}$ ): Aligns $T_2$ 's features directly with the corresponding class prototypes.
3. Information Maximization ( $L_{IM}$ ): Ensures $T_2$ maintains both prediction certainty and class diversity to avoid collapse.
- Stochastic Restoration: To mitigate catastrophic forgetting, $T_2$ employs a stochastic restoration method that blends updated weights with the original source model weights after each gradient step.
Prediction Ensembling & Calibration:
- Final predictions are an ensemble of the Student and $T_2$ .
- Dynamic Label Calibration (DLC): Adjusts the predictive posterior to account for shifts in label distributions between source and target domains by estimating the target prior online.

3. Key Contributions

SloMo-Fast Framework: A novel dual-teacher CTTA method that eliminates the need for source data while simultaneously enhancing adaptability (via $T_1$ ) and generalization/retention (via $T_2$ ).
Source-Free Prototype Generation: A method for generating class-wise prototypes directly from unlabeled target data using a dual-criterion filtering strategy (entropy and sensitivity) to ensure reliability.
Cyclic-TTA Benchmark: The introduction of a new benchmark setting where domain shifts repeat over time (cyclic), specifically designed to evaluate a model's ability to retain knowledge of recurring domains, a scenario often overlooked in existing benchmarks like CCC (Continually Changing Corruptions).
Performance: The method achieves a mean error rate of 34.7% across 11 diverse CTTA scenarios on 5 datasets (CIFAR-10-C, CIFAR-100-C, ImageNet-C, ImageNet-R, ImageNet-Sketch), outperforming state-of-the-art methods by at least 1.5%.

4. Experimental Results

The paper evaluates SloMo-Fast against baselines (TENT, CoTTA, RoTTA, ROID, etc.) across various settings:

General CTTA: SloMo-Fast consistently outperforms baselines in Continual, Mixed, Gradual, and Episodic settings.
Cyclic-TTA: In scenarios where domains repeat, SloMo-Fast demonstrates superior stability and lower forgetting ratios compared to methods that suffer from catastrophic collapse or require re-adaptation.
Efficiency: The lightweight variant (updating only Batch Normalization layers, 4.9% parameters) achieves performance comparable to or better than full-parameter methods like ROID, with lower inference time.
Robustness: Ablation studies confirm that removing any component (Contrastive Loss, MSE, Information Maximization, DLC, or Stochastic Restoration) degrades performance. The method also shows resilience to noisy pseudo-labels and teacher disagreement.

5. Significance and Claims

The paper claims that SloMo-Fast addresses the critical gap in CTTA regarding long-term knowledge retention in the absence of source data. By decoupling the adaptation speed (Fast-Teacher) from the generalization stability (Slow-Teacher) and utilizing dynamically generated prototypes, the framework enables models to:

Adapt quickly to new, unseen domains.
Retain knowledge of previously encountered domains, even when they reappear in cyclic patterns.
Operate effectively in privacy-sensitive and resource-constrained environments where source data is unavailable.

The authors emphasize that their approach offers a practical solution for real-world deployment where environments evolve continuously and unpredictably, providing a robust trade-off between plasticity (adapting to new data) and stability (remembering old data).