💻 computer science

Cross-Platform Domain Adaptation for Multi-Modal MOOC Learner Satisfaction Prediction

This paper introduces ADAPT-MS, a cross-platform domain adaptation framework that effectively predicts multi-modal MOOC learner satisfaction on target platforms with limited or no labels by integrating frozen LLM text encoding, domain-adversarial alignment, latent-variable rating calibration, and gated modality fusion.

Original authors: Jakub Kowalski, Magdalena Piotrowska

Published 2026-04-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jakub Kowalski, Magdalena Piotrowska

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a teacher who has spent years grading essays and watching students in a specific classroom (let's call it Classroom A). You know exactly how the students in this room talk, what kind of homework they turn in, and how they use the 1-to-5 star rating system on your feedback forms. You've built a super-smart AI assistant that predicts how happy these students are based on their essays and their homework habits.

Now, your boss says, "Great! Take that AI and put it in Classroom B and Classroom C."

You try it, but it fails miserably. Why?

The Language is Different: Students in Classroom B write in a slangier, more casual way.
The Students are Different: Classroom C has mostly engineers, while Classroom A has mostly artists.
The Homework Logs are Broken: Classroom B doesn't track "video rewinds" like Classroom A does.
The Rating System is Broken: In Classroom A, a 5-star means "Perfect." In Classroom C, students give 5 stars just for showing up. If your AI sees a 5-star rating, it thinks "Perfect!" in Classroom A, but in Classroom C, it's just "Average."

This is the problem the paper solves. The authors built a new system called ADAPT-MS (think of it as a "Universal Translator and Calibrator") that allows your AI to work perfectly in any new classroom, even if it has never seen a single student from that room before.

Here is how ADAPT-MS works, broken down into four simple tricks:

1. The "Universal Dictionary" (Text & Behavior)

Instead of teaching the AI to memorize specific words like "awesome" or "terrible," the system uses a frozen, pre-trained brain (a Large Language Model) that already understands human language deeply. It doesn't care if the student writes "This course rocks!" or "The pedagogy is exemplary"; it understands the feeling behind both.

For the homework logs (behavior), the system ignores the messy, specific details of each school. Instead, it translates everything into a "Canonical Vocabulary"—a standard list of 6 universal behaviors (like "minutes watched," "forum posts," "days active"). It's like converting every country's currency into a single "Universal Coin" so the AI can compare them easily.

2. The "Blindfolded Detective" (Domain Adversarial Training)

This is the coolest part. The system has a "Detective" whose job is to guess which classroom a student is from. But here's the twist: the main AI tries to trick the Detective.

The main AI is trained to make its predictions so good that the Detective cannot tell if the student is from Classroom A, B, or C. If the AI succeeds, it means it has learned the universal rules of student satisfaction, ignoring the specific "flavor" of each school. This forces the AI to focus on what actually matters, not on the school's quirks.

3. The "Rating Translator" (Calibration)

Remember how Classroom C gives out 5 stars too easily? The AI might predict a student is "Very Happy" (a 5) when they are actually just "Okay" (a 3).

ADAPT-MS adds a Calibration Layer. Think of this as a translator for numbers.

If the AI predicts a "5" in Classroom C, the translator knows, "Ah, in this room, a 5 actually means a 3.5."
It adjusts the final score based on the school's history. It's like realizing that in one culture, a nod means "yes," but in another, it means "maybe," and adjusting your interpretation accordingly.

4. The "Safety Net" (Handling Missing Data)

Sometimes, the new classroom (Target) doesn't have the same homework logs as the old one. Maybe they don't track "video rewinds."

ADAPT-MS has a Gated Fusion system. Imagine a smart switch. If the "video rewind" data is missing, the switch automatically turns off that part of the AI and relies only on the student's written reviews. It's like a chef who can cook a delicious meal even if they are missing one specific spice; they just adjust the recipe slightly and keep going.

The Results: Why It Matters

The researchers tested this on three massive online learning platforms (like Coursera, edX, and XuetangX).

Without ADAPT-MS: The AI was confused and made big mistakes (high error rate).
With ADAPT-MS: The AI became accurate, even when it had zero labeled data from the new school.
The "Few-Shot" Magic: Even if they gave the AI just 50 examples of students from the new school to "tune" the rating translator, the system became incredibly accurate.

The Big Picture

This paper is about adaptability. In the real world, you can't build a new AI for every single school or company. You need one smart system that can walk into a new environment, figure out the local dialect and rating habits, and start working immediately.

ADAPT-MS is that system. It's the difference between bringing a rigid, rule-bound robot to a new country and bringing a flexible, culturally-aware diplomat who can learn the ropes in minutes.

1. Problem Definition

The paper addresses the challenge of predicting learner satisfaction in Massive Open Online Courses (MOOCs) when deploying models trained on one platform (Source) to another (Target). While existing models perform well within a single platform, they suffer significant performance degradation when transferred due to cross-platform domain shifts.

The authors identify four specific axes of shift:

Linguistic Style Shift: Differences in vocabulary, formality, and cultural sentiment expression in review texts.
Population and Course Mix Shift: Variations in learner demographics and subject distributions.
Behavioral Logging Schema Shift: Inconsistent definitions or granularities of interaction logs (e.g., "video view") across platforms.
Rating Scale Semantics Drift: Systematic bias in how users utilize rating scales (e.g., some platforms use 5 stars as a default positive, while others reserve it for perfection).

The goal is to learn a predictor $f$ that minimizes error on the target platform $T$ using labeled source data and potentially unlabeled (or few-labeled) target data.

2. Methodology: ADAPT-MS

The authors propose ADAPT-MS (Adaptive Domain-Transfer Multi-modal Satisfaction predictor), a unified framework designed to handle representation alignment, label calibration, and missing modalities. The architecture consists of four key components:

A. Multi-Modal Encoder with Canonical Features

Text Encoder: Uses a frozen pre-trained LLM (RoBERTa-base) to encode review text. Freezing prevents overfitting to source-platform vocabulary while retaining general linguistic representations.
Canonical Behavioral Features: To handle schema shifts, raw logs are mapped to a platform-agnostic vector of six aggregates: minutes watched, quiz attempts, forum reads, forum posts, active days, and video rewatch rate. These are processed by a two-layer MLP.
Gated Fusion with Modality Dropout: A gating mechanism ( $\alpha_i$ ) dynamically weights the behavioral embedding based on availability. During training, modality dropout (randomly zeroing behavioral inputs) forces the model to learn robust text-only representations, ensuring stability when behavioral logs are missing in the target domain.

B. Domain-Adversarial Representation Alignment

To learn domain-invariant features, the framework employs a Domain Discriminator trained to predict the platform source. A Gradient Reversal Layer (GRL) is inserted between the encoder and discriminator. During backpropagation, gradients are multiplied by $-\lambda$ , encouraging the encoder to generate features that confuse the discriminator, thereby aligning the source and target distributions.

C. Platform-Specific Rating Calibration

Even with aligned representations, label semantics drift causes systematic bias. ADAPT-MS introduces a latent-variable calibration layer that applies a platform-specific affine transformation ( $\hat{y}_i = a_p s_i + b_p$ ) to the predicted satisfaction score.

Supervised Calibration: With labeled target data, $(a_T, b_T)$ are optimized jointly with a small learning rate.
Unsupervised Calibration: With zero labels, parameters are estimated via moment matching, aligning the mean and standard deviation of target predictions with historical target rating statistics (obtainable from platform metadata).

3. Key Contributions

Problem Quantification: Identified and quantified four distinct sources of cross-platform shift (feature distribution, logging gaps, population mix, and rating semantics).
Framework Design: Proposed ADAPT-MS, which uniquely combines domain-adversarial alignment, label calibration, and missing-modality robustness in a single architecture.
Rigorous Evaluation: Established a cross-platform evaluation protocol covering unsupervised, few-shot, and pairwise transfer scenarios, demonstrating consistent improvements over strong baselines.

4. Experimental Results

Experiments were conducted on a dataset spanning three major platforms (A, B, C) with 480,000 enrollments.

Unsupervised Transfer (Zero Labels):
- ADAPT-MS achieved an RMSE of 0.66 on the target platform.
- This represents a 16% relative reduction in RMSE compared to a naive pooling baseline (0.76) and a 5.7% reduction over a domain-adversarial baseline without calibration (0.70).
- The calibration layer was identified as the primary driver for correcting the systematic over-prediction bias caused by rating scale differences.
Few-Shot Adaptation:
- With only 50 labeled target samples, ADAPT-MS achieved an RMSE of 0.63.
- With 1,000 samples, it reached 0.60, outperforming full fine-tuning (which collapsed to 0.79 with 0 labels and required more data to converge).
- Updating only the lightweight calibration parameters proved more data-efficient and stable than fine-tuning the entire network.
Ablation Studies:
- Removing the calibration layer increased RMSE by 0.05.
- Removing domain-adversarial alignment increased RMSE by 0.08.
- Removing modality dropout/gating increased RMSE by 0.04, highlighting its importance for platforms with high missing-data rates (40% on Platform C).

5. Significance and Impact

Practical Deployment: The framework enables MOOC platforms to onboard new learners or migrate to new systems without requiring massive amounts of labeled target data, solving a critical bottleneck in learning analytics.
Robustness to Data Scarcity: The ability to achieve high accuracy with zero or very few target labels makes the approach viable for real-world scenarios where target data is scarce or unlabeled.
Handling Heterogeneity: By explicitly modeling rating scale drift and missing modalities, ADAPT-MS offers a more realistic solution for cross-platform analytics than standard transfer learning methods.
Future Directions: The authors suggest extending the work to multilingual settings, handling temporal concept drift, and integrating fairness auditing to ensure calibration does not disadvantage underrepresented groups.

In conclusion, ADAPT-MS provides a state-of-the-art solution for cross-platform satisfaction prediction, demonstrating that combining adversarial alignment with explicit label calibration is essential for overcoming the complex heterogeneity of MOOC ecosystems.