Stress Detection in Digital Assessment Environments: A Multimodal Wearable Analysis by Language Background

Technical Summary: Stress Detection in Digital Assessment Environments by Language Background

Problem Statement
In educational assessment, a student's background—specifically language proficiency and cultural familiarity—can significantly influence their experience and performance. When an exam is administered in a language different from a student's home language, it introduces cognitive and emotional demands, such as increased processing effort and slower comprehension. These demands can generate context-specific stress that distorts performance, threatening the construct validity of assessment scores. While wearable technology has enabled real-time stress monitoring, few studies have investigated how stress responses vary across specific student subgroups, particularly between English Learners (EL) and English Speakers (ES) in digital assessment environments. Existing research often treats students as a homogeneous group, failing to account for how linguistic and cultural factors shape stress responses.

Methodology
The study employed a controlled, multimodal user study involving 20 college students (10 EL and 10 ES) participating in a simulated digital assessment. The assessment covered three subject areas: reading comprehension, history, and science, all administered in English with embedded linguistic and cultural challenges.

• Data Collection:
  • Physiological Data: Collected via the Embrace Plus wristband, capturing Electrodermal Activity (EDA), Blood Volume Pulse (BVP), Skin Temperature (ST), and Interbeat Interval (IBI).
  • Behavioral Data: Captured via the Qualtrics assessment platform, including clickstream data (number of clicks per question), response times, time limit violations, and section performance scores.
  • Ground Truth: Self-reported stress levels were collected via a 3-item, 5-point Likert scale questionnaire administered after each section, focusing on language, cultural context, and time pressure.
• Data Preprocessing:
  • Physiological signals were filtered (e.g., low-pass filtering for EDA, Winsorization for BVP) and synchronized.
  • Features were extracted using a 5-second sliding window with a 1-second overlap. Statistical features (min, max, mean, std) were computed for physiological signals.
  • Behavioral features included total clicks, total duration, number of questions exceeding time limits, and relative performance percentages.
  • Ground truth labels ("high" vs. "low" stress) were generated by binarizing self-reported scores based on group-wise medians.
• Machine Learning Approach:
  • Five models were evaluated: Support Vector Classifier (SVC), Gaussian Naive Bayes (NB), k-Nearest Neighbors (KNN), Decision Tree (DT), and XGBoost (XGB).
  • Two experimental settings were tested: using only physiological features (P) and using a combined set of physiological and task-related behavioral features (P+T).
  • Performance was evaluated using 10-fold cross-validation with metrics including Precision, Recall, F1-score, and Accuracy.
  • SHAP (Shapley Additive Explanations) values were used to interpret feature importance and compare predictive patterns between EL and ES groups.

Key Results

• Statistical Differences: Mann-Whitney U tests revealed significant differences in self-reported stress between EL and ES groups in the reading ($p < .01$) and history ($p < .01$) sections, but not in the science section. Follow-up analysis indicated that cultural factors drove stress differences in reading, while language factors were the primary driver in history. Effect sizes were medium to large.
• Model Performance:
  • Models using only physiological data (P) achieved moderate performance, with SVC reaching the highest accuracy of 69%.
  • Integrating behavioral data (P+T) significantly improved performance across all models, with gains of up to 20%.
  • XGBoost (XGB) achieved the best overall performance with the P+T feature set, reaching an F1-score of 87% and Accuracy of 88%.
  • The inclusion of behavioral features notably reduced misclassification of "high" stress instances, which is critical for real-world intervention.
• Feature Importance: SHAP analysis identified Total Clicks as the most predictive feature for both groups, followed by Total Duration and Performance. While 13 of the top 15 features were common to both groups, subtle differences were observed (e.g., phasic EDA features were more prominent for EL, while BVP standard deviation was unique to ES). Generally, higher click counts and lower performance correlated with higher predicted stress.

Significance and Contributions
The authors claim that this study contributes to the fields of ubiquitous computing and Human-Computer Interaction (HCI) by demonstrating the feasibility of using wearable-based multimodal systems to detect stress patterns specific to language backgrounds. The primary contributions include:

1. Context-Specific Stress Detection: Providing empirical evidence that stress responses in digital assessments differ significantly between EL and ES students, particularly in language-heavy and culturally specific tasks.
2. Multimodal Efficacy: Demonstrating that combining physiological signals with fine-grained behavioral traces (clicks, timing) yields significantly more accurate stress detection than physiological data alone.
3. Design Implications: Offering a data-driven basis for designing human-centered educational technologies. The authors suggest that these findings can inform the development of AI-driven assessment tools capable of providing personalized support (e.g., simplifying instructions or offering hints) when elevated stress is detected, thereby improving the fairness and validity of digital assessments.

The paper maintains a modest tone, acknowledging limitations such as the controlled study environment, the use of a brief self-report questionnaire, and the need for future longitudinal research to understand stress evolution over time. It does not propose immediate deployment but rather frames the findings as a foundational step toward more adaptive and supportive educational technologies.