Prediction of Challenging Behaviors Associated with Profound Autism in a Classroom Setting Using Wearable Sensors

Technical Summary: Prediction of Challenging Behaviors Associated with Profound Autism in a Classroom Setting Using Wearable Sensors

Problem Statement
Approximately one-quarter of individuals with Autism Spectrum Disorder (ASD) are classified as having "profound autism," a subgroup prone to frequent and intense episodes of challenging behaviors such as self-injurious behavior (SIB), aggression, and agitation. These behaviors pose significant safety risks and disrupt learning. Current monitoring relies on time-intensive, subjective human observation or video review, which lacks the capacity to detect the subtle physiological precursors that often precede behavioral episodes. While wearable sensors have been applied to behavior detection, prior work has largely been confined to controlled laboratory environments, focused on retrospective detection rather than proactive prediction, and limited by single-modality data or shallow architectures. There is a critical need for scalable, objective, and context-aware tools capable of predicting challenging behaviors in real-world special education classrooms to enable proactive intervention.

Methodology
The study was conducted at The Center for Discovery (TCFD) involving nine male participants (aged 10–21) with profound autism. Data collection occurred during standard classroom sessions using Q-Sensor bands (Affectiva Inc.) worn on the wrist or ankle, recording tri-axial accelerometry, electrodermal activity (EDA), and skin temperature at 30 Hz.

• Data Annotation: Trained behavior analysts annotated video footage to label specific challenging behaviors, which were grouped into three high-level categories: Aggression, Self-Injurious Behavior (SIB), and Stereotypy. The dataset comprised approximately 110.7 hours of labeled multimodal time-series data.
• Preprocessing: Signals were segmented using a 5-second sliding window with a 1-second stride. EDA signals were decomposed into tonic and phasic components, with only the tonic component retained for analysis to represent overall sympathetic arousal. Data quality was gated based on EDA validity to ensure proper skin contact.
• Model Architecture: The authors employed a multimodal fusion framework leveraging state-of-the-art foundation models fine-tuned for the specific task:
  • Accelerometry: Evaluated HarNet5 (ResNet-style, pretrained on Capture-24), DeepConvLSTM, and a Transformer-based classifier (Actigraphy Transformer).
  • EDA: Utilized a convolutional autoencoder pretrained on the WESAD dataset.
  • Skin Temperature: Modeled using a lightweight 1D CNN.
  • Fusion Strategies: Three fusion approaches were compared: (1) Naive feature concatenation of temporally pooled features followed by an MLP; (2) Temporal self-attention using a Vision Transformer (ViT) on time-aligned multimodal embeddings; and (3) Cross-time cross-modal attention using ViT to capture interactions across modalities and time steps.
• Experimental Protocol: The study utilized subject-wise five-fold cross-validation (or four-fold for multiclass tasks) to assess generalizability. The primary task was binary detection/prediction of challenging behaviors vs. no behavior. A secondary task involved multiclass prediction (None, Aggression, SIB, Stereotypy). Prediction horizons were tested from 30 seconds up to 30 minutes by shifting labels forward in time.

Key Contributions

1. Real-World Validation: This work demonstrates the feasibility of predicting challenging behavior episodes in a naturalistic special education classroom setting, moving beyond controlled laboratory environments.
2. Multimodal Foundation Models: The study introduces a framework that fine-tunes pretrained foundation models for multimodal wearable time-series analysis (accelerometry, EDA, skin temperature), addressing the scarcity of labeled data in clinical settings.
3. Proactive Prediction: Unlike prior retrospective detection systems, this approach successfully predicts challenging behaviors up to 10 minutes in advance, establishing a concrete operational window for teacher intervention.
4. Comprehensive Analysis: The paper provides a detailed analysis of modality contributions, fusion strategies, and the impact of prediction horizons, highlighting the dominance of accelerometry while quantifying the marginal but complementary role of autonomic signals.

Results

• Detection Performance: In binary detection tasks, the naive feature concatenation approach achieved the best performance with an AUC-ROC of 0.793. Among unimodal models, the accelerometer-based HarNet5 model performed best (AUC-ROC 0.778), outperforming EDA and skin temperature models.
• Prediction Performance: The system successfully predicted challenging behaviors up to 10 minutes in advance with an average AUC-ROC of 0.78 (95% CI ±0.10). Performance gradually declined as the prediction horizon increased to 30 minutes, though it remained above 0.75.
• Modality Importance: MM-SHAP analysis revealed that accelerometry was the dominant predictor (contributing 90% to the decision), while EDA and skin temperature provided marginal contributions (8% and ~2%, respectively). Grad-CAM visualizations confirmed that model attention aligned with motion-rich intervals preceding behaviors.
• Multiclass Limitations: The four-class prediction task (None, Aggression, SIB, Stereotypy) yielded an AUC-ROC of 0.65, driven almost entirely by the ability to detect the "None" class. The model failed to distinguish between rare behavior types (Aggression and SIB) due to extreme class imbalance. A coarser three-class grouping (High-risk vs. Low-risk vs. None) resulted in an AUC-ROC of 0.53, indicating that fine-grained behavior type discrimination is not currently feasible with the available data scale.
• Per-Subject Variability: Performance varied significantly across subjects (AUC-ROC ranging from 0.65 to 0.90), correlating more strongly with the density of labeled behavioral events than with age or sensor placement location.

Significance and Claims
The paper claims that its results establish a "concrete foundation" for developing proactive in-class intervention systems. By achieving a 10-minute prediction horizon with an AUC-ROC of 0.78, the system offers a practical window for teachers to intervene and minimize safety risks before an episode escalates. The authors emphasize that while the system prioritizes sensitivity (recall of 0.55) over precision (0.31) to ensure safety, this trade-off is clinically defensible in special education contexts where the cost of a missed event outweighs that of a false alarm.

The study concludes that multimodal wearable sensing is a viable approach for anticipating challenging behaviors in profound autism. However, the authors remain modest regarding the current limitations, noting that the system's inability to distinguish specific behavior types is a direct result of data scarcity and class imbalance. They posit that future improvements in generalization and fine-grained prediction will require larger, more diverse datasets and the integration of additional contextual signals (e.g., video, activity schedules). The work does not claim to solve the problem of behavior prediction entirely but rather demonstrates a significant step toward scalable, data-driven, proactive support for one of the most underserved populations in special education.