CARE: Contrastive Alignment for ADL Recognition from Event-Triggered Sensor Streams

1. Problem Statement

The recognition of Activities of Daily Living (ADLs) using ambient, event-triggered sensors (e.g., motion, door, temperature) is critical for Ambient Assisted Living (AAL). However, existing methods face significant limitations due to the nature of the data:

• Data Characteristics: Ambient sensor data is sparse, irregular, noisy, and event-triggered (unlike continuous wearable signals).
• Limitations of Sequence-Based Methods: While they preserve temporal order, they often ignore absolute daily time context, are highly sensitive to noise, and lack explicit mechanisms to capture spatial relationships between sensors.
• Limitations of Image-Based Methods: These transform streams into 2D images to capture global patterns and spatial correlations. However, they often distort sensor layouts (due to 1D indexing), compress fine-grained temporal dynamics, and ignore sensor activation frequency.
• Limitations of Naïve Fusion: Simple feature concatenation of sequence and image views fails to enforce alignment between the two representations, leading to underutilization of their complementary strengths.

Core Challenge: How to jointly leverage temporal dynamics and spatial context in a shared latent space without suffering from noise sensitivity or layout distortion, while ensuring the representations are both view-consistent and class-discriminative.

2. Methodology: The CARE Framework

The authors propose CARE (Contrastive Alignment for ADL Recognition), an end-to-end framework that unifies sequence and image representations via Sequence–Image Contrastive Alignment (SICA).

A. Preprocessing and Encoding

The framework processes raw event streams through two parallel branches:

1. Time-Aware, Noise-Resilient Sequence Encoding:

   • Temporal Binning: Instead of ignoring timestamps or using only inter-event intervals, the method assigns a coarse-grained temporal embedding (e.g., 1-hour bins) to every event. This captures daily context (e.g., morning vs. night) while suppressing minute-level noise.
   • Frequency-Based Event Filtering (FEF): A threshold-based filter removes low-frequency sensor activations (spurious noise) that occur only once or twice within an activity segment, retaining only salient events.
   • Encoder: A recurrent network (LSTM/BiLSTM) processes the filtered, time-enriched sequence to generate temporal embeddings.

2. Spatially-Informed, Frequency-Sensitive Image Representation:

   • Dual-Perspective Image Construction:
     • Temporal Image: Encodes event order, sensor ID, and signal state into color channels, highlighting temporal dynamics.
     • Spatial Image: Maps sensors to their actual floor-plan coordinates. Nodes represent sensor positions (colored by activation frequency), and edges represent transition weights between sensors. This preserves true 2D spatial layout and transition statistics.
   • Encoder: A Convolutional Neural Network (ResNet) processes the concatenated temporal and spatial images to generate spatial embeddings.

B. Sequence–Image Contrastive Alignment (SICA)

To unify the two views, CARE employs a supervised contrastive learning objective:

• Projection Heads: Lightweight MLPs project sequence and image embeddings into a shared latent space.
• Positive/Negative Pair Construction:
  • Intra-view Positives: Samples of the same class within the same view (e.g., sequence-sequence).
  • Cross-view Positives: The paired sequence and image embeddings of the same sample, plus embeddings of the same class from the other view.
  • Negatives: All embeddings from different ADL classes.
• Objective: The SICA loss pulls cross-view embeddings of the same activity together and pushes different activities apart, enforcing view-invariance and class consistency.

C. End-to-End Joint Optimization

Unlike two-stage pipelines (pre-train then classify), CARE integrates SICA and classification into a single stage:

• Joint Loss Function: $L_{CARE} = \beta L_{SICA} + (1 - \beta) L_{CE}$, where $L_{CE}$ is the cross-entropy loss for activity classification.
• Benefit: The classification head provides task-specific supervision that regularizes the encoders, preventing overfitting to view-specific biases and ensuring the learned embeddings are directly useful for ADL recognition.

3. Key Contributions

1. CARE Framework: A novel, backbone-agnostic framework that aligns heterogeneous sequence and image views of event-triggered data in a shared latent space.
2. SICA Objective: A supervised contrastive alignment mechanism that explicitly enforces consistency between temporal and spatial views while maintaining class discriminability.
3. Robust Encoding Strategies:
   • Temporal Binning: Integrates absolute daily time context into sequence encoding.
   • Frequency-Based Filtering: Systematically removes noise by pruning low-frequency sensor events.
   • Dual-Perspective Images: Combines temporal dynamics with true spatial layout and transition frequency.
4. End-to-End Learning: A unified training strategy that avoids the complexity of multi-stage pipelines while achieving superior performance.

4. Experimental Results

The framework was evaluated on three CASAS datasets: Milan, Cairo, and Kyoto7.

• State-of-the-Art Performance: CARE achieved the highest accuracy across all datasets:
  • Milan: 89.8% Accuracy (vs. 88.7% for the next best, TDOST).
  • Cairo: 88.9% Accuracy.
  • Kyoto7: 73.3% Accuracy.
• Ablation Studies:
  • Cross-View vs. Uni-View: Cross-view alignment consistently outperformed uni-view baselines (LSTM, ResNet) and within-view alignment by ~1–3%.
  • SICA vs. Naïve Fusion: Replacing simple concatenation (Cross-Entropy only) with SICA improved F1 scores, particularly on the sparse/noisy Kyoto7 dataset (+1.3% F1).
  • Temporal/Spatial Integration: Concatenating temporal and spatial images yielded 5–7% gains over single-view inputs.
• Robustness:
  • Sensor Corruption: CARE showed high resilience to sensor malfunctions (replacing events with defaults) and layout perturbations (Gaussian noise on coordinates), degrading by only 1–3% compared to baselines which dropped significantly more.
  • Temporal Drift: In time-aware splits (training on early data, testing on later data), CARE outperformed baselines by up to 13% on Kyoto7, demonstrating better generalization to temporal shifts.
• Efficiency: Despite using dual streams, CARE is efficient (~0.01s/batch), comparable to image-based baselines and significantly faster than graph-based models.

5. Significance and Impact

• Bridging the Gap: CARE successfully bridges the gap between sequence-based (temporal) and image-based (spatial) approaches, proving that they are mutually reinforcing when aligned via contrastive learning.
• Real-World Applicability: The demonstrated robustness to sensor failures, layout changes, and temporal drift makes CARE highly suitable for real-world smart home deployments where data quality and sensor configurations are unstable.
• Methodological Advance: The paper establishes that explicit cross-view alignment is superior to naïve fusion for event-triggered data, offering a new paradigm for multimodal representation learning in sparse, noisy environments.
• Future Directions: The work opens avenues for applying contrastive alignment to other multimodal domains (audio, text, physiology) and developing adaptive mechanisms for evolving sensor infrastructures.