🤖 machine learning

VaCDA: Variational Contrastive Alignment-based Scalable Human Activity Recognition

This paper proposes VaCDA, a multi-source domain adaptation framework that integrates variational autoencoders and contrastive learning to learn a shared latent space, effectively addressing data heterogeneity and improving human activity recognition across diverse devices, positions, and users.

Original authors: Soham Khisa, Avijoy Chakma

Published 2026-06-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Soham Khisa, Avijoy Chakma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize human activities like walking, running, or sitting. You have a lot of data from people wearing smartwatches on their wrists, but you want the robot to work for people wearing sensors on their ankles, or for people using a different brand of phone.

The problem is that the data looks very different depending on where the device is, who is wearing it, or what device they are using. It's like trying to teach a chef to recognize a "chicken dish" by only showing them pictures of fried chicken, but then asking them to identify a roasted chicken, a grilled chicken, and a chicken soup. The ingredients are the same, but the presentation is totally different. This is called "domain shift," and it usually makes the robot get confused.

The authors of this paper, Soham Khisa and Avijoy Chakma, built a new system called VaCDA to fix this. Here is how it works, using simple analogies:

1. The Problem: The "Messy Room"

Imagine you have several rooms (datasets) filled with toys (activity data).

Room A has toys scattered on the floor.
Room B has the same toys neatly stacked on shelves.
Room C has the toys floating in water.

If you train a robot to find a "ball" only in Room A, it will fail miserably in Room B or C. Traditional methods try to force the rooms to look exactly the same, but that's hard when the rooms are so different.

2. The Solution: The "Universal Translator" (VAE)

The first part of their solution is a Variational Autoencoder (VAE). Think of this as a super-smart translator or a "compression machine."

Instead of trying to make the messy floor look like the neat shelf, the VAE takes all the different toys from all the different rooms and translates them into a single, universal language (a "latent space").
In this universal language, a "ball" is just a "ball," regardless of whether it was on the floor, a shelf, or in water.
This allows the system to ignore the noise (like the specific device or body position) and focus on the core shape of the activity.

3. The Refinement: The "Spot the Difference" Game (Contrastive Learning)

There is a risk with the translator: it might get too good at generalizing. It might decide that a "ball" and a "cube" are the same because they are both roundish objects.

To fix this, the authors added Contrastive Learning. Imagine a game of "Spot the Difference" or a strict teacher:

Positive Pairs: The teacher shows the robot two pictures of the same activity (e.g., two different people walking) and says, "These are the same! Keep them close together in your mind."
Negative Pairs: The teacher shows a picture of walking and a picture of running and says, "These are totally different! Push them far apart in your mind."

By combining the Translator (VAE) with the Strict Teacher (Contrastive Learning), the system learns to group similar activities together while keeping different activities separate, even if they come from different devices or people.

4. The Result: A Scalable Team Player

Most old systems could only learn from one source (e.g., one person's data) to help another. VaCDA is special because it is scalable.

Imagine you are trying to learn a new language. Old methods might let you talk to one native speaker.
VaCDA lets you listen to ten different native speakers at once, all speaking different dialects, and it figures out the common grammar rules that apply to all of them.

What Did They Find?

The authors tested this system on four real-world datasets involving smartwatches, smartphones, and different body positions.

Cross-Position: When moving a sensor from a wrist to an ankle, VaCDA was much better at recognizing activities than previous methods.
Cross-Device: When switching from a smartphone to a smartwatch, VaCDA performed the best.
Cross-Person: When trying to adapt to a new person, it did very well, though it was slightly behind the best existing method in some specific cases.

In short: VaCDA is a new way to teach computers to understand human movement by translating messy, different data into a common language and then using a "spot the difference" game to make sure it doesn't get confused. It works better than older methods when the data comes from different places, people, or devices.

Technical Summary: VaCDA – Variational Contrastive Alignment-based Scalable Human Activity Recognition

1. Problem Statement

The proliferation of wearable devices has generated vast amounts of unlabeled sensor data for Human Activity Recognition (HAR). However, training robust models is hindered by two primary challenges:

Label Scarcity: Manual annotation is labor-intensive and error-prone, making supervised learning difficult.
Distribution Heterogeneity (Domain Shift): Data distributions vary significantly due to device placement (e.g., wrist vs. leg), device type (smartphone vs. smartwatch), and user behavior (age, BMI, movement patterns). Traditional supervised models, which assume training and test data follow the same distribution, perform suboptimally when applied to these shifted domains.

While Domain Adaptation (DA) addresses this shift, existing methods often struggle with Multi-Source Domain Adaptation (MSDA), where multiple labeled source domains must be aligned with an unlabeled target domain. Furthermore, many approaches rely on discriminative techniques (e.g., adversarial learning) that may not fully capture the underlying generative structure of the data, while generative approaches like Variational Autoencoders (VAEs) risk over-generalizing distinct activities into the same latent space.

2. Methodology: The VaCDA Framework

The authors propose VaCDA (Variational Contrastive Domain Adaptation), a scalable MSDA framework that integrates the feature generalization capabilities of VAEs with the discriminative power of contrastive learning. The architecture operates via two distinct pipelines:

A. Pre-training Pipeline (Feature Alignment)

This pipeline aligns features across all domains (multiple sources and the target) to learn a shared, low-dimensional latent space. It combines three loss functions:

Variational Autoencoder (VAE) Loss: The model learns a shared distribution $D$ $D$ by minimizing the Evidence Lower Bound (ELBO). This involves:
- Reconstruction Loss: Ensuring the decoder can reconstruct input signals from latent variables.
- KL Divergence: Regularizing the learned latent distribution to match a standard normal prior, forcing the model to generalize across different source and target distributions.
Self-Supervised Contrastive Loss: Inspired by SimCLR, this loss pulls together augmented views of the same sample (positive pairs) and pushes apart different samples (negative pairs) in the latent space. This helps the model learn domain-invariant features.
Supervised Contrastive Loss: Applied to labeled source data, this loss encourages samples with the same class label to be closer in the latent space while separating different classes. This ensures the model retains discriminative power for specific activities, preventing the VAE from over-generalizing distinct activities.

B. Classifier Pipeline

Once the encoder is pre-trained, it is frozen. A separate classifier (a softmax-based neural network) is trained using the latent representations from the labeled source domains. The classifier is optimized using Cross-Entropy Loss on both original and augmented source inputs. During inference, the target domain data is passed through the frozen encoder and the trained classifier to predict activity labels.

Scalability

A key design feature is that the framework does not require separate pipelines for each domain. By leveraging the VAE's ability to map arbitrary numbers of source domains into a shared latent space, VaCDA can handle an arbitrary number of sources without architectural changes.

3. Key Contributions

The paper outlines three primary contributions:

Novel Probabilistic Framework: The proposal of VaCDA, which uniquely combines VAEs for distribution generalization with contrastive learning for feature discrimination.
Scalable MSDA Solution: A method capable of handling an arbitrary number of source domains, eliminating the need for domain-specific pipelines and addressing the complexity of real-world multi-source scenarios.
Extensive Empirical Evaluation: Validation across four public datasets (DSADS, OPPORTUNITY, PAMAP2, WISDM) under three heterogeneity scenarios: cross-position, cross-person, and cross-device.

4. Experimental Results

The authors evaluated VaCDA against six baselines: STranGAN, CoDATS, Deep CORAL, DAN, CALDA, and CrossHAR. The performance was measured using the Micro F1 score.

Cross-Position Heterogeneity: VaCDA outperformed all baselines on the DSADS dataset (69.13% vs. 66.02% for the next best, CALDA) and the PAMAP2 dataset (45.75% vs. 32.44% for CrossHAR). The authors noted a significant performance boost (approx. 13%) on PAMAP2, though they acknowledged that all models struggled on this dataset, potentially due to data misalignment issues.
Cross-Person Heterogeneity: VaCDA showed competitive performance. While it did not outperform the best baselines (CALDA and CoDATS) on DSADS and OPPORTUNITY, it achieved a 6% improvement over baselines on the WISDM dataset.
Cross-Device Heterogeneity: In the WISDM smartphone-to-smartwatch adaptation task, VaCDA marginally outperformed all baselines (49.67% vs. 44.86% for CALDA).
Ablation Study: Removing contrastive learning resulted in a performance drop across all datasets. The most significant gain was observed in the WISDM cross-person scenario (+18.19%), highlighting the critical role of contrastive learning in handling user-specific variability.

5. Significance and Claims

The paper positions VaCDA as a robust solution for the specific challenges of wearable HAR, where data heterogeneity is inherent.

Mitigation of Heterogeneity: The framework effectively addresses the "domain shift" caused by device placement, type, and user behavior by learning a shared latent space.
Scalability: It solves the scalability issue of multi-source adaptation, allowing the system to adapt to new domains without retraining the entire architecture from scratch.
Modest Claims: The authors are cautious in their conclusions. They acknowledge that while VaCDA excels in cross-position and cross-device scenarios, it does not strictly outperform state-of-the-art baselines in the cross-person scenario, though it remains competitive. They attribute some baseline underperformance (e.g., STranGAN) to the use of custom feature extractors not originally designed for IMU data.

The authors conclude that VaCDA offers a significant step forward in building scalable, robust HAR systems for real-world deployment, with future work planned to address open-set domain adaptation and novel category recognition.