💻 computer science

GenHAR: Generalizing Cross-domain Human Activity Recognition for Last-mile Delivery

GenHAR is a novel framework that addresses cross-domain distribution shifts in Human Activity Recognition by tokenizing sensor data and employing selective masking with efficient attention to achieve state-of-the-art accuracy and efficiency, as validated by large-scale real-world deployment in last-mile delivery.

Original authors: Zhiqing Hong, Zelong Li, Xiubin Fan, Guang Yang, Baoshen Guo, Haotian Wang, Tian He, Desheng Zhang

Published 2026-05-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhiqing Hong, Zelong Li, Xiubin Fan, Guang Yang, Baoshen Guo, Haotian Wang, Tian He, Desheng Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize when a delivery driver is walking, climbing stairs, or standing still. You train the robot using data from a specific group of drivers in Beijing. But when you send that same robot to work with drivers in Shanghai, or even just different drivers in the same city, it starts making mistakes. It gets confused because the "voice" of the data sounds different—maybe the phones are held in different pockets, or the drivers walk with different styles.

This is the problem the paper GenHAR solves. It's a new system designed to help computers recognize human activities (like walking or climbing stairs) even when the data comes from a completely different "world" than the one it was trained on.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Accent" Issue

Think of sensor data (from a phone or watch) like a person speaking. If you train a translator to understand a specific person's accent, they might struggle when that person speaks to a friend with a different accent. In the world of delivery, every courier moves slightly differently, wears their phone in a different spot, and uses a different device. This creates a "distribution shift"—the data looks different enough that the old AI gets confused and fails.

Most previous AI tried to fix this by showing it examples from the new group (the "target" domain) during training. But in the real world, you often can't get that data beforehand. You need a system that learns from one group and instantly works for any other group without needing extra training.

2. The Solution: Listening to the "Music" Instead of the "Words"

The authors realized that looking at the raw data (the "words" or the timeline of movement) is too noisy. Instead, they decided to translate the movement into frequency space (the "music").

The Analogy: Imagine two people singing the same song. One sings it fast, one sings it slow. If you look at the words (time), they look very different. But if you look at the sheet music (frequency), the melody and the notes are almost identical.
The Innovation: GenHAR converts the raw movement data into this "frequency sheet music." It found that the "melody" of an activity (like walking) stays the same even if the "tempo" or the "instrument" (the specific phone or person) changes. This makes the AI much more robust.

3. The Secret Sauce: The "Sensor Orchestra"

Once the data is in this frequency format, GenHAR uses a special type of attention mechanism.

The Analogy: Imagine an orchestra. A regular AI might listen to the Violin section and try to understand the whole song just by looking at how the violin notes change over time.
GenHAR's Approach: GenHAR acts like a conductor who listens to how the Violins, Cellos, and Drums talk to each other. It realizes that the relationship between the different sensors (accelerometers and gyroscopes) is the key to understanding the activity.
Selective Masking: To save energy, it doesn't listen to every possible conversation between instruments. It knows that the "Left-Right" accelerometer doesn't really need to talk to the "Up-Down" gyroscope. It silences those unnecessary conversations, making the system incredibly fast and efficient.

4. The Results: Faster, Smarter, and Real-World Tested

The paper claims three major wins:

Accuracy: It is significantly more accurate than current state-of-the-art methods (about 10% better on average). It didn't just guess; it learned the true "melody" of the activity.
Efficiency: It is much lighter. It requires 6.4 times less computing power than the best existing models. This is crucial because delivery drivers use these models on their phones, which have limited battery and processing power.
Real-World Deployment: The team didn't just test this in a lab. They partnered with JD Logistics, a major delivery company in China. They installed GenHAR on the phones of 9,985 couriers across 4 cities.
- In just one month, the system successfully detected 2.15 billion activities.
- It accurately identified when couriers were walking, climbing stairs, or standing still, even though it was trained on data from a completely different set of people.

5. Why This Matters for Delivery

The paper highlights two specific ways this helps the logistics company:

Fairer Workloads: By knowing exactly how much climbing or walking a courier does, the company can assign jobs more fairly. If a building has no elevator, the system knows that courier did "extra work" and can adjust their pay or task load accordingly.
Better Maps: The system helps verify shipping addresses. If a courier says they are at a specific address but the phone says they are driving continuously, the system knows the address might be wrong. This helped them map over 6.8 million shipping addresses with high accuracy.

In summary: GenHAR is a smart, lightweight AI that learns the "universal melody" of human movement. It ignores the noise of different phones and people, allowing it to recognize activities perfectly anywhere, anytime, without needing to be retrained.

Technical Summary: GenHAR – Generalizing Cross-domain Human Activity Recognition for Last-mile Delivery

1. Problem Statement

Human Activity Recognition (HAR) using Inertial Measurement Unit (IMU) sensors is critical for applications like smart healthcare and last-mile delivery logistics. However, a significant barrier to real-world deployment is cross-domain distribution shift. Models trained on data from a specific source domain (e.g., a specific user, device placement, or environment) often suffer marked performance degradation when tested on unseen target domains due to variations in sensor positioning, individual behavioral patterns, and environmental noise.

Existing solutions typically rely on Domain Adaptation, which requires access to target domain data (often unlabeled) during training. However, in practical scenarios like last-mile delivery, collecting and labeling target domain data is often infeasible, time-consuming, and costly. Consequently, the challenge is to achieve Domain Generalization: training a model exclusively on source domain data that can generalize effectively to unseen target domains without any target data access. Furthermore, these models must be computationally efficient enough to run on resource-constrained mobile and edge devices in real-time.

2. Methodology: The GenHAR Framework

GenHAR is a novel framework designed to learn domain-invariant sensor representations purely from source domain data. It addresses the challenges of distribution shift and computational efficiency through three core design principles:

A. Frequency Space Transformation

Unlike most existing HAR methods that operate in the time domain, GenHAR transforms raw IMU time-series data into the frequency domain using the Fast Fourier Transform (FFT).

Rationale: Time-domain features are highly sensitive to random factors like noise and subtle motion preferences, leading to significant distribution shifts between domains. In contrast, frequency features (specifically amplitude) are more robust to time shifts and exhibit smaller distribution gaps between source and target domains.
Amplitude vs. Phase: The framework specifically utilizes amplitude features rather than phase or complex values. Empirical analysis shows that phase features are highly sensitive to device orientation and noise, whereas amplitude features remain relatively stable across different domains. The Earth Mover's Distance (EMD) between domains is significantly lower in the amplitude space compared to the phase or time spaces.

B. Sensor-wise Self-Attention Mechanism

GenHAR employs a Transformer-based architecture but introduces a novel Sensor-wise Self-Attention mechanism in the frequency space.

Tokenization: Instead of treating individual time steps as tokens, the entire sequence of amplitude components for a single sensor channel is treated as one token.
Correlation Learning: The attention mechanism is calculated across the sensor channel dimension (e.g., between the X, Y, Z axes of an accelerometer and gyroscope) rather than along the temporal sequence. This captures the inherent physical correlations between different sensor channels, which the authors argue are more domain-invariant than individual channel signals.
Selective Masking: To improve efficiency and physical relevance, a selective masking strategy is applied. It masks correlations between physically unrelated measurements (e.g., the X-axis of an accelerometer and the Y-axis of a gyroscope) while preserving correlations within the same sensor type or across corresponding axes (e.g., $Acc_x$ and $Gyro_x$ ). This reduces unnecessary computation and focuses the model on meaningful intra- and inter-sensor relationships.

C. Efficiency Optimization

To meet the real-time requirements of mobile devices, GenHAR significantly reduces computational complexity:

By calculating attention along the sensor channel dimension (typically 6 axes) rather than the full temporal sequence length, the complexity is drastically reduced.
The selective masking mechanism further reduces matrix multiplication operations by approximately 33.3%.
The framework removes the "DC component" and the symmetric second half of the FFT spectrum to further minimize input size.

3. Key Contributions

The paper claims the following contributions:

First Frequency-Space Cross-Domain HAR: To the authors' knowledge, this is the first work to address cross-domain HAR by learning sensor attention specifically within the signal frequency space, without requiring any target domain training data.
Novel Architecture: The design of GenHAR, a frequency-space Transformer that captures sensor correlations via a channel-wise self-attention module, demonstrating that generalizable IMU representations can be learned purely from the frequency domain.
Superior Performance and Efficiency: Extensive experiments show GenHAR outperforms state-of-the-art (SoA) baselines while being significantly more efficient.
Real-World Deployment: The framework was successfully deployed in a large-scale industrial setting for last-mile delivery.

4. Experimental Results

The authors evaluated GenHAR on four real-world datasets (UCI, Shoaib, Motion, HHAR) involving four shared activities (walking, standing/sitting, walking upstairs, walking downstairs).

Accuracy: GenHAR achieved an average cross-domain accuracy of 81.42%, outperforming the second-best model (UniHAR) by 9.97%. It consistently surpassed other baselines, including pre-training methods (LIMUBERT, ContraTSC) and domain adaptation/generalization methods (SDMix, UniHAR).
Efficiency:
- FLOPs: GenHAR reduced Floating Point Operations by 6.4 times compared to the best baseline model.
- Self-Attention Computation: The sensor-wise attention design resulted in a 43.2 times reduction in self-attention computation compared to standard frequency attention.
- Inference Time: On mobile devices (PDAs), the average inference time was 0.00647 seconds, and on smartwatches, it was 0.02505 seconds.
Ablation Studies:
- Removing frequency features (using time-domain inputs) caused a 13.51% drop in accuracy.
- Replacing amplitude with phase features caused a 31.19% drop in accuracy, confirming the superiority of amplitude features for domain invariance.
- Replacing sensor-wise attention with standard temporal attention caused a 17.10% drop in accuracy.

5. Real-World Deployment and Significance

The paper highlights the deployment of GenHAR at JD Logistics across 4 cities involving 9,985 delivery couriers.

Scale: In one month, the system detected 2.15 billion real-time activities (1.84 billion still, 215 million walking, 54.8 million upstairs, 39.8 million downstairs).
Applications:
- Workload Understanding: The system identifies high-effort delivery scenarios (e.g., buildings without elevators) to support equitable compensation and fair dispatching algorithms.
- Address Location Mining: By combining activity recognition with GPS trajectories, the system inferred the locations of over 6.8 million shipping addresses with 93.3% accuracy (within 100 meters), improving logistics navigation and parcel allocation.

6. Claimed Significance

The authors position GenHAR as a significant advancement for last-mile delivery and general sensor-based AI. The significance lies in:

Solving the Data Scarcity Problem: It provides a viable solution for cross-domain HAR where target domain data is unavailable, a common constraint in real-world logistics.
Physical-Driven Design: The success of the frequency-space and sensor-wise attention approach suggests that modeling the physical generation principles of IMU signals (rather than just statistical patterns) leads to more robust and generalizable models.
Edge Deployment Viability: By drastically reducing computational costs while maintaining high accuracy, GenHAR demonstrates that sophisticated domain-generalization models can be deployed on low-resource mobile devices for real-time, large-scale industrial applications.