Activity Recognition Using mm-Wave Radar and Deep Learning: Prayer Tracker Case Study

1. Problem Statement

The paper addresses the critical challenge of privacy preservation in surveillance and activity monitoring systems. Traditional video-based monitoring is often restricted in sensitive environments (e.g., places of worship, hospitals, or private homes) due to privacy laws and cultural modesty concerns.

• Specific Use Case: The authors focus on Prayer Tracking (Salat) for Muslims. Prayer involves a specific sequence of four repetitive postures (Standing, Bowing, Prostrating, Sitting).
• User Needs: Individuals, particularly the elderly or beginners, often lose count of prayer cycles (rakat) or make errors in movement sequences. They require real-time feedback and assistance without the intrusion of video cameras.
• Gap: While radar technology is used for vital sign monitoring and posture classification, there was no prior research applying it specifically to prayer tracking or providing a comprehensive system framework for this purpose.

2. Methodology

The proposed framework utilizes Frequency-Modulated Continuous Wave (FMCW) mm-Wave radar combined with Deep Learning to classify human postures without capturing visual images.

A. Hardware and Signal Acquisition

• Sensor: Texas Instruments AWR-1642 module (77 GHz, 2 Tx, 4 Rx antennas).
• Configuration: The radar is positioned 1.5 meters in front of the user at a 60-degree elevation angle.
• Signal Processing:
  • The system transmits chirp signals and processes the beat frequency to estimate Range, Angle (Azimuth/Elevation), and Velocity (Doppler).
  • It generates a point cloud of scatterers representing the human body.
  • Preprocessing: Raw point cloud data is filtered to remove noise and static clutter. The data is then transformed into a 2-channel image representation (35x40 pixels):
    1. Channel 1: Peak reflection amplitude (Range-Angle map).
    2. Channel 2: Doppler shift values.

B. Feature Extraction and Classification

The authors evaluated multiple machine learning approaches to classify the four prayer postures:

1. Traditional ML:
   • Common Sense Features: Hand-crafted features (max angle, mean range) fed into Decision Trees.
   • Principal Component Analysis (PCA): Dimensionality reduction followed by Logistic Regression, Euclidean Distance, or Dense Neural Networks.
   • Sparse Image Representation: Using Orthogonal Matching Pursuit (OMP) for sparse decomposition.
2. Deep Learning (CNN):
   • ResNet: A deep convolutional neural network known for high accuracy.
   • SqueezeNet: A lightweight architecture optimized for computational efficiency.
   • Compression Strategy: The authors tested resizing images (35x40 to 20x21) to reduce complexity, observing the impact on accuracy.

C. System Logic

• State Machine: A Moore Finite State Machine (FSM) manages the prayer sequence.
• Feedback Loop: To ensure robustness, the system buffers the last 3 predictions and uses a majority vote to determine state transitions, preventing false positives during posture transitions.
• Error Handling: The system is designed to detect deviations from the expected sequence and provide real-time feedback to the user.

3. Key Contributions

1. Privacy-Preserving Framework: Demonstrates a viable alternative to video surveillance for sensitive applications, using mm-Wave radar to track activity without capturing identifiable visual data.
2. First-of-its-Kind Application: The first research to apply radar-based activity recognition specifically to the complex, sequential movements of Islamic prayer.
3. Comprehensive Algorithmic Comparison: A rigorous evaluation of various feature extraction methods (PCA, Sparse, Hand-crafted) against Deep Learning models (ResNet, SqueezeNet) across different data scenarios.
4. Channel Analysis: An investigation into whether the Doppler channel is necessary. The study found that the Reflection Amplitude channel alone contains sufficient information for high-accuracy classification, allowing for reduced computational load.
5. Real-World Validation: The system was tested on a dataset of ~40,000 radar shots across 10 scenarios, including different individuals and environments.

4. Results

The study evaluated performance across three distinct data division scenarios:

1. Random Split: All data mixed (known individuals/conditions).
2. Same Person, Different Scenarios: Training on 9 scenarios, testing on the 10th (same person).
3. Unseen Person (Generalization): Training on 9 scenarios, testing on a completely new individual (most critical for real-world deployment).

Key Findings:

• Best Performer: ResNet achieved the highest accuracy, reaching 95.4% on unseen data (Scenario 3).
• Efficiency vs. Accuracy: SqueezeNet without image compression performed nearly as well as ResNet in generalization tasks while being computationally lighter.
• Compression Impact: Applying image compression (resizing) significantly degraded the performance of SqueezeNet, while ResNet remained robust but slightly less accurate.
• Traditional Methods: PCA-based methods and Decision Trees showed poor generalization on unseen data (dropping to ~45-50% in Scenario 3), highlighting the necessity of Deep Learning for this task.
• Channel Efficiency: Using only the Reflection channel yielded results comparable to using both Reflection and Doppler channels, suggesting Doppler is less critical for static posture classification.

5. Significance and Future Work

• Significance: This work proves that mm-Wave radar, coupled with deep learning, can effectively replace video cameras in privacy-sensitive monitoring. It offers a practical solution for the elderly and religious communities to perform daily rituals with dignity and accuracy.
• Scalability: The framework is not limited to prayer; it can be adapted for rehabilitation monitoring, fall detection, and general activity tracking in hospitals or smart homes.
• Future Directions: The authors suggest extending the system to dynamic recognition (video-like sequence analysis) by incorporating the Doppler channel more heavily and utilizing more complex temporal models (e.g., RNNs or LSTMs) to track movement sequences over time rather than just static postures.

Conclusion: The paper successfully demonstrates a robust, privacy-compliant system for prayer tracking, achieving high accuracy (up to 95.4%) using ResNet on radar point cloud data, with SqueezeNet offering a viable, efficient alternative for deployment.