Technical Summary: IZSafe – On-Body Deep Learning for Real-Time Firefighter Motion Recognition

Problem Statement

Existing artificial intelligence systems in fire protection and crisis management primarily address macro-level tasks, such as fire detection, spread prediction, and robotic support. These systems lack micro-level situational awareness, specifically the ability to monitor the real-time activity and operational state of individual firefighters during interventions. Monitoring individual movements is particularly challenging in environments characterized by smoke, darkness, narrow spaces, and physically demanding conditions. Furthermore, existing solutions often rely on laboratory settings, classical machine learning, or heavy edge computing devices, failing to provide a robust, low-latency, embedded deep-learning solution that operates directly on resource-constrained wearable hardware without continuous data streaming.

Methodology

Dataset Collection and Preparation

The study utilized a dataset collected from 112 professional male firefighters performing 33 standardized movements across three realistic environments: a gym, a staircase, and an area near a fire truck with extendable and vertical ladders.

• Sensors: Data was captured at 200 Hz using Inertial Measurement Units (IMU) and Magnetometers (MAG) integrated into an oxygen-bottle carrier (waist unit) and, initially, boot units.
• Preprocessing: Raw data underwent transformation from device to body coordinate systems. A segmentation pipeline combined automatic splitting (based on boot sensor signals) with manual video verification to label 303 complete exercises.
• Class Merging: To address the high similarity between certain movements (e.g., different ladder angles), the 33 original movements were merged into 16 broader categories (e.g., "Walking forward," "Stairs up," "Aerial ladder up").
• Sensor Configuration: While initial development used boot sensors for segmentation, the final deployed model relies exclusively on the waist unit (accelerometer and gyroscope) to ensure practical feasibility, eliminating wired boot sensors and unreliable wireless boot data transmission.

Model Architecture and Training

The core of the system is a compact Long Short-Term Memory (LSTM) recurrent neural network designed for sequential inertial data.

• Architecture: The model features a TimeDistributed LSTM layer, a Flatten layer, a Dense layer with ReLU activation, a Dropout layer (0.2), and an output Dense layer with Softmax activation.
• Constraints: The architecture was explicitly designed to run on an ARM Cortex-M4 microcontroller. To meet these constraints, the authors avoided complex architectures like Transformers or BiLSTMs and opted for a unidirectional LSTM.
• Quantization Strategy: Unlike many TinyML approaches, the final implementation retains 32-bit floating-point representation without quantization or pruning, as the model size was sufficiently small to fit within the microcontroller's memory and processing limits.
• Magnetometer Handling: Two configurations were tested: one with magnetometer data and one without. The final deployment configuration omits the magnetometer to eliminate dependence on magnetic heading and reduce sensitivity to magnetic interference, accepting a slight trade-off in offline accuracy for increased practical robustness.

Inference System

The system operates on an embedded pipeline consisting of three main components:

1. Interrupt Service Routine (ISR): Collects sensor data, averages samples (12 samples), and manages a hopping window (1200 ms).
2. Main Loop: Executes the AI model inference on the averaged window, averages the outputs over 10 results to reduce noise, and applies a probability threshold (40%) to classify movements as "known" or "unknown."
3. Movement Transmission Task: Aggregates the most frequent recognized movement over a transmission interval (3 seconds) and sends it via radio network to the command interface.

Key Results

Model Performance

• IMU/MAG Model (16 classes): Achieved 93.85% accuracy and a weighted F1-score of 0.94 on the test set.
• Magnetometer-Free Model (16 classes): Achieved 85.22% accuracy and a weighted F1-score of 0.852. This configuration was selected for deployment to ensure orientation and magnetic robustness.
• Confusion Analysis: The model showed high accuracy for walking, crawling, and stair movements. Confusion was primarily observed between similar ladder movements (e.g., vertical vs. intervention ladders) and specific ladder angles.

System-Level Performance

After implementing temporal aggregation and merging the hard-to-distinguish ladder classes (e.g., merging "Intervention ladder up" with "Aerial ladder up"), the complete inference system achieved a weighted recognition success rate of 95.47%.

• Latency: The inference time on the ARM Cortex-M4 is approximately 52 ms.
• Reaction Time: The effective reaction time for detecting a change in movement is approximately 1.2 seconds (1200 ms).
• Initialization Time: The system initialization time is approximately 2.4 seconds.

Deployment Verification

The system was verified during real firefighter exercises. The recognized movements were transmitted via radio, stored in a database, and visualized on a tablet interface for the intervention commander. The system successfully tracked firefighter location and activity in real-time, demonstrating stability in noisy, real-world conditions.

Significance and Contributions

The paper positions IZSafe not as a breakthrough in general Human Activity Recognition (HAR) architecture, but as a deployment-oriented contribution that successfully integrates several distinct research areas into a functional, operational system.

1. Micro-Level Situational Awareness: The system extends existing macro-level monitoring by providing real-time, individual activity data, complementing fire scene and spread prediction systems.
2. Embedded Deep Learning: It demonstrates the feasibility of running a recurrent LSTM model directly on a low-power ARM Cortex-M4 microcontroller without requiring continuous raw data streaming to a remote server or a powerful edge device (like a Raspberry Pi).
3. Minimalist Sensor Configuration: The solution achieves high recognition rates using only a single waist-mounted unit (accelerometer and gyroscope), avoiding the complexity and cabling issues of multi-sensor suits or boot sensors.
4. Practical Trade-offs: The work highlights a successful trade-off between classification detail, robustness, and latency. By removing the magnetometer and merging indistinguishable classes, the system prioritizes operational stability and robustness over theoretical maximum accuracy.
5. Real-World Validation: Unlike many HAR studies limited to offline evaluation or simulated environments, IZSafe was trained on data from 112 professional firefighters and verified in realistic training scenarios with actual intervention equipment.

The authors conclude that while limitations exist regarding dataset diversity (male-only, single organization) and the need for anomaly detection in future iterations, IZSafe proves that micro-level situational awareness via deep learning on wearable microcontrollers is technically feasible and practically useful for emergency response.