Autonomous Camera-driven Environmental Risk Intelligence Framework Integrating Digital Twin Analytics for Smart Firework Industry Safety

Technical Summary: Autonomous Camera-Driven Environmental Risk Intelligence Framework for Smart Firework Industry Safety

Problem Statement
Firework manufacturing involves high-risk operations characterized by combustible chemicals, pyrotechnic compositions, and thermal sensitivity. Workers face significant threats from fire outbreaks, smoke accumulation, toxic gas emissions, and deteriorating air quality. Existing monitoring solutions are largely disjointed and reactive, relying on isolated CCTV systems or separate environmental sensors. These traditional methods fail to capture the complex, dynamic interactions between visual hazards (smoke/fire), environmental conditions (pollutants, weather), and worker physiological states. Consequently, current systems lack the capability to predict air quality deterioration, smoke dispersion, or fire hazards before they occur, severely limiting timely decision-making and proactive emergency response.

Methodology
The study proposes an Autonomous Camera-Driven Environmental Risk Intelligence Framework that integrates multimodal data sources into a unified predictive architecture. The system comprises four primary data acquisition layers:

1. Visual Surveillance: A dynamic PTZ (Pan-Tilt-Zoom) 360° camera monitors indoor production and storage areas (100m range), while a drone-mounted vision system patrols outdoor drying yards and storage zones at low altitudes (15–20 ft).
2. Environmental Sensing: Distributed nodes measure particulate matter (PM2.5, PM10), toxic gases (CO, SO₂), temperature, humidity, and meteorological data (wind speed/direction).
3. Wearable IoT: 50 workers voluntarily wear sensors tracking physiological parameters (body temperature, heart rate, SpO₂, motion activity) and proximity to hazardous zones via UWB.
4. Data Fusion and Analytics:
   • Preprocessing: Video frames undergo enhancement, smoke/fire region normalization, and object localization.
   • Vision Foundation Model (VFM): Extracts high-level semantic visual features (smoke density, worker activity, anomalies) from PTZ and drone streams.
   • Spatio-Temporal Transformer (STT): Fuses visual embeddings with sensor data (environmental, physiological, meteorological) to learn temporal dependencies and spatial relationships.
   • Environmental Digital Twin: A virtual replica of the factory continuously simulates industrial conditions, predicting risk evolution and smoke propagation trajectories.
   • Autonomous Response: The system triggers automated alarms, smart sprinkler activation, and dashboard notifications based on real-time risk predictions.

Key Contributions

1. Unified Multimodal Framework: The development of a single architecture integrating dynamic PTZ surveillance, drone monitoring, wearable physiological sensing, and environmental stations, moving beyond single-modality hazard detection.
2. Advanced Predictive Modeling: The proposal of a Vision Foundation Model integrated with a Spatio-Temporal Transformer to perform multimodal fusion for predicting Air Quality Index (AQI), fire hazard probability, worker exposure risk, and smoke propagation behavior.
3. Digital Twin Integration: The implementation of an Environmental Digital Twin that continuously replicates industrial conditions to support predictive safety analytics and autonomous emergency response.
4. Autonomous Emergency Mechanism: A system capable of automatically triggering fire alarms, sprinkler systems, and evacuation recommendations without human intervention when safety thresholds are breached.

Experimental Results
The framework was evaluated using a dataset comprising over 370 GB of visual data and 3.5 million sensor observations per day from a fireworks manufacturing facility.

• Hazard Detection: The proposed VFM-STT model achieved 99.31% accuracy, 99.18% precision, 99.42% recall, and 99.30% F1-score, outperforming CNN, ResNet50, EfficientNet-B7, ViT, and Swin Transformer baselines.
• AQI Prediction: The model demonstrated superior forecasting with an MAE of 2.84, RMSE of 3.67, and an R² of 0.9968, significantly reducing errors compared to Linear Regression, Random Forest, and LSTM models.
• Fire Hazard Prediction: The system achieved an overall accuracy of 98.23% in predicting fire risks across various severity levels, with the Critical Risk class reaching 99.24% accuracy.
• Worker Safety: Physiological abnormality detection (e.g., fatigue, respiratory distress, low SpO₂) achieved 98.84% accuracy. Worker exposure risk was successfully assessed, identifying 6 workers in critical risk categories with an average exposure duration of 212 minutes.
• Smoke Propagation: The Digital Twin maintained high predictive stability (R² > 0.97) for smoke trajectory forecasting, with the 10-minute forecast showing the highest accuracy (MAE: 1.84).
• Response Time: The autonomous emergency response system achieved an average total response time of 2.9 seconds for critical incidents, with detection times as low as 1.0–2.1 seconds.

Significance and Claims
The authors claim that this framework represents a paradigm shift from reactive industrial monitoring to proactive, intelligent safety ecosystems. By fusing visual, environmental, physiological, and meteorological data through advanced deep learning (VFM and STT) and Digital Twin analytics, the system provides comprehensive situational awareness that isolated systems cannot offer. The study asserts that this approach significantly enhances industrial safety performance by enabling the prediction of hazardous conditions before they escalate into incidents, thereby reducing occupational injuries and improving regulatory compliance in high-risk environments like the firework industry. The integration of autonomous response mechanisms further ensures rapid mitigation of threats, establishing a new standard for industrial risk intelligence and environmental sustainability.