Enhancing Deep Learning Air Quality Forecasting through Bayesian Kalman Filter Preprocessing

Technical Summary: Enhancing Deep Learning Air Quality Forecasting through Bayesian Kalman Filter Preprocessing

Problem Statement
Accurate short-term forecasting of air pollutant concentrations is critical for public health protection and policy intervention. However, standard deep learning models, such as Gated Recurrent Units (GRU) and Long Short-Term Memory (LSTM) networks, often struggle when applied to real-world environmental data. These datasets are frequently characterized by noisy observations, missing data points due to sensor malfunctions, and complex interactions with meteorological variables. While hybrid models integrating Kalman filters directly into neural architectures have been proposed, this study addresses the specific challenge of improving input data quality before model training to prevent the learning process from fitting to noise or irregularities rather than meaningful temporal patterns.

Methodology
The study proposes a unified forecasting framework that integrates Bayesian Kalman filtering as a causal, forward-only preprocessing step prior to training deep learning architectures. The methodology proceeds through five distinct stages:

1. Causal Kalman Filtering and Imputation: Unlike traditional smoothing which uses future data, a forward-only Kalman filter is applied to each pollutant and meteorological variable independently. The variables are modeled as univariate local-level (random-walk) processes. The filter simultaneously reduces observational noise and estimates missing values. Crucially, when an observation is missing, the update step is bypassed, and the prior state is propagated, effectively imputing the value without information leakage. The process also outputs a filtered state variance ($\sigma^2_t$), which serves as a dynamic feature representing imputation uncertainty.

   • Parameter Estimation: The process noise ($Q$) and measurement noise ($R$) variances are estimated using the Expectation-Maximization (EM) algorithm, strictly applied to the chronological training portion (first 80%) of the dataset to ensure causality.

2. Feature Selection: Following preprocessing, features are selected based on the absolute Pearson correlation ($|\rho| \ge 0.10$) between the Kalman-filtered variables and the target pollutant. If fewer than two predictors meet this threshold, the top six are selected to ensure input diversity. The target pollutant is always included as an autoregressive predictor.

3. Input Construction: The selected Kalman-filtered values and their corresponding uncertainty variances are normalized and combined into overlapping lookback windows (12 or 24 hours) to serve as input for the deep learning models.

4. Recurrent Encoding with Attention: The study evaluates four deep learning architectures: standalone GRU and LSTM, and their variants enhanced with a temporal attention mechanism. The attention layer allows the model to assign adaptive weights to different time steps in the hidden state sequence, enabling the network to focus on the most informative historical patterns (e.g., recent peaks or periods of high imputation uncertainty).

5. Evaluation: The models are trained and tested on hourly data from three Canadian cities: Toronto, Calgary, and Saskatoon. Performance is evaluated for five pollutants (CO, NO$_2$, O$_3$, PM$_{2.5}$, and SO$_2$) using Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Coefficient of Determination ($R^2$), and Standard Error (SE).

Key Results
The study compares seven models: standalone GRU, standalone LSTM, a standalone Kalman Filter, and four hybrid models (Kalman-GRU, Kalman-LSTM, Kalman-Attention-GRU, and Kalman-Attention-LSTM).

• General Improvement: Across all pollutants and locations, models utilizing Kalman-filtered inputs consistently outperformed their standalone deep learning counterparts. This confirms that preprocessing to reduce noise and handle missing data significantly enhances forecasting accuracy.
• Pollutant-Specific Performance:
  • CO: Kalman-LSTM provided the most consistent and best performance across all three cities.
  • PM$_{2.5}$ and SO$_2$: Attention-enhanced Kalman models (specifically Kalman-Attention-LSTM for PM$_{2.5}$ and a mix of Kalman-Attention-GRU/LSTM for SO$_2$) achieved the highest accuracy, suggesting these pollutants benefit from the ability to weigh specific historical time steps.
  • NO$_2$ and O$_3$: The optimal architecture varied by city. For NO$_2$, Kalman-LSTM/GRU performed best in Toronto and Calgary, while Kalman-Attention-GRU was superior in Saskatoon. For O$_3$, Kalman-Attention-LSTM was best in Calgary and Saskatoon, whereas Kalman-GRU performed best in Toronto.
• Visual Validation: Time-series plots demonstrated that the Kalman-preprocessed models effectively tracked short-term fluctuations and concentration peaks, closely following observed data trends.

Significance and Claims
The primary contribution of this work is the systematic evaluation of Bayesian Kalman filtering as a standalone preprocessing technique for deep learning-based air quality forecasting. The authors claim that the framework's main advantage lies not in a specific neural network architecture, but in the preprocessing stage itself, which provides cleaner, more reliable inputs.

The study concludes that while Kalman filtering is universally effective for improving data quality, there is no single "best" deep learning architecture for all scenarios. The optimal model selection must consider both the specific pollutant type and local environmental conditions. For instance, while Kalman-LSTM is robust for CO, the addition of attention mechanisms yields superior results for particulate matter and sulfur dioxide. The findings suggest that model complexity (such as adding attention) should be tailored to the temporal characteristics of the data rather than assumed to universally improve performance.