Leakage-Controlled Machine Learning for European Cd, Hg, and Pb Emission Inventories

Technical Summary: Leakage-Controlled Machine Learning for European Cd, Hg, and Pb Emission Inventories

Problem Statement
Timely, spatially resolved inventories of cadmium (Cd), mercury (Hg), and lead (Pb) are critical for transboundary pollution assessment and source screening. However, existing annual grid products often suffer from reporting delays, incomplete spatial submissions, and inconsistencies between releases (e.g., varying sector coverage or aggregation labels). A significant methodological gap exists in applying machine learning (ML) to these inventories: standard workflows often fail to control for data leakage (e.g., using future information or preprocessing statistics derived from the full dataset), ignore spatial dependence in validation (e.g., random row-wise splitting), and conflate the prediction of absolute emissions with the prediction of temporal changes. High accuracy in predicting absolute emissions often merely reflects the persistence of baseline spatial patterns rather than genuine temporal updating skill.

Methodology
The authors propose an auditable, leakage-controlled framework designed to convert heterogeneous European emission products into machine-learning-ready datasets. The workflow is built on five non-negotiable principles: explicit separation of predictors, targets, and metadata; spatial partitioning before any data-driven selection; train-only fitting of preprocessing operations; compatibility checks before pooling target years; and strict separation of validated outputs from exploratory projections.

1. Data Harmonization and Auditing:

   • Raw gridded data from the EMEP Centre on Emission Inventories and Projections (CEIP) for 2017–2023 were processed.
   • A "Dataset v3" construction pipeline was implemented. This involved metal-specific harmonization (Dataset v1) where target metals were separated from co-pollutant predictors.
   • Target-Family Auditing: The study identified that 2017–2019 "nationaltotal" grids were compatible, whereas 2020–2023 "sumallsectors" grids had fundamentally different sparsity and magnitude. Consequently, only 2018 and 2019 were used as targets for the validated demonstration to avoid semantic inconsistency.
   • Leakage Control: Spatial blocks were assigned to training, validation, and test sets before any feature screening, imputation, or scaling. All preprocessing (median imputation, standardization) and feature selection (removing high-missingness or low-variance features) were fitted exclusively on the training partition. Future-dated predictors were strictly excluded.

2. Task Definition:

   • Two distinct tasks were evaluated separately:
     • Absolute Emission: Predicting $\log_{10}(E_{i,t})$.
     • Baseline-Relative Change: Predicting $\Delta y_{i,t} = \log_{10}(E_{i,t} / E_{i,2017})$.
   • This dual-task design prevents stable source geography from being mistaken for temporal updating capability.

3. Model Benchmarking:

   • Ten regression algorithms (including ElasticNet, Random Forest, XGBoost, LightGBM, CatBoost, and Deep Neural Networks) were compared.
   • Hyperparameters were tuned using validation-based early stopping.
   • Model selection was based on minimizing validation RMSE. An inverse-RMSE weighted ensemble was tested but rejected as it did not significantly outperform the best single model.

4. Evaluation and Interpretation:

   • Models were tested on held-out spatial blocks (geographically distinct from training data).
   • Performance was benchmarked against deterministic operational baselines: a "persistence" baseline (assuming 2017 values remain constant) for absolute tasks and a "no-change" baseline (assuming zero change) for change tasks.
   • Feature importance was assessed via repeated permutation importance, interpreted strictly as predictive dependence within the dataset, not as causal attribution.

Key Results

• Dataset Construction: The framework produced strict, metal-specific datasets containing approximately 77,000 cells each for Cd, Hg, and Pb, with 300 selected predictors dominated by co-emission variables (e.g., PM, SOx, NOx, BC) rather than broad demographic covariates (population variables were filtered out).
• Model Performance:
  • LightGBM was selected for five of the six tasks (all absolute and Cd/Hg change), while XGBoost was selected for Pb change.
  • Absolute Emissions: Selected models achieved high $R^2$ values on held-out spatial blocks (0.973–0.985) and reduced RMSE by 43.9–73.5% compared to persistence baselines.
  • Relative Change: Performance was lower but still significant ($R^2$ = 0.712–0.861), with RMSE reductions of 43.9–73.5% compared to no-change baselines.
  • Temporal Horizon: Performance degraded as the forecast horizon increased from 2018 (1-year) to 2019 (2-year), particularly for change tasks, highlighting compounding uncertainty.
• Feature Dependence: The 2017 baseline grid was the strongest predictor. Beyond the baseline, co-emission inventories (transport, combustion, industrial sectors) provided more predictive information than population density. Annual precipitation was a significant predictor only for Hg.

Significance and Claims
The paper claims its primary contribution is not the novelty of a specific algorithm, but the integration of a rigorous, auditable framework that addresses common pitfalls in environmental machine learning.

• Methodological Rigor: By enforcing "split-before-processing" and strict target-family auditing, the study demonstrates how to construct defensible ML inputs that prevent data leakage and semantic inconsistency.
• Operational Utility: The framework is positioned as a tool for provisional inventory updating, anomaly screening, and prioritizing grid cells for expert review. It is explicitly not a replacement for bottom-up compilation or a method for unrestricted forecasting to entirely unseen years.
• Interpretive Clarity: The study emphasizes that high accuracy in absolute emission prediction often reflects spatial persistence. The dual-task evaluation and comparison against operational baselines are necessary to distinguish between reproducing static patterns and genuinely estimating temporal changes.
• Scope Limitation: The authors modestly note that the evaluation is a "short-horizon spatial updating" stress test within the 2018–2019 window and does not establish generalization to calendar years entirely unseen during development. The framework serves to bridge gaps in inventory releases rather than to generate final regulatory estimates without reconciliation with source-sector reviews.