An intercomparison of generative machine learning methods for downscaling precipitation at fine spatial scales

Technical Summary: An intercomparison of generative machine learning methods for downscaling precipitation at fine spatial scales

Problem Statement
Global Climate Models (GCMs) operate at coarse resolutions (100–150 km), which are insufficient for assessing local climate impacts. Regional Climate Models (RCMs) provide higher resolution (10–25 km) but are computationally expensive, limiting the ability to generate large ensembles necessary to sample uncertainty in model, scenario, and internal variability. While machine learning (ML) offers a computationally efficient alternative for RCM emulation, standard deterministic regression methods (e.g., U-Nets trained with Mean Squared Error) tend to "regress to the mean," producing overly smooth outputs that underestimate fine-scale variability and extreme precipitation events. Although stochastic generative models (Generative Adversarial Networks [GANs], diffusion models, and flow matching) have shown promise in generating realistic ensembles, there is a lack of direct comparative studies evaluating their skill under consistent conditions, particularly regarding their ability to reproduce climate change signals for extremes.

Methodology
The study benchmarks eight generative model configurations against a deterministic U-Net baseline for downscaling daily precipitation over New Zealand. The models are trained on 137 years of 12 km dynamical downscaled simulations from the Conformal Cubic Atmospheric Model (CCAM) driven by the ACCESS-CM2 GCM. Evaluation is performed on four independent CCAM simulations driven by withheld GCMs (EC-Earth3, NorESM2-MM, AWI-CM-1-MR, and CNRM-CM6-1) covering historical (1985–2014) and end-of-century (2070–2099) periods.

The study compares three families of generative models:

1. GANs: Two variants (ResGAN-v1 and ResGAN-v2) utilizing a residual correction framework where the generator learns to predict residuals from a deterministic U-Net baseline. These employ Wasserstein distance-based loss functions with gradient penalties and intensity constraints.
2. Diffusion Models: Two variants (DDPM and ResDDPM) based on the Denoising Diffusion Probabilistic Model (DDPM) framework. These use a reverse-time stochastic differential equation (SDE) or probability-flow ordinary differential equation (ODE) to denoise samples.
3. Flow Matching: Three variants (RCMFlow, RCM-tFlow, and Res-tFM) that learn a velocity field to transport samples from a base distribution (Gaussian or heavy-tailed Student-t) to the data distribution.

A critical methodological component is the residual correction framework, where a deterministic U-Net first predicts the conditional mean, and the generative model learns the stochastic residuals. This approach is applied to ResGAN-v1, ResGAN-v2, ResDDPM, and Res-tFM.

To address the computational cost of diffusion and flow matching models, which traditionally require hundreds or thousands of sampling steps, the study investigates the use of higher-order differential equation solvers. Specifically, it compares first-order Euler solvers against Adams–Bashforth 2-step Predictor–Corrector (AB2-PC) solvers for flow matching and DDIM sampling for diffusion.

Key Contributions

• Solver Efficiency: The study demonstrates that using higher-order solvers (AB2-PC for flow matching and DDIM for diffusion) allows for a drastic reduction in sampling steps (from ~1000 to 25) with only a minor reduction in skill. This reduces the inference cost by approximately 40-fold compared to prior high-step implementations, making these models more viable for operational large-ensemble downscaling.
• Comprehensive Benchmarking: The paper provides a rigorous intercomparison of GANs, diffusion, and flow matching models across a diverse set of metrics, including spatial structure (RALSD), distributional properties (LHD), ensemble calibration (Spread-Skill Ratio, Rank Histograms), and climate change signals.
• Climate Change Signal Evaluation: Unlike many benchmarks that focus solely on present-day statistics, this study explicitly evaluates the models' ability to reproduce the climate change signal (CCS) of extreme precipitation (Rx1Day) and mean precipitation (DJF) between historical and future periods.

Results

• Solver Performance: Flow matching models using the AB2-PC solver with just 25 steps outperformed Euler solvers with 100 steps in spatial structure (RALSD), intensity distribution (LHD), and ensemble calibration (Spread-Skill Ratio). Diffusion models using DDIM with 25 steps achieved comparable performance.
• Ensemble Calibration: Diffusion and flow matching models produced significantly better-calibrated ensembles than GANs. GANs (particularly ResGAN-v1) were found to be under-dispersive (nearly deterministic), whereas flow matching models (e.g., RCM-tFlow, Res-tFM) exhibited near-flat rank histograms and superior Spread-Skill Ratios.
• Spatial Structure and Extremes: Diffusion and flow matching models generated more physically realistic spatial structures, particularly for complex weather events like ex-tropical cyclones, where GANs failed to resolve the cyclonic structure. However, regarding the precipitation intensity distribution, GAN configurations (specifically ResGAN-v2) outperformed diffusion and flow matching models, likely due to the explicit intensity constraints in their loss functions. Most generative models slightly overestimated the frequency of very high-intensity events compared to the ground truth.
• Climate Change Signals: A critical finding is that most models, despite good performance on standard climatological metrics, systematically underestimated the climate change signal of extreme precipitation (Rx1Day) in the future period. Only ResGAN-v1 and RCM-tFlow (Flow Matching with a Student-t prior) reliably reproduced the CCS for both mean and extreme precipitation across all evaluation GCMs.

Significance and Claims
The paper claims that while diffusion and flow matching models offer superior ensemble calibration and spatial realism compared to GANs, they are not universally superior across all metrics. The study highlights that standard evaluation metrics (e.g., spatial correlation, mean error) and visual realism do not guarantee skill in predicting climate change signals.

The authors conclude that:

1. Computational Viability: The perception that diffusion and flow matching models are prohibitively expensive is mitigated by the use of higher-order solvers and reduced step counts, making them competitive for large-ensemble applications.
2. Application-Dependent Selection: Model selection should be guided by the specific application. For weather-scale applications requiring well-calibrated ensembles and physical realism, flow matching and diffusion models are preferred. For applications prioritizing climatological accuracy and climate change signal reconstruction (where GANs showed superior performance on intensity distributions and CCS in this study), ResGAN-v1 or RCM-tFlow are the most suitable.
3. Limitations in CCS Prediction: The systematic underestimation of extreme CCS by most models suggests that current training objectives (focused on daily prediction) are insufficient for capturing decadal trends in rare events. The authors suggest that future work must explore training strategies that explicitly target climate change signal fidelity, potentially through physical constraints or noise-level-modulated loss functions, as standard metrics fail to identify models capable of robust CCS reproduction.