🤖 machine learning

Amortized Probabilistic Retrieval of Atmospheric CO2 from OCO-2 Spectra Using Deep Learning with Laplace Approximations and Normalizing Flows

This paper presents a novel deep learning framework that utilizes Laplace approximations and normalizing flows to enable fast, accurate, and robust probabilistic retrieval of atmospheric CO2 from OCO-2 spectra, significantly outperforming traditional operational algorithms in speed, uncertainty quantification, and handling of model errors.

Original authors: Alejandro Calle-Saldarriaga, Felix Jimenez, Jack Grosskreuz, Jiazheng Wang, Jonathan Hobbs, Matthias Katzfuss

Published 2026-06-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Alejandro Calle-Saldarriaga, Felix Jimenez, Jack Grosskreuz, Jiazheng Wang, Jonathan Hobbs, Matthias Katzfuss

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The Slow, Stiff Detective

Imagine you are a detective trying to figure out exactly how much carbon dioxide (CO2) is in the air above a specific spot on Earth. You have a satellite (NASA’s OCO-2) that acts like a super-powered eye, snapping photos of sunlight bouncing off the Earth. These photos aren’t just pictures; they are detailed "spectra"—like rainbows that tell you exactly which colors of light were absorbed by CO2 molecules.

Currently, scientists use a traditional method to solve this puzzle. Think of this method as a very meticulous, old-school accountant.

It’s Slow: For every single photo the satellite takes, the accountant has to do thousands of complex physics calculations by hand. It takes about 150 seconds (2.5 minutes) to solve just one puzzle. With millions of photos, this creates a massive traffic jam.
It’s Stiff: The accountant assumes the world is simple and predictable. It assumes errors are random and follow a perfect bell curve (Gaussian). But the real atmosphere is messy. Clouds, dust, and water vapor create weird, lumpy, asymmetric problems that the accountant’s simple rules can’t handle.
It’s Blind to Systematic Errors: The accountant assumes the satellite’s camera is perfect. But in reality, the camera has slight quirks and biases. The accountant ignores these quirks, leading to inaccurate answers.

The Solution: The Fast, Flexible AI Apprentice

The authors of this paper propose a new way to solve the puzzle using Deep Learning (a type of Artificial Intelligence). Think of this AI not as an accountant, but as a highly trained apprentice who has studied millions of practice puzzles.

Here is how their new system works:

1. Learning from "Perfect" Practice Tests (Simulation)

You can’t train an AI on real satellite data easily because we don’t know the exact amount of CO2 in the air for every single photo the satellite takes. It’s like trying to teach a student to read if you don’t have the answer key for every book.

So, the researchers created a high-fidelity simulation. They built a virtual world that mimics the real Earth, including all the messy details like clouds, dust, and even the known "quirks" (errors) of the satellite’s camera. In this virtual world, they know the exact answer for every puzzle. They trained their AI on this virtual data so it could learn the correct patterns without getting confused by real-world noise.

2. The Multi-Branch Brain (Architecture)

The AI doesn’t look at the whole rainbow at once. It has a specialized brain with different departments:

Department A (Oxygen Band): Looks at a part of the light spectrum to figure out air pressure and clouds.
Department B (Weak CO2 Band): Looks at a cleaner signal to estimate the total CO2.
Department C (Strong CO2 Band): Looks at a messier signal affected by water vapor and dust.
Department D (Auxiliary Info): Considers extra info like the angle of the sun and the type of dust in the air.

These departments talk to each other (using a "Transformer" module, similar to how modern language models work) to combine their insights into a final answer.

3. Two Ways to Estimate Uncertainty (The "Confidence" Check)

A good scientist doesn’t just give an answer; they say, "I’m 95% sure the answer is between X and Y." The paper introduces two ways for the AI to do this:

Method A: The Laplace Approximation (The "Quick Estimate"):
Imagine the AI finds the best answer and then looks at the "hill" around that answer. If the hill is steep, the AI is very confident (small error bar). If the hill is flat, the AI is unsure (large error bar). This is fast and gives a standard bell-curve estimate of uncertainty.
Method B: Normalizing Flows (The "Shape-Shifter"):
This is the fancy part. Sometimes, the uncertainty isn’t a simple bell curve. It might be lopsided or have two peaks (like a camel’s hump). The "Normalizing Flow" is like a clay sculptor. It takes a simple ball of clay (a standard bell curve) and stretches, twists, and molds it into the exact weird shape that matches the real atmospheric data. This allows the AI to capture complex, non-standard uncertainties that the traditional accountant misses.

The Results: Why the AI Wins

The researchers tested their AI against the traditional "accountant" method on the simulated data. Here is what they found:

Speed: The AI is orders of magnitude faster. While the traditional method takes 150 seconds per photo, the AI takes milliseconds. It’s like comparing a horse-drawn carriage to a sports car. This allows for real-time processing of massive amounts of data.
Accuracy: The AI’s point estimates (the specific CO2 number) were more accurate than the traditional method. Because the AI was trained on data that included the satellite’s known quirks, it learned to "correct" for those errors automatically.
Better Confidence Intervals:
- The traditional method was often overconfident. It said, "I’m 95% sure," but it was actually wrong more often than it should be. Its error bars were too narrow.
- The AI’s "Laplace" method was a bit too cautious (error bars were too wide).
- The AI’s "Normalizing Flow" method was the most accurate. It hit the sweet spot, providing uncertainty estimates that matched reality much better than the traditional method.
Handling Weird Shapes: The AI successfully identified that the uncertainty in some situations was lopsided or curved, not a perfect bell curve. The traditional method forced a bell curve onto this data, which was a bad fit.

Summary

Think of the traditional method as a rigid, slow calculator that assumes the world is simple and perfect. The new method is a flexible, fast AI apprentice that has practiced on a realistic virtual world. It learns to ignore the satellite’s quirks, processes data instantly, and understands that uncertainty in the real world is often messy and lopsided, not just a simple bell curve.

This doesn’t replace the physics; it uses the physics (via simulation) to teach an AI how to interpret the data faster and more accurately, paving the way for next-generation climate monitoring systems.

Technical Summary: Amortized Probabilistic Retrieval of Atmospheric CO2 from OCO-2 Spectra Using Deep Learning with Laplace Approximations and Normalizing Flows

Problem Statement
The accurate estimation of column-averaged dry-air mole fractions of carbon dioxide (XCO2) from satellite observations is critical for constraining the global carbon budget. NASA’s Orbiting Carbon Observatory-2 (OCO-2) performs this task by measuring reflected sunlight in specific spectral bands. The standard operational retrieval algorithm, ACOS Level 2, relies on optimal estimation (OE) techniques that solve a nonlinear inverse problem iteratively. While rigorous, this "full-physics" approach suffers from three primary limitations:

Computational Expense: Iterative radiative transfer calculations create bottlenecks for real-time processing of massive data streams.
Inadequate Uncertainty Quantification (UQ): Standard OE assumes locally Gaussian posteriors, which often fails to capture nonlinearities caused by cloud/aerosol scattering or misspecified priors, leading to poorly calibrated uncertainty estimates.
Neglect of Model Discrepancy: Operational solvers typically assume the forward model is perfect ( $\delta = 0$ ), ignoring systematic structural errors (model discrepancy) known to be a dominant source of bias.

Furthermore, validating probabilistic retrievals against ground truth is difficult because the Total Carbon Column Observing Network (TCCON) is geographically sparse and lacks the joint posterior distribution data required to benchmark high-dimensional UQ.

Methodology
The authors propose an amortized probabilistic inference framework using deep learning, validated on a high-fidelity OCO-2 simulation dataset that explicitly incorporates quantified forward-model discrepancies ( $\delta$ ).

Data: The training data consists of 47,900 spectral-state pairs generated from 479 reference soundings across 9 geophysical clusters. The simulation includes realistic instrument noise and systematic model errors derived from previous calibration studies, providing absolute ground truth for the full 20-layer atmospheric state vector.
Network Architecture: A multi-branch neural network encodes the three OCO-2 spectral bands (O2 A-band, Weak CO2, Strong CO2) using separate 1D convolutional encoders. These embeddings, along with auxiliary variables (solar zenith angle, aerosol types, etc.), are fused via a Transformer-based attention module. A task-specific MLP head produces predictions for scalar XCO2, the full 20D CO2 profile, or near-surface layers.
Uncertainty Quantification: Two methods are employed to convert point estimates into probabilistic posteriors:
1. Laplace Approximation: A Gaussian approximation is placed on the final linear layer parameters of the deterministic network. Epistemic uncertainty is derived from the curvature of the loss function, while aleatoric uncertainty is estimated from validation residuals.
2. Normalizing Flows (NF): A conditional normalizing flow replaces the MLP head. Using piecewise-rational-quadratic coupling transforms, the model learns a flexible, potentially non-Gaussian posterior density conditioned on the spectral embeddings. This allows for the modeling of asymmetric, multi-modal, or skewed distributions.

Key Contributions

Amortized Inference: The framework reduces inference time from seconds (operational) to milliseconds, enabling real-time processing.
Robustness to Model Error: By training on simulations with explicit model discrepancies, the network learns to correct for systematic errors neglected by standard physical inversions.
High-Accuracy Point Estimates: The method achieves superior predictive accuracy compared to traditional optimal estimation.
Calibrated UQ: Both Laplace approximations and Normalizing Flows provide better-calibrated uncertainty estimates than operational Gaussian error estimates.
Non-Gaussian Posteriors: Normalizing Flows successfully model complex, asymmetric posterior distributions, overcoming the restrictive Gaussian assumptions of traditional retrieval theory.

Results

Scalar XCO2 Prediction: The Laplace approximation achieved the lowest Root Mean Squared Error (RMSE) (1.53 ppm), closely followed by Normalizing Flows (1.55 ppm), both significantly outperforming the NASA Retrieval (2.85 ppm). Normalizing Flows achieved the best Negative Log-Likelihood (NLL) and the best probabilistic calibration, with Probability Integral Transform (PIT) histograms closest to uniform and empirical coverage curves tracking the nominal diagonal. The NASA Retrieval showed severe undercoverage and miscalibration.
Full CO2 Profile Prediction: Normalizing Flows achieved the lowest joint NLL, demonstrating superior ability to capture joint uncertainty structures across pressure levels. While Laplace approximations maintained competitive RMSE, Normalizing Flows provided better likelihood scores.
Near-Surface Levels: Dedicated 4-output Normalizing Flow models (trained specifically on near-surface layers) outperformed marginals extracted from full-profile models in both RMSE and NLL, suggesting that task-specific training is preferable for focused scientific interests.
Non-Gaussianity: Visualizations of bivariate posterior slices revealed that Normalizing Flows capture asymmetric and curved structures in the posterior distribution, whereas Gaussian approximations (NASA and Laplace) produce elliptical contours that often miss the true state or misrepresent uncertainty scale.
Computational Efficiency: Test-time inference for the deep learning models took milliseconds per sounding, compared to approximately 150 seconds for the operational ACOS full-physics retrieval.

Significance and Conclusion
The paper argues that simulation-based deep learning offers a viable path for next-generation operational processing systems. By combining the speed of amortized inference with rigorous probabilistic modeling, the framework addresses the computational and statistical limitations of current operational retrievals. The ability to learn bias corrections from simulated model discrepancies and to represent non-Gaussian uncertainties via Normalizing Flows establishes a new standard for probabilistic soundness in remote sensing. The authors acknowledge the "synthetic gap" and suggest future work should involve domain adaptation and fine-tuning on real coincident measurements from TCCON to ensure validity across real-world conditions.