📊 statistics

An uncertainty-aware Bayesian framework for machine learning classification models: A case study in land cover classification

This paper proposes a Bayesian framework for generative machine learning classification models that explicitly incorporates input measurement uncertainty, demonstrating through land cover classification and synthetic simulations that this approach offers superior interpretability, robustness, and computational efficiency compared to popular models like random forests and neural networks.

Original authors: Samuel Bilson, Miles McCrory, Anna Pustogvar

Published 2026-05-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Samuel Bilson, Miles McCrory, Anna Pustogvar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to identify different types of terrain from space—forests, farms, cities, and grasslands—using satellite photos. This is called "Land Cover Classification." Usually, computers (Machine Learning models) are very good at this, but they have a big flaw: they act like overconfident students. They give you an answer, but they don't tell you how sure they are, or if the photo they are looking at is blurry or distorted.

In the real world, satellite photos aren't perfect. The sensors have errors, the atmosphere scatters light, and the data processing introduces "noise." This paper introduces a new way to teach computers to be humble and honest about these errors.

Here is the breakdown of their approach using simple analogies:

1. The Problem: The "Blind" vs. The "Aware"

Most current AI models are like a chef who tastes a soup but refuses to believe the salt shaker might be leaking. They assume the ingredients (the satellite data) are perfect. If the data is slightly off, the chef (the AI) might confidently say, "This is definitely a forest," when it's actually a field.

The authors argue that in science and metrology (the science of measurement), you must account for the fact that your measuring tape might be slightly stretched. They want an AI that knows, "Hey, this pixel looks like a forest, but the sensor was a bit shaky, so I'm only 80% sure."

2. The Solution: The "Bayesian Detective"

The authors propose a Bayesian framework. Think of this as a detective who doesn't just look at the evidence (the photo) but also keeps a notebook of how unreliable the witness (the sensor) usually is.

The Old Way (Standard AI): Looks at the photo and guesses.
The New Way (Bayesian QDA): Looks at the photo and asks, "How much noise is in this picture? If the picture is blurry, I will widen my guess to be safer."

They specifically used a model called Bayesian Quadratic Discriminant Analysis (BQDA).

Analogy: Imagine you are sorting marbles by color.
- A standard model draws a sharp line on the table: "Everything to the left is Blue, everything to the right is Red."
- The BQDA model draws a fuzzy cloud for Blue and a fuzzy cloud for Red. It knows that a "Blue" marble might roll a little bit into the "Red" zone because the table is shaking (measurement uncertainty). It calculates the probability of the marble landing in either zone based on how much the table is shaking.

3. The Experiment: Real and Fake Data

The team tested this new "humble" model in two ways:

A. Real Life (The UK Landscape):
They used satellite images of the UK from 2020 and 2021. They simulated "noise" by pretending the atmospheric data used to clean the images was slightly wrong (like guessing the humidity was 5% off).

The Result: When the data was messy or the training set was small, the new BQDA model was much more reliable. It didn't get confused as easily as the popular "Random Forest" or "Neural Network" models.
The "Cropland" Problem: They found that "Cropland" (farms) was the hardest to identify because it looks like bare soil in some photos and green plants in others. The new model admitted this confusion rather than guessing wildly.

B. Synthetic Data (The Simulation):
They created fake worlds with perfect math to test how the models handle different levels of "noise" (from 0.1 to 1.0).

The Result: As the noise got louder, the standard models started to fail faster. The BQDA model held its ground better. It was like a runner who slows down gracefully in the rain, while the others slipped and fell.

4. Why This Matters (The "Trust" Factor)

The paper claims three main benefits for this new model:

Trustworthiness: It explicitly tells you why it's unsure. It separates "I don't know because the data is bad" from "I don't know because the model is bad."
Interpretability: You can look at the model's math and say, "Ah, it thinks this is a forest because the average color of forests in the training data was X." Complex models like Neural Networks are "black boxes" where you can't see the reasoning.
Efficiency: It is much faster to run. While complex models take hours to train, this one is quick, making it easier to use for large-scale maps.

Summary

The paper doesn't claim this will cure diseases or predict the stock market. It specifically claims that for mapping the Earth's surface, using a model that acknowledges the imperfections of its own sensors leads to more trustworthy, interpretable, and efficient results. It turns a "confident guesser" into a "careful scientist."

Technical Summary: An Uncertainty-Aware Bayesian Framework for Machine Learning Classification Models

Problem Statement

Machine learning (ML) classification models are increasingly ubiquitous in applications such as land cover (LC) mapping, yet they often lack "trustworthiness" due to an inability to explain decisions or quantify uncertainty. While current research in uncertainty quantification (UQ) focuses heavily on epistemic uncertainty (model uncertainty), it frequently neglects input measurement uncertainty. In metrology and Earth Observation, input data often carries inherent uncertainty due to device limitations or results from previous measurement models in a traceability chain. Ignoring this input uncertainty breaks the traceability chain for downstream applications (e.g., climate modeling, greenhouse gas accounting). Furthermore, standard complex models like Random Forests (RFs) and Neural Networks (NNs) do not naturally admit the incorporation of input measurement uncertainty without computationally expensive Monte Carlo ensemble approaches.

Methodology

The authors propose a general Bayesian framework for incorporating input measurement uncertainty into generative ML classification models. The core methodology relies on an Errors-in-Variables (EiV) approach, treating measured inputs as realizations of true, unknown values.

Framework Formulation:
- The model assumes measured inputs $x$ are realizations of true inputs $\chi$ subject to measurement error $\epsilon_x$ ( $x = \chi + \epsilon_x$ ).
- Using Bayes' theorem, the class posterior predictive probability is derived by integrating over the true unknown inputs and model parameters.
- The framework distinguishes between cases where measurement uncertainty parameters are known/fixed versus unknown (estimated alongside model parameters). The paper focuses on the latter, where uncertainty parameters are learned via Bayesian inference.
Specific Implementation: Bayesian Quadratic Discriminant Analysis (BQDA):
- The authors instantiate the framework using Quadratic Discriminant Analysis (QDA), a parametric generative classifier that models class-conditional distributions as multivariate Gaussians.
- Measurement Model: Input measurement uncertainty is modeled as a multivariate Gaussian distribution centered on the true input.
- Analytic Solution: By assuming conjugate priors for the Gaussian parameters, the authors derive an analytic expression for the posterior predictive distribution. This avoids the need for sampling techniques like Markov Chain Monte Carlo (MCMC), which are typically required for intractable posteriors in complex models.
- Resulting Distribution: The final predictive distribution for a class is a multivariate t-distribution, which naturally accounts for both the uncertainty in the model parameters (epistemic) and the input measurement noise (aleatoric).
Case Study: Land Cover Classification:
- Data: Copernicus Sentinel-2 data (2020–2021) over a 20 km × 20 km area in Scotland.
- Labels: Derived automatically from UKCEH maps using a temporal stability criterion (unchanged labels over three years) and a quality criterion (>95% classification probability).
- Uncertainty Generation: To simulate input uncertainty, the authors varied the Aerosol Optical Depth (AOD) used in atmospheric correction (Sen2Cor). They generated 25 realizations of Bottom-of-Atmosphere (BOA) reflectance for each pixel by sampling AOD from a Gaussian distribution based on CAMS data uncertainties.
- Classes: Four LULUCF classes: Forest, Cropland, Grassland, and Settlements.

Key Contributions

Framework for Input Uncertainty: A concrete Bayesian methodology that explicitly incorporates input measurement uncertainty into generative classifiers, moving beyond the standard assumption of noise-free inputs.
Analytic BQDA Derivation: The derivation of a closed-form solution for BQDA that models input noise, allowing for efficient computation without ensemble averaging or MCMC.
Benchmarking: A comprehensive comparison of BQDA against standard classifiers (Standard QDA, LDA, Random Forests, and Neural Networks) on both real-world LC data and synthetic datasets with varying noise strengths and dimensions.

Results

The study evaluates performance using macro F1/F2 scores, Cross-Entropy loss, and the Brier score (a proper scoring rule for probability calibration), alongside computational cost.

Real-World Land Cover Data:
- Small Data Regime: BQDA significantly outperforms standard QDA and other benchmarks when training data is scarce (e.g., <1000 pixels). Standard QDA often fails (singular covariance matrices) in these scenarios, whereas the Bayesian prior in BQDA acts as a regularizer.
- Generalizability: When training and testing on different years (e.g., train 2020, test 2021), BQDA maintains the most consistent performance (lowest Brier score), indicating superior handling of predictive uncertainty across domains.
- Class Performance: While all models struggled with "Cropland" due to its high spectral variance, BQDA provided more reliable probability outputs.
- Efficiency: BQDA, QDA, and LDA were significantly more computationally efficient than RFs and NNs. Unlike RFs/NNs, BQDA produces reproducible results given the same data, as it relies on analytic expressions rather than random number generation during training.
Synthetic Data Experiments:
- Experiments were conducted on synthetic Gaussian data with 2, 10, and 50 input variables, with varying noise strengths (Gaussian and Uniform).
- Robustness: As noise strength increased, BQDA consistently achieved better metric scores than other models.
- High-Dimensional Noise: In high-dimensional settings (50 variables), BQDA not only outperformed benchmarks but also decayed more slowly as noise increased, demonstrating the regularizing effect of the Bayesian prior against overfitting.
- Distribution Independence: The performance advantage of BQDA held for both Gaussian and Uniform input noise distributions.

Significance and Claims

The paper claims that the proposed BQDA framework offers a more trustworthy and interpretable alternative to complex black-box models for applications requiring metrological traceability.

Interpretability: Unlike NNs or RFs, BQDA parameters directly map to the mean and covariance of input data, making the model's decision boundaries and uncertainty sources transparent.
Efficiency: It provides a computationally efficient way to account for input uncertainty without the heavy cost of Monte Carlo ensembles.
Metrological Relevance: By explicitly modeling input measurement uncertainty, the framework supports the "traceability chain" required in metrology, ensuring that downstream applications (like climate modeling) can propagate uncertainties correctly.

The authors modestly note limitations: the current work does not address class labeling uncertainty (e.g., mixed pixels) and focuses on cases where the input uncertainty distribution must be learned rather than being known a priori. However, the framework is presented as extensible to these future challenges.