📊 statistics

Geometry-Aware Bayesian Quantification via Compositional Data Analysis

This paper introduces a geometry-aware Bayesian quantification method that leverages log-ratio representations and Aitchison geometry to address the limitations of Euclidean kernels in modeling classifier posteriors, thereby improving label shift adaptation and prevalence estimation across diverse data domains.

Original authors: Alejandro Moreo, Pablo González, Juan José del Coz

Published 2026-07-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Alejandro Moreo, Pablo González, Juan José del Coz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out the makeup of a crowd you can't see directly. You have a "crystal ball" (a machine learning model) that looks at people and guesses their profession. Sometimes the crystal ball is very confident ("This is definitely a doctor!"), and sometimes it's unsure ("Maybe a nurse, maybe a teacher?").

The problem is that the crowd you are looking at now (the test data) might have a different mix of professions than the crowd the crystal ball was trained on. Maybe the training crowd was 50% doctors, but the new crowd is 90% doctors. This is called Label Shift.

Your goal is Quantification: You don't need to guess the job of every single person; you just need to accurately estimate the percentage of doctors, nurses, and teachers in the new crowd.

The Problem: The "Flat Map" Mistake

The paper argues that most current methods for solving this problem are using a "flat map" to navigate a "curved world."

The Curved World (The Simplex): The probabilities your crystal ball gives you (e.g., 80% Doctor, 20% Nurse, 0% Teacher) always add up to 100%. In math, this is called a Simplex. Think of it like a triangular piece of paper. You can't move "off" the paper; if you increase the Doctor percentage, you must decrease the Nurse or Teacher percentage. They are locked together.
The Flat Map (Euclidean Space): Standard methods treat these probabilities like points on a flat grid (like a standard X-Y graph). They use a "Gaussian kernel," which is essentially a bell curve.
The Mistake: If you draw a bell curve on a flat grid near the edge of your triangular paper, half of that curve spills off the paper. In math terms, this assigns probability to impossible things (like having -5% Doctors or 110% Doctors). This "leakage" causes the math to get messy and inaccurate, especially when the crystal ball is very confident (which pushes the data right up against the edges of the triangle).

The Solution: A New Compass and a Safety Net

The authors propose a new way to do this math that respects the shape of the triangle.

1. The New Compass (Compositional Data Analysis):
Instead of using a flat grid, they use a special mathematical transformation (called log-ratio) that turns the triangular paper into a flat surface without tearing it or spilling off the edges. It's like unfolding a map of the Earth so that distances and angles make sense relative to each other. This allows them to draw their "bell curves" (kernels) in a way that stays perfectly inside the triangle.

2. The Safety Net (Shrinkage):
There's a catch. When the crystal ball is too confident (saying 99.9% Doctor), the math on this new map can get wobbly and unstable near the very edges.
To fix this, the authors add a "safety net" called Shrinkage. Imagine pulling every point on the triangle slightly toward the center.

If the point is at the very edge (99.9% Doctor), the safety net pulls it back a tiny bit to (95% Doctor).
This stops the math from breaking down at the edges.
Crucially, they adjust the "zoom level" (bandwidth) of their bell curves so that this pulling doesn't distort the final answer. It's like zooming out just enough so that the pulled-back points still represent the original crowd accurately.

The Result: A Bayesian Crystal Ball

By combining this new "curved-world" map with the "safety net," the authors create a system that can do two things:

Point Estimation: Give you a single best guess for the crowd's makeup (e.g., "It's 85% Doctors").
Bayesian Inference: Give you a range of likely answers with a confidence level (e.g., "It's likely between 80% and 90% Doctors"). This is like the detective saying, "I'm pretty sure, but here is the margin of error."

What the Experiments Show

The team tested this on 42 different datasets (text, images, and spreadsheets).

Accuracy: Their method was consistently better than the old "flat map" methods, especially when the data was near the edges (high confidence predictions).
Robustness: Without the "safety net" (shrinkage), the new method sometimes failed spectacularly on specific datasets. With the safety net, it became reliable.
Uncertainty: Their method provided better confidence intervals than other Bayesian methods, meaning they were better at telling you when they were unsure.

In Summary

The paper says: "Stop trying to measure a triangular, interconnected system with a flat, independent ruler. Use a map designed for triangles, but add a safety net to keep the math from breaking at the edges. This gives you more accurate guesses about crowd demographics and a better sense of how sure you can be about those guesses."

Technical Summary: Geometry-Aware Bayesian Quantification via Compositional Data Analysis

1. Problem Statement

The paper addresses the problem of quantification (or class prevalence estimation) under label shift. In this setting, a classifier trained on a source distribution $P_s$ must estimate the unknown class distribution $P_t(Y)$ of a target domain, assuming the conditional distribution of features given labels remains invariant ( $P_s(X|Y) = P_t(X|Y)$ ).

While many methods operate directly on classifier posterior probabilities, the authors identify a fundamental geometric mismatch in existing continuous density-based approaches (specifically KDE-based quantifiers like KDEy). Posterior probability vectors lie on the probability simplex ( $\Delta^{K-1}$ ), constituting compositional data. Standard Kernel Density Estimation (KDE) relies on Euclidean Gaussian kernels, which ignore the relative structure of compositions. This leads to two critical issues:

Boundary Violation: Euclidean kernels assign non-zero probability mass outside the simplex boundaries.
Distorted Geometry: Euclidean distance does not respect the relative nature of compositions, potentially distorting similarity relationships between posterior vectors.

This mismatch is particularly severe near the simplex boundaries, where confident classifier predictions (common in real-world settings, especially with neural networks) concentrate.

2. Methodology

The proposed solution integrates Compositional Data Analysis (CoDA) with Bayesian inference and a novel shrinkage regularization.

A. Geometry-Aware Representation
Instead of modeling densities in the ambient Euclidean space, the method transforms posterior vectors $p$ into log-ratio coordinates using the Centered Log-Ratio (CLR) transformation:
$\phi(p) = \left( \log \frac{p_1}{g(p)}, \dots, \log \frac{p_K}{g(p)} \right)$
where $g(p)$ is the geometric mean of the components. This maps the simplex to a zero-sum subspace of $\mathbb{R}^K$ , enabling density estimation consistent with Aitchison geometry.

B. Shrinkage Regularization
The authors note that naive log-ratio transformations become unstable near the simplex boundary (where components approach zero). To mitigate this, they introduce a shrinkage transformation $T_\lambda$ :
$T_\lambda(p) = (1-\lambda)p + \lambda u$
where $u = (1/K, \dots, 1/K)$ is the barycenter and $\lambda \in [0, 1)$ .

Effect: This contracts the effective simplex toward the center, ensuring transformed compositions remain bounded away from zero.
Bandwidth Adjustment: The shrinkage uniformly rescales distances in the simplex hyperplane by $(1-\lambda)$ . Consequently, the effective bandwidth is adjusted to $h_{eff} = (1-\lambda)h$ to preserve smoothing levels.
Theoretical Justification: As $\lambda \to 1$ , the transformation linearizes the CLR map near the barycenter, regularizing the geometry without collapsing the model into a standard Euclidean KDE.

C. Bayesian Formulation
The method formulates quantification as a mixture density estimation problem. The likelihood of the target prevalence vector $\pi$ given observed posteriors $P$ is derived. Crucially, the Jacobian term arising from the change of variables (from simplex to log-ratio space) is independent of $\pi$ . This allows for a practical Bayesian posterior inference:
$p(\pi | P) \propto p(\pi) \prod_{i=1}^n \left( \sum_{k=1}^K \pi_k f_k^h(z^{(i)}) \right)$
where $z^{(i)} = \phi(T_\lambda(p^{(i)}))$ and $f_k^h$ are the class-conditional densities estimated in the transformed space. This formulation supports both point estimation (via Maximum Likelihood) and full Bayesian inference (via sampling, e.g., NUTS) over class prevalences.

3. Key Contributions

Geometry-Aware KDE: The first KDE-based quantifier that respects the simplex geometry of posterior probabilities using log-ratio transformations and Aitchison geometry.
Shrinkage Regularization: A novel mechanism to stabilize log-ratio modeling near the simplex boundaries, preventing numerical instability while preserving compositional structure.
Bayesian Framework for KDE: A principled derivation of Bayesian inference for KDE-based quantifiers, overcoming previous limitations where such formulations were restricted to discretized (histogram) models or binary problems.
Empirical Validation: Comprehensive evaluation across 42 datasets (tabular, text, and image) demonstrating competitive performance against state-of-the-art point estimators and superior results among Bayesian quantification methods.

4. Experimental Results

The authors evaluated the method (denoted KDEy(Ait- $\lambda$ ) for point estimation and Bayes-KDEy(Ait- $\lambda$ ) for Bayesian inference) against baselines including CC, PCC, BBSE/ACC, MLLS/EMQ, and standard Gaussian KDE (KDEy(Gau)).

Point Estimation: The proposed method achieved the best average rank in both Absolute Error (AE) and squared weight-ratio error (W) across all modalities (tabular, text, image). It frequently outperformed standard Gaussian KDE, particularly on tabular and image datasets.
Uncertainty Quantification: In the Bayesian regime, the method provided strong coverage and reasonable interval amplitudes. While Bayes-ACC showed slightly better raw coverage gaps at the cost of larger intervals and weaker point estimation, the proposed KDE-based Bayesian methods offered the best overall aggregate ranking.
Robustness to Shift Severity: The method dominated across low, medium, and high label shift regimes, with the exception of specific text datasets in low/medium shift where other Bayesian methods performed slightly better.
Boundary Performance: In experiments with target prevalences concentrated near the simplex boundary ( $\alpha_{test}=0.1$ ), the shrinkage-regularized Aitchison KDE significantly outperformed the Euclidean Gaussian KDE and bootstrap baselines, validating the necessity of respecting simplex geometry in these regimes.
Ablation: Removing shrinkage ( $\lambda=0$ ) led to sharp failures on specific benchmarks, confirming that shrinkage is not merely a numerical safeguard but an adaptive mechanism essential for stability.

5. Significance and Claims

The paper claims that respecting the simplex geometry of posterior probabilities is not only theoretically motivated but practically beneficial for multiclass quantification. The authors argue that:

Standard Euclidean KDEs introduce geometric mismatches that lead to biased or unstable prevalence predictions, especially near boundaries.
The proposed shrinkage-regularized log-ratio representation bridges the gap between the theoretical requirements of compositional data analysis and the computational practicality required for machine learning pipelines.
The derived Bayesian formulation enables principled uncertainty quantification for continuous density-based methods, a capability previously lacking in the multiclass setting.

The authors remain modest regarding limitations, noting that the current approach relies on validation-based selection of shrinkage and bandwidth parameters, and that a fuller Bayesian treatment of these hyperparameters remains an open direction. They also acknowledge that the method is currently specific to label shift, with covariate shift variants left for future work.