Nonparametric Riemannian Empirical Bayes, and Denoising Measurements on Manifolds

Technical Summary: Nonparametric Riemannian Empirical Bayes and Denoising on Manifolds

Problem Formulation
The paper addresses the problem of denoising latent variables $\Theta$ and their measurements $X$ when both lie on a compact, connected Riemannian manifold $M$. The statistical model assumes a latent variable $\Theta \sim G$, where $G$ is an unknown probability distribution on $M$, and a measurement $X$ conditioned on $\Theta = \theta$ follows a Riemannian Gaussian distribution $P_{\theta, \sigma^2}$. The density of this likelihood is given by $p_{\theta, \sigma^2}(x) \propto \exp(-d^2(x, \theta)/(2\sigma^2))$, where $d$ is the geodesic distance and $\sigma^2$ is a known noise scale. The objective is to estimate the latent variables $\Theta_1, \dots, \Theta_n$ from observed measurements $X_1, \dots, X_n$ by finding a denoiser $\delta: M \to M$ that minimizes the expected squared geodesic distance risk.

Methodology
The authors develop a nonparametric empirical Bayes approach based on $f$-modeling (estimating the marginal density of the data). The methodology proceeds through three main theoretical and algorithmic steps:

1. The Tangential Bayes Surrogate:
   The optimal Bayesian denoiser, $\delta_B(x)$, is the posterior Fréchet mean, which minimizes the posterior expected loss. However, $\delta_B$ is intractable as it depends on the unknown $G$ and requires iterative Riemannian optimization for each data point.
   The authors derive a Riemannian version of the Tweedie-Eddington formula:
   $$E[\text{Log}_x \Theta \mid X = x] = \sigma^2 \nabla_x \log f(x)$$
   where $f$ is the marginal density of $X$ and $\text{Log}_x$ is the logarithmic map. This identity motivates the definition of a surrogate oracle denoiser, the oracle tangential Bayes denoiser $\delta_T$:
   $$\delta_T(x) := \text{Exp}_x(\sigma^2 \nabla_x \log f(x))$$
   Geometrically, $\delta_T(x)$ corresponds to a single step of Riemannian gradient descent on the posterior loss function, initialized at the naive denoiser $\delta_N(x) = x$.

2. Empirical Approximation:
   To construct a fully data-driven estimator $\hat{\delta}_T$, the marginal density $f$ is estimated using spectral theory. Specifically, the authors utilize the eigenfunctions $\{\phi_j\}$ of the Laplace-Beltrami operator $\Delta$ on $M$ to construct a kernel density estimator:
   $$\hat{f}_n(x) = \frac{1}{n} \sum_{i=1}^n K(x, X_i, \ell_n), \quad \text{where } K(x, y, \ell) = \sum_{j=0}^\ell \phi_j(x)\phi_j(y)$$
   The empirical tangential Bayes denoiser is then defined as:
   $$\hat{\delta}_T(x) := \text{Exp}_x\left( \sigma^2 \frac{\nabla_x \hat{f}_n(x)}{\max\{\hat{f}_n(x), \rho\}} \right)$$
   where $\rho > 0$ is a regularization parameter to handle low-density regions.

3. Convergence Analysis:
   The paper establishes that $\delta_T$ is close to $\delta_B$ in the low-noise regime ($\sigma^2 \to 0$) and that $\hat{\delta}_T$ converges to $\delta_T$ as the sample size $n \to \infty$.

Key Contributions and Results

• Oracle-Level Risk Comparison: The authors prove that for any compact Riemannian manifold, the risk difference between the tangential Bayes denoiser and the optimal Bayes denoiser is $O(\sigma^5)$ as $\sigma^2 \to 0$. This establishes $\delta_T$ as a near-optimal surrogate that avoids the computational complexity of calculating the posterior Fréchet mean.
• Nonparametric Convergence Rates: Unlike the Euclidean setting where Gaussian mixture densities are infinitely differentiable, Riemannian Gaussian mixture densities can exhibit singularities at the cut locus of the Fréchet mean. This "smeariness" phenomenon leads to slower, genuinely nonparametric convergence rates for $\hat{\delta}_T$.
  • The convergence rate of the distance $D_T(\hat{\delta}_T)$ depends on the Sobolev smoothness $r$ of the marginal density $f$.
  • For the circle $M=S^1$, the authors establish matching upper and lower bounds, proving that $\hat{\delta}_T$ is minimax-optimal with a rate of $n^{-(2s+2)/(2s+5)}$, where $s$ is the smoothness of the mixing density $G$. This confirms the problem exhibits a nonparametric rate of convergence.
• Sobolev Regularity Analysis: The paper provides detailed bounds on the Sobolev norms of Riemannian Gaussian mixtures. It highlights that the smoothness of the marginal density $f$ is not automatically infinite (as in Euclidean space) but depends on the geometry of $M$ and the smoothness of $G$. For Lie groups, $f$ inherits the smoothness of $G$; for the sphere $S^2$, $f$ gains smoothness but with specific constraints due to cut locus singularities.
• Scientific Applications: The methodology is applied to two real-world problems:
  1. Astronomy: Denoising the locations of 2,702 gamma-ray bursts on the sphere $S^2$. The empirical Bayes denoiser successfully identified clusters of bursts that were obscured by noise, outperforming the naive approach.
  2. Structural Biology: Denoising pairs of torsion angles (Ramachandran plots) for amino acids in proteins on the torus $S^1 \times S^1$. The denoiser effectively shrunk data toward high-density regions corresponding to known secondary structures ($\alpha$-helices and $\beta$-sheets) while stretching away from low-density regions.

Significance and Claims
The paper claims to initiate the study of nonparametric empirical Bayes denoising in the setting of Riemannian manifolds with Riemannian Gaussian likelihoods. Its primary significance lies in:

1. Bridging Theory and Practice: It provides a computationally tractable, fully data-driven denoiser ($\hat{\delta}_T$) that achieves nearly the Bayes risk without requiring knowledge of the latent distribution $G$ or expensive iterative optimization for each point.
2. Characterizing Non-Euclidean Phenomena: It rigorously identifies and quantifies the "smeariness" effect caused by the cut locus in Riemannian geometry, demonstrating that this leads to slower, nonparametric convergence rates distinct from the nearly-parametric rates found in Euclidean empirical Bayes.
3. Minimax Optimality: By establishing matching lower bounds for the circle case, the authors demonstrate that their proposed method is minimax-optimal, confirming that the slower rates are inherent to the problem geometry rather than a limitation of the estimator.
4. Practical Utility: Through simulations and scientific applications, the paper demonstrates that pooling information across measurements via empirical Bayes (shrinking toward high-density regions) substantially outperforms treating measurements independently, even in complex geometric settings like the sphere and torus.

The authors maintain a modest stance regarding the generality of their smoothness analysis, noting that a complete quantitative control of the smoothing properties of Riemannian Gaussian mixtures for arbitrary manifolds remains an open problem, particularly regarding the precise behavior near cut loci. They frame their work as a foundational step for future extensions to $g$-modeling and compound decision problems.