⚡ electrical engineering

A Hierarchical Likelihood Model for Non-linear Inverse Problems under Additive and Multiplicative Noise

This paper proposes a general hierarchical Bayesian model coupled with an efficient MCMC algorithm to solve complex non-linear inverse problems involving additive and multiplicative noise and censored data, demonstrating superior predictive performance and computational efficiency compared to existing approximate methods without requiring hyperparameter calibration.

Original authors: Nicolas Goeman, Pierre-Antoine Thouvenin, Pierre Chainais

Published 2026-07-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nicolas Goeman, Pierre-Antoine Thouvenin, Pierre Chainais

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but your clues are a bit broken. Sometimes your magnifying glass is a little blurry (that's additive noise, like static on a radio), and sometimes the light source itself flickers unpredictably (that's multiplicative noise, like a candle in a draft). To make things worse, your flashlight has a "blind spot" where it just can't see anything below a certain brightness (this is censored data). In the real world, scientists face this exact problem when trying to figure out what's happening deep inside stars or clouds of gas. They have a mathematical model—a "forward operator"—that predicts what the data should look like if they knew the answer. But because the math is wildly complicated and the clues are messy, working backward to find the answer is like trying to guess the shape of a hidden object by feeling a shadow that keeps changing size and shape.

Usually, when the math gets too hard, scientists have to make a guess. They might pretend the flickering light doesn't exist, or they might use a simplified version of the math that is easier to calculate but isn't perfectly true. This is a bit like trying to navigate a city using a map that only shows the main roads and ignores all the winding alleys; you might get close, but you'll miss the details. The big question is: how do we get a reliable answer when the clues are this messy, and how do we know how much we can trust that answer? This is where Bayesian inference comes in. Instead of giving you just one single "best guess," it gives you a whole range of possibilities and tells you how likely each one is, effectively handing you a confidence score along with the answer.

This paper introduces a new, more sophisticated way to solve these messy puzzles. The authors, Nicolas Goeman, Pierre-Antoine Thouvenin, and Pierre Chainais, propose a "hierarchical" model. Think of this as adding a middleman to your detective work. Instead of trying to jump straight from the broken clues to the hidden truth, they introduce an invisible "helper" variable that separates the blurry static from the flickering light. By doing this, they can handle both types of noise and the blind spots at the same time without having to simplify the math or make shaky guesses.

The team tested this new method using synthetic data—computer-generated simulations of an interstellar cloud, which is a giant cloud of gas and dust in space. They set up a scenario where the data had high dynamic ranges (meaning some parts were incredibly bright and others incredibly dim), was corrupted by both types of noise, and had about 20% of the data "censored" (hidden because it was too dim to see). They compared their new hierarchical model against three other approaches: a method that just looks at the additive noise, a method that just looks at the multiplicative noise, and a popular "interpolated" method that tries to blend the two.

The results, based on these simulations, show that the new hierarchical approach is the most versatile and accurate. It managed to reconstruct the hidden parameters of the cloud with less error than the other methods. Crucially, it also provided a much better picture of the uncertainty, telling the scientists exactly where their guesses were shaky and where they were solid. The paper argues that the older methods, which rely on approximations, can lead to biased results because they force a complex reality into a simpler box. The new method, however, bypasses the need for these tricky approximations and the difficult calibration of "hyperparameters" (the dials you have to turn to make the math work). While the paper doesn't claim to have solved every inverse problem in the universe, it suggests that for problems involving non-linear models, mixed noise, and censored data, this hierarchical approach is a superior tool that yields state-of-the-art results in both accuracy and computational efficiency.

Technical Summary: A Hierarchical Model for Non-linear Inverse Problems under Additive and Multiplicative Noise

Problem Statement
The paper addresses a class of ill-posed non-linear inverse problems characterized by three simultaneous challenges: highly non-linear forward operators, the presence of both additive and multiplicative noise sources, and censored data (observations limited by instrument sensitivity thresholds). These conditions are prevalent in applications such as astronomy, ultrasound imaging, and Synthetic Aperture Radar (SAR). A critical difficulty arises when the forward model spans a high dynamic range, causing the dominant noise source to vary with amplitude. Furthermore, in many practical scenarios, ground truth data is unavailable, necessitating robust uncertainty quantification to assess estimation reliability.

Existing approaches often simplify the problem by neglecting one noise source or employing tractable approximations of the likelihood function (e.g., smoothing transitions between noise regimes or using Bayesian approximation error methods). However, these approximations introduce biases that are difficult to calibrate and have not been comprehensively assessed. Specifically, prior works handling all these factors simultaneously often rely on approximate likelihoods with intractable normalization constants or require fine-tuned hyperparameters for the transition between noise regimes.

Methodology
The authors propose a general hierarchical Bayesian model combined with an efficient Markov Chain Monte Carlo (MCMC) algorithm to address these limitations without resorting to likelihood approximations.

Reparameterization: To handle the high dynamic range of physical parameters $\Phi$ , the model operates in a log-normalized space $\Theta$ via an element-wise transformation. This improves numerical stability during inference.
Hierarchical Formulation: The core contribution is a two-stage reformulation of the observation model. An auxiliary variable $u_{n,\ell}$ $u_{n, ℓ}$ is introduced to decouple the multiplicative noise from the additive noise and the censoring mechanism.
- The observation equation is rewritten as $y_{n,\ell} = \max\{\omega_{n,\ell}, u_{n,\ell} + \epsilon_{n,\ell}\}$ , where $u_{n,\ell} = \xi_{n,\ell} f_\ell(\theta_n)$ .
- Here, $\xi_{n,\ell}$ represents the multiplicative noise (log-normal) and $\epsilon_{n,\ell}$ the additive noise (Gaussian).
- This formulation yields a tractable likelihood function that exactly accounts for both noise sources and censoring, eliminating the need for approximate likelihoods or complex hyperparameter calibration.
Prior Distribution: The model employs a prior that enforces spatial smoothness via a squared $\ell_2$ -norm of discrete gradients and constrains parameters within a compact set using a smooth approximation of an indicator function.
Inference Algorithm: To sample from the resulting complex, potentially multimodal posterior distribution, the authors propose a conditional Metropolis-within-Gibbs sampler.
- Block Updates: Parameters $\theta_n$ and auxiliary variables $u_n$ are updated jointly to account for their strong coupling.
- Parallelization: A chromatic Gibbs approach updates independent pixel groups in parallel.
- Kernel Mixture: The transition kernel is a random mixture of two strategies:
  - A local exploration kernel using a Position-Dependent Preconditioned Metropolis-Adjusted Langevin Algorithm (PMALA).
  - A global exploration kernel using an Independent Multiple-Try Metropolis (I-MTM) algorithm to escape local modes.
- Adaptive Proposals: For the auxiliary variables, the proposal distribution adapts based on whether the observation is censored. For non-censored data, a Gamma distribution is used to approximate the skewed conditional posterior; for censored data, ancestral sampling from the prior is employed.

Key Contributions

Exact Hierarchical Model: The paper introduces a hierarchical formulation that bypasses the need for approximate likelihoods, providing an exact treatment of additive noise, multiplicative noise, and censoring simultaneously.
Efficient MCMC Sampler: A novel sampling strategy is developed that combines local gradient-based moves with global exploration, specifically tailored to handle the non-linearity and multimodality of the posterior.
Comprehensive Evaluation: The study provides a rigorous comparison between the proposed exact model and state-of-the-art approximate methods (including an interpolated likelihood approach and models considering only one noise type).
Model Selection Guide: By evaluating Expected Log Predictive Density (ELPD) and reconstruction quality, the work offers a guide for practitioners to select appropriate likelihood models based on specific application contexts.

Results
The proposed method was evaluated on synthetic astronomical data simulating an interstellar cloud, featuring $N=64$ pixels, $L=10$ spectral bands, and $D=4$ parameters. Experiments covered various levels of multiplicative noise (low, moderate, high) and censoring rates (~20%).

Reconstruction Quality: The proposed hierarchical model achieved state-of-the-art performance in terms of Mean Absolute Error (MAE) for point-wise estimates, outperforming the interpolated likelihood and single-noise models.
Uncertainty Quantification: The method provided superior predictive performance, as measured by ELPD, indicating a better alignment with the true generative distribution of the data compared to approximate baselines.
Computational Cost: The approach demonstrated competitive computing costs per iteration, leveraging parallel updates and efficient proposal distributions.
Robustness: Results were consistent across multiple random seeds, and the method successfully handled the high dynamic range and non-linearity without the bias introduced by likelihood approximations.

Significance
The paper claims that its primary significance lies in offering a versatile, general framework for solving complex non-linear inverse problems where ground truth is unavailable. By avoiding the biases inherent in approximate likelihoods, the proposed method yields more reliable uncertainty quantification and parameter estimates. The comprehensive study serves as a practical guide for researchers to understand the impact of likelihood modeling choices on estimation quality, advocating for the use of exact hierarchical formulations when computational resources permit. The work is supported by the ANR project "Chaire IA Sherlock" and utilizes resources from the University of Lille.

More like this