📊 statistics

MCMC Methods for Parameter Inference in Structurally Nonidentifiable Models

This paper proposes two novel Markov chain Monte Carlo methods that leverage structural identifiability analysis to improve sampling efficiency and convergence when inferring parameters in ordinary differential equation models characterized by structural non-identifiability.

Original authors: Xuyuan Wang, Donglin Han, Michael Y. Li

Published 2026-07-16

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Xuyuan Wang, Donglin Han, Michael Y. Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but the clues you have are a bit tricky. You are looking at a machine that does something complex, like a virus spreading through a city or a chemical reaction in a test tube. To understand how this machine works, you need to figure out the settings of its internal knobs—how fast the virus spreads, how quickly people recover, or how fast chemicals mix. This is the world of mathematical modeling, where scientists use equations to describe how things change over time. Usually, if you watch the machine long enough, you can figure out exactly what every single knob is set to. But sometimes, the machine is a "trickster." It has a secret: you can turn two different knobs in opposite directions, and the machine behaves exactly the same way. You can't tell which knob is which just by watching the output. In the scientific world, this is called "structural non-identifiability." It's like trying to guess the price of a sandwich and the price of a drink just by knowing the total cost of your lunch; you know the sum, but you can't know the individual prices without more information. This creates a huge problem for scientists who use computers to guess these settings. Their computers get stuck in a loop, spinning their wheels because they can't tell which direction is the "right" one, leading to slow, confusing results.

This paper tackles that exact problem. The authors, researchers from the University of Alberta, realized that when these "trickster" machines appear, standard computer methods for guessing the settings (called MCMC) are like a person trying to walk through a foggy maze by taking tiny, random steps. They get lost and take forever to find the exit. The paper proposes two new, smarter ways to navigate this maze by using the map of the trickiness itself.

The first method is like giving the detective a special "teleportation" power. Instead of just taking small steps, the computer learns to slide effortlessly along the "foggy paths" where the settings look identical. It can jump from one valid setting to another valid setting instantly, exploring the whole maze quickly before taking a step to a new area. The second method is even more clever: instead of trying to guess every single knob at once, the computer first guesses the combinations of knobs that actually matter (like the total cost of the lunch). Once it figures out the total, it works backward to guess the individual prices. This shrinks the giant, confusing maze into a much smaller, easier one to solve.

The researchers tested these new methods on two real-world scenarios: a model of how a flu-like illness spreads (the SI model) and a model of how HIV infects cells. In both cases, the standard computer methods were slow and got stuck, taking thousands of tries to get a decent answer. The new methods, however, were incredibly fast. In the flu model, the new "pseudo-marginal" method found the answer so efficiently that it produced over 5,000 useful guesses in the same time the old method only managed about 70. In the HIV model, the new methods were able to explore the "tricky" parts of the problem that the old methods completely missed. The paper shows that by understanding the specific way a model is "tricky," scientists can build better tools to solve them, turning a frustrating, slow process into a fast and reliable one.

Technical Summary: MCMC Methods for Parameter Inference in Structurally Non-identifiable Models

Problem Statement
The paper addresses the challenge of parameter inference in ordinary differential equation (ODE) models that exhibit structural non-identifiability. In such models, distinct parameter values yield identical observable outputs, creating lower-dimensional manifolds of observationally equivalent solutions within the parameter space. This phenomenon is prevalent in fields such as control theory, systems biology, and public health (e.g., infectious disease modeling).

Standard Bayesian inference and Markov chain Monte Carlo (MCMC) methods struggle in this context. Because the likelihood function remains constant along these non-identifiable manifolds, the posterior distribution is supported entirely on these manifolds (determined by the prior). Consequently, standard random-walk MCMC algorithms become trapped, exhibiting poor mixing, slow convergence, and high autocorrelation as they fail to explore the manifold efficiently.

Methodology
The authors propose two MCMC frameworks that explicitly incorporate information derived from structural identifiability analysis. Let $\xi(\theta)$ denote the mapping from the full parameter space $\Theta$ to a space of structurally identifiable combinations $C$ . The level sets $M_c = \{\theta \in \Theta : \xi(\theta) = c\}$ represent the non-identifiable manifolds.

Identifiability-Aware Geometric MCMC:
This method operates directly on the full parameter space $\Theta$ but utilizes the geometric structure of the manifolds $M_c$ . It employs a two-step proposal mechanism:
- Teleportation Move: Uses constrained Hamiltonian dynamics (specifically the RATTLE integrator) to generate proposals that move along the manifold $M_c$ . This allows for large, efficient jumps within the non-identifiable directions while preserving the constraint $\xi(\theta) = c$ .
- Transition Move: Proposes moves transverse to the manifold to explore different identifiable combinations. This uses an anisotropic proposal distribution that favors directions normal to the manifold, constructed using the Jacobian of $\xi$ .
  The combination of these moves is corrected via a Metropolis–Hastings acceptance step to ensure the chain targets the correct posterior distribution.
Identifiability-Aware Pseudo-Marginal MCMC:
This method performs inference on the lower-dimensional space of identifiable combinations $C$ rather than the full parameter space. It follows a two-stage process:
- Marginal Sampling: A Metropolis–Hastings algorithm samples the identifiable combinations $c \sim \pi_C(c)$ . Since the marginal likelihood involves an intractable integral over the manifold $M_c$ , the authors employ a pseudo-marginal approach. They construct an unbiased estimator of the marginal likelihood using importance sampling on the manifold.
- Reconstruction: Conditional on an accepted $c$ , the full parameter vector $\theta$ is reconstructed by sampling from the conditional distribution $\pi(\theta | c)$ on the manifold $M_c$ . This involves solving the algebraic system $\xi(\theta) = c$ (often via polynomial root finding) and using importance resampling to select a representative parameter sample.

Key Contributions

Algorithmic Development: The paper introduces two novel MCMC algorithms tailored for rational ODE models with structural non-identifiability. The geometric method extends existing frameworks to general rational ODEs using symplectic integrators, while the pseudo-marginal method generalizes likelihood-informed subspace approaches to nonlinear manifolds.
Theoretical Guarantees: The authors provide rigorous convergence proofs for both methods. They demonstrate that the geometric MCMC kernel is uniformly geometrically ergodic under standard compactness and continuity assumptions. For the pseudo-marginal method, they establish Harris ergodicity and provide error decomposition bounds separating the mixing error of the marginal chain from the reconstruction error of the conditional samples.
Dimension Reduction: The pseudo-marginal approach effectively reduces the dimensionality of the inference problem by separating identifiable and non-identifiable directions, mitigating the numerical instabilities associated with non-identifiable parameters.

Results
The methods were evaluated on two case studies: a Susceptible–Infectious (SI) model and an HIV infection model, both of which possess known structural non-identifiabilities.

SI Model: The baseline random-walk MCMC showed poor mixing with effective sample sizes (ESS) between 64 and 76. The geometric MCMC (Algorithm 2) improved ESS by an order of magnitude. The pseudo-marginal MCMC (Algorithm 4) achieved the highest efficiency, with ESS exceeding 4,500 for all parameters and integrated autocorrelation times (IACT) reduced to approximately 18–22.
HIV Model: Similar trends were observed. The proposed methods significantly outperformed the baseline for non-identifiable parameters ( $\lambda$ and $N$ ), reducing IACT from over 1,100 to under 30. Crucially, the results demonstrated that while individual non-identifiable parameters remain diffuse (consistent with theory), the proposed methods accurately recover the posterior distributions of identifiable parameter combinations (e.g., $\lambda N / c$ ), whereas the baseline method failed to do so.

Significance and Claims
The paper claims that structural identifiability analysis should not merely serve as a diagnostic tool for model calibration but can act as a principled foundation for designing efficient Bayesian computation algorithms. By exploiting the geometric structure induced by identifiable combinations, the proposed methods enable statistically valid and computationally efficient inference in settings where conventional sampling algorithms fail.

The authors note that while these methods improve sampling efficiency, they do not resolve the fundamental lack of information regarding non-identifiable parameters; rather, they correctly characterize the posterior uncertainty. The pseudo-marginal approach is highlighted as particularly effective for models with a small number of parameters where global solutions to the algebraic system are computationally feasible. The work suggests a path toward integrating identifiability analysis directly into the inference loop for complex mathematical models.

More like this