A likelihood-based framework for simultaneously learning both noise and growth dynamics using biologically-informed neural networks

This paper introduces a likelihood-based extension to Biologically-Informed Neural Networks (BINNs) that simultaneously learns mechanistic growth dynamics and heteroscedastic noise structures from sparse data, thereby improving prediction accuracy over existing methods that assume homoscedastic Gaussian noise.

The Gist

Imagine you are trying to figure out how a population of animals (like fish in a pond or coral on a reef) grows over time. You have a notebook of observations, but there's a catch: your notebook is messy. The numbers are a bit fuzzy, and the "fuzziness" (noise) gets worse as the population gets bigger.

Traditionally, scientists trying to understand this growth have used two main tools:

1. The Mechanistic Model: A set of math rules (like a recipe) that describes how the population should grow based on biology.
2. The Machine Learning Model: A smart computer program that looks at the messy data and tries to guess the pattern without needing a pre-written recipe.

The problem is that most existing methods treat the "messiness" in the data as a boring, constant background error. They assume the noise is the same whether the population is tiny or huge. But in biology, noise often changes. If you have a small group, a few missing fish might not matter much. If you have a massive group, a small percentage error means a huge number of fish. This is called heteroscedastic noise (fancy talk for "changing noise").

The New Solution: The "Smart Detective" (NLL-BINN)

The authors of this paper, Rebecca Crossley and Ruth Baker, created a new tool called NLL-BINN. Think of it as a super-smart detective that doesn't just look at the clues (the data) but also investigates how the clues were collected.

Here is how it works, using simple analogies:

1. Learning the Recipe and the Messiness at the Same Time
Imagine you are trying to guess a secret recipe for a cake, but every time you taste it, the flavor is slightly off because the chef's hand shakes.

• Old methods would say, "The chef's hand shakes the same amount every time," and try to average out the taste.
• The new method (NLL-BINN) says, "Wait, the chef's hand shakes more when the cake is bigger." It learns the recipe (how the population grows) and learns the rule for how the shaking (noise) changes as the cake grows, all at the same time.

2. The "Power-Law" Rule
The team taught their AI a specific rule for how the noise behaves. They call it a "power-law."

• If the noise is Additive (like static on a radio), it stays the same no matter how big the population is.
• If the noise is Multiplicative (like a percentage error), it gets bigger as the population gets bigger.
• The new framework can figure out which rule applies just by looking at the data, without being told beforehand.

3. Why This Matters: Better Guesses
The paper tested this on three classic growth models (Logistic, Gompertz, and Richards). These are like three different types of "growth recipes" that look very similar on the surface but have different underlying mechanics.

• The Result: When the data was messy, the old methods (which assumed constant noise) got confused and guessed the wrong recipe. The new NLL-BINN method correctly identified the recipe and figured out that the noise was changing.
• The Analogy: It's like trying to hear a conversation in a noisy room. If you know the noise gets louder as the speaker gets louder, you can tune your ears better to understand the words. If you assume the noise is constant, you might miss the words entirely.

4. Real-World Test: Coral Reefs
They also tested this on real data from coral reefs in Australia. In this case, they only had one line of data (no repeated experiments to measure the noise separately).

• Even with this limited data, the AI successfully figured out that the coral growth followed a specific pattern and that the uncertainty in the measurements grew as the coral cover increased.
• This is a big deal because usually, you need lots of repeated experiments to understand how "noisy" your data is. This method can find that pattern from a single, messy dataset.

The Bottom Line

This paper introduces a way to teach computers to understand biological growth that is more honest about reality. Instead of pretending the data is perfect or that the errors are boring and constant, the new framework admits that uncertainty is part of the system.

By learning the "rules of the mess" alongside the "rules of growth," the computer can make much more accurate predictions about how biological systems behave, even when the data is sparse or noisy. It turns the "noise" from a nuisance into a useful clue about how the system works.

Technical Summary

Technical Summary: A Likelihood-Based Framework for Simultaneously Learning Noise and Growth Dynamics

Problem Statement
Biological data is inherently noisy, sparse, and often exhibits heteroscedasticity, where the magnitude of variability depends on the system's state (e.g., population density). While Biologically-Informed Neural Networks (BINNs) have shown promise in learning mechanistic laws from such data, existing approaches typically rely on implicit assumptions of homoscedastic (constant-variance) Gaussian noise, often implemented via Mean Squared Error (MSE) or Root Mean Squared Error (RMSE) loss functions. This treatment views variability merely as a nuisance to be minimized, failing to account for potentially meaningful structures in biological variability. Consequently, these methods may produce biased dynamics or poor uncertainty quantification when the underlying noise is state-dependent.

Methodology: The NLL–BINN Framework
The authors propose the Negative Log-Likelihood trained BINN (NLL–BINN), a likelihood-based extension to the standard BINN framework that jointly learns system dynamics and a parametric noise model.

1. Model Formulation:

   • Dynamics: The system is modeled as an Ordinary Differential Equation (ODE): $du/dt = u \cdot g(u; p)$, where the growth law $g(u)$ is approximated by a neural network $g_\phi(u)$. The solution $u(t)$ is approximated by a trajectory network $u_\theta(t)$.
   • Noise Model: Instead of assuming constant variance, the framework models observations $u^o_i$ as $u^o_i = u(t_i) + \epsilon \sigma(u(t_i))$, where $\epsilon \sim \mathcal{N}(0, 1)$. The noise magnitude $\sigma(u)$ is parameterized using a power-law form: $\sigma(u) = \sigma_0 |u|^\alpha$.
     • $\alpha = 0$: Additive (homoscedastic) noise.
     • $\alpha = 1$: Multiplicative noise.
     • $0 < \alpha < 1$: Intermediate heteroscedastic noise.
   • Learnable Parameters: Both the network weights ($\theta, \phi$) and the noise parameters ($\sigma_0, \alpha$) are learned simultaneously from the data.

2. Loss Function:
   The total loss function combines three terms:
   $$L_{total} = w_{data}L_{data} + w_{ODE}L_{ODE} + w_{bio}L_{bio}$$

   • Data Loss ($L_{data}$): Derived from the negative log-likelihood (NLL) under the Gaussian noise assumption with density-dependent variance. This term penalizes the discrepancy between observations and predictions, weighted by the learned variance.
   • ODE Loss ($L_{ODE}$): Enforces the governing equation $du/dt - u \cdot g(u) = 0$ at randomly sampled collocation points across the time domain.
   • Biological Loss ($L_{bio}$): Enforces soft constraints, specifically the non-negativity of population densities.

3. Training Procedure:
   The framework utilizes Multi-Layer Perceptrons (MLPs) with tanh activations, trained via Adam optimization. To ensure robustness, an ensemble of ten independent models is trained with different random seeds, and results are reported as ensemble means and standard deviations.

Key Contributions

• Joint Learning of Dynamics and Noise: The primary contribution is a framework that infers the noise structure (specifically the scaling exponent $\alpha$) directly from data, rather than prescribing it.
• Mechanistic Interpretation of Uncertainty: By treating noise as a function of population density, the approach links statistical variability to underlying biological mechanisms, moving beyond uncertainty as a mere statistical artifact.
• Unified Power-Law Formulation: The use of a single power-law parameterization allows the model to seamlessly interpolate between additive, multiplicative, and intermediate noise regimes without requiring predefined function libraries.

Results
The framework was evaluated on synthetic datasets generated from Logistic, Gompertz, and Richards' growth models, as well as real-world coral reef re-growth data.

1. Recovery of Dynamics and Noise:

   • The NLL–BINN successfully reconstructed smooth underlying trajectories and the correct crowding functions ($g(u)$) for all three synthetic models, even in the presence of significant noise.
   • The framework accurately identified the noise scaling parameters ($\alpha$ and $\sigma_0$). For additive noise, $\alpha \approx 0$; for multiplicative noise, $\alpha \approx 1$; and for intermediate noise, the model correctly recovered intermediate values.
   • Note: A small bias in $\alpha$ was observed for additive noise near zero density, attributed to the "clipping" of negative values to ensure biological plausibility, which distorts the Gaussian distribution.

2. Uncertainty Calibration:

   • The predictive uncertainty intervals were well-calibrated. Approximately 68% of observations fell within one standard deviation and 95% within two, matching theoretical expectations for Gaussian noise, without explicit enforcement of coverage constraints during training.

3. Mechanistic Accuracy vs. RMSE Training:

   • Comparing NLL–BINN to standard BINNs trained with RMSE loss, the NLL–BINN consistently achieved lower "mechanistic error" (the error between the learned dynamics and the true noise-free system).
   • This improvement was most pronounced for Logistic and Gompertz models. For the Richards' model, improvements were more modest due to practical non-identifiability issues arising from its additional shape parameter, which allows different parameters to produce similar trajectories.

4. Real-World Application:

   • Applied to coral reef re-growth data from the Great Barrier Reef (single trajectories without replicates), the framework inferred density-dependent noise ($\alpha \approx 1.57 - 1.64$) and distinct growth mechanisms for two different sites. This demonstrated the ability to extract noise structure from limited data where replication is unavailable.

Significance and Claims
The paper claims that the NLL–BINN framework establishes a general, likelihood-based approach for jointly learning dynamics and heteroscedastic noise within mechanistic neural networks. Its significance lies in:

• Improved Robustness: By weighting observations according to their noise level, the method reduces the influence of high-variance data points on the learned dynamics, leading to more accurate recovery of the underlying biological laws compared to MSE-based approaches.
• Interpretability: It provides a data-driven estimate of observational uncertainty and links noise structure to population size, offering insights into the sources of variability in biological systems.
• Computational Efficiency: Unlike fully Bayesian approaches that require expensive posterior inference, this method offers a computationally efficient, interpretable representation of heteroscedastic noise.

The authors conclude that while the method improves mechanistic recovery and uncertainty quantification, the identifiability of the model remains dependent on data distribution; regions with sparse data or data populated solely by noise (e.g., densities exceeding carrying capacity due to measurement error) may yield higher uncertainty. The framework is presented as a complementary tool to probabilistic neural ODEs, trading full posterior inference for efficiency and interpretability.