Reliable mechanistic operator recovery with biologically-informed neural networks: principles for architecture and optimisation design

Technical Summary: Reliable Mechanistic Operator Recovery with Biologically-Informed Neural Networks

Problem Statement
Biological processes are often governed by complex dynamical mechanisms described by partial differential equations (PDEs). While experimental data volumes are increasing, these datasets are frequently sparse, noisy, and irregularly sampled. A central challenge in mathematical biology is the inverse problem of recovering the underlying constitutive operators (e.g., diffusion, advection, reaction terms) from such limited observations. Classical inference methods struggle under these conditions, particularly when the mathematical form of governing operators is unknown. While Physics-Informed Neural Networks (PINNs) and their biological variants (BINNs) offer a hybrid framework by embedding differential equations into the loss function, the reliability of operator recovery in BINNs is highly sensitive to architectural design, optimization strategies, and data informativeness. There is a lack of systematic understanding regarding how these factors interact to determine whether a BINN successfully recovers the true biological mechanisms versus merely interpolating noisy data.

Methodology
The authors present a systematic empirical study using a general Advection-Diffusion-Reaction BINN (ADR-BINN) framework applied to a suite of canonical one-dimensional PDE models.

• Models: The study spans increasing mechanistic complexity, including linear diffusion, nonlinear (porous-medium) diffusion, Burgers' equation (nonlinear advection-diffusion), linear diffusion with nonlinear advection, and a fully nonlinear reaction-diffusion-advection system.
• Framework: The ADR-BINN employs four Multi-Layer Perceptrons (MLPs): one ($u_\theta$) to approximate the latent solution $u(x,t)$, and three separate networks ($D_\phi, V_\zeta, G_\psi$) to learn the unknown constitutive operators for diffusivity, advection velocity, and reaction, respectively.
• Training: The networks are trained jointly to minimize a composite loss function $L_{total} = w_{data}L_{data} + w_{PDE}L_{PDE} + w_{BC}L_{BC}$. This includes a data-fitting term, a PDE residual term evaluated at randomly sampled collocation points, and a boundary condition term.
• Evaluation Metric: A critical distinction is made between the learned solution ($u_\theta$), which interpolates the data, and the forward-predicted solution ($u_p$), obtained by numerically solving the PDE using the learned operators. The authors argue that agreement between $u_p$ and the true dynamics is the necessary condition for successful mechanistic recovery, whereas $u_\theta$ accuracy alone is insufficient.
• Variables Investigated: The study systematically varies network expressivity (depth/width), learning rates, loss weightings ($w_{data}$ vs. $w_{PDE}$), and mini-batch sizes across synthetic datasets with varying noise levels ($\sigma^2$) and spatial sampling densities.

Key Contributions and Results
The study identifies that reliable operator recovery is governed by balancing competing objectives rather than maximizing any single aspect of the model or optimization.

1. Robustness to Noise and Architecture: BINNs are robust to moderate observation noise. However, overly complex networks (e.g., 6 layers, 128 nodes) are more susceptible to over-fitting noise compared to moderately expressive architectures (e.g., 4 layers, 32 nodes). For noisy biological data, simpler architectures provide beneficial implicit regularization.
2. Learning Rate Optimization: Intermediate learning rates (approximately $10^{-4}$) are optimal. Very small rates lead to under-fitting and slow convergence, while very large rates cause unstable optimization and divergence. Validation loss curves serve as effective diagnostics for identifying these regimes.
3. Loss Weighting Balance: Accurate operator recovery requires a specific balance between data-fitting and PDE residual losses. If the PDE weight ($w_{PDE}$) is too low, the model interpolates data perfectly but fails to recover the correct operators (high $u_\theta$ accuracy, poor $u_p$ accuracy). If $w_{PDE}$ is too high, the model prioritizes the PDE constraint at the expense of data fidelity. The authors find that $w_{data} \gtrsim w_{PDE}$ is generally effective.
4. Operator Network Expressivity: Successful recovery requires only sufficient network capacity to represent the complexity of the underlying constitutive functions. Increasing network depth and width beyond this point yields diminishing returns, increases computational cost, and can reduce reproducibility due to mild over-fitting.
5. Batch Size Trade-offs: Intermediate batch sizes (e.g., 40–160) provide the best compromise. Small batches introduce high stochasticity that aids exploration but increases computational cost and variability between runs. Large batches reduce stochasticity but limit the exploration of the loss landscape, leading to poorer reconstruction accuracy within fixed time budgets.

Significance and Claims
The paper claims that reliable mechanistic inference with BINNs is not achieved by maximizing network complexity or optimization effort, but by appropriately balancing model expressivity, optimization parameters, physical consistency, and data informativeness.

• Diagnostic Framework: The authors provide practical diagnostics for recognizing failure modes (over-fitting, unstable optimization, poor mechanistic recovery) in the absence of ground truth. Specifically, they advocate monitoring the discrepancy between the learned solution ($u_\theta$) and the forward-predicted solution ($u_p$); a large discrepancy indicates that the operators have not been correctly identified despite good data interpolation.
• Guidance for Biological Applications: The findings offer evidence-based guidelines for deploying BINNs in biological model discovery. The authors suggest that practitioners should prefer moderately expressive architectures, ensure data-fitting remains influential relative to PDE residuals, and use intermediate learning rates and batch sizes.
• Limitations: The study acknowledges that its benchmarks rely on synthetic, one-dimensional data with known ground truth. It does not address higher-dimensional domains, heterogeneous media, or formal Bayesian uncertainty quantification, focusing instead on isolating the effects of architecture and optimization to establish foundational principles.

In conclusion, the work demonstrates that BINNs can serve as credible tools for biological model discovery, provided that the competing objectives of data fidelity, physical consistency, and optimization stability are carefully balanced.