A Closed-loop Framework to Discriminate Models Using Optimal Control

1. Problem Statement

Mathematical modeling is essential for predicting and controlling dynamical systems, particularly in biology. However, a significant challenge arises when multiple candidate models with varying complexities and mechanisms can fit the same available experimental data. This phenomenon, often called model indistinguishability, occurs because:

• Experimental data often corresponds to macroscopic observables (e.g., firing rates, currents) that depend on many microscopic degrees of freedom.
• Standard data collection often occurs within a limited subspace of the system's phase space, where different models may appear equally valid.
• Fully data-driven machine learning approaches struggle to generalize outside the training data's phase space, while traditional statistical model selection (e.g., BIC, AIC) may fail if the data lacks features distinguishing the true underlying mechanism.

The core problem is how to actively design experiments (specifically, input control signals) that force the system into regions of the phase space where candidate models diverge significantly, thereby allowing for the unambiguous selection of the most accurate model.

2. Methodology

The authors propose a closed-loop iterative framework that combines parameter fitting with optimal control to discriminate between candidate models. The workflow operates as follows:

A. System Formulation

The system is modeled as a set of candidate dynamical systems:
$$\dot{X}_k = F_k(X_k, u(t); \Theta_k)$$
$$Y_k = H_k(X_k; \Theta_k)$$
where $u(t)$ is the control input, $X_k$ are state variables, $\Theta_k$ are parameters, and $Y_k$ are observables. The goal is to find the control $u(t)$ that maximizes the discrepancy between the outputs of two competing models ($Y_1$ and $Y_2$).

B. The Iterative Loop

1. Initialization: An arbitrary initial control input $u(t)$ is applied to the "reference system" (the real system or a simulation) to generate initial data.
2. Parameter Fitting: Parameters ($\Theta_1, \Theta_2$) for all candidate models are fitted to the observed data using a gradient-based optimization (e.g., Adam optimizer) to minimize a loss function (Euclidean distance between model output and data).
3. Optimal Control Design: An optimal control problem is solved to find a new input signal $u^*(t)$ that maximizes a performance measure $J$:
   $$J(u) = \int_0^T \left( \|Y_1(u) - Y_2(u)\|^2 - C(u) \right) dt$$
   • The first term maximizes the difference between model outputs.
   • The second term $C(u)$ penalizes unrealistic control signals (e.g., excessive amplitude) and includes a "variety" term to prevent the algorithm from getting stuck in repetitive control profiles.
   • Constraints ensure the control remains within physical bounds and the models' dynamics are satisfied.
4. Execution: The optimized control signal $u^*(t)$ is applied to the reference system to generate a new dataset.
5. Iteration: Steps 2–4 repeat. The process stops when:
   • One model's parameters converge and its prediction error falls below a tolerance.
   • Both models converge, in which case the simpler model (Occam's Razor) is selected.
   • A maximum number of iterations is reached.

C. Implementation Details

• Discretization: The continuous optimal control problem is discretized using the trapezoidal rule and solved using the interior-point method solver Ipopt.
• Hardware-Software Interface: For experimental validation, the authors built a closed-loop system using Symphony (Matlab-based electrophysiology software) communicating via TCP with a Julia server for rapid computation of control signals.

3. Key Contributions

• Active Model Discrimination: Unlike passive observation, this framework actively designs inputs to maximize the discriminability between models, effectively "probing" the system where models disagree.
• Closed-Loop Integration: The method uniquely integrates parameter fitting and control design in a loop. It re-injects the designed signal into the real system, allowing the algorithm to adaptively refine the discrimination strategy based on new data.
• Handling Structural Identifiability: The approach does not require unique parameter recovery (structural identifiability) for every parameter; it only requires that the model structure that best predicts the system's response can be identified.
• Robustness to Noise: The framework was tested with stochastic differential equations (SDEs) to simulate real-world noise, demonstrating that the iterative nature helps prevent overfitting and still selects the correct model structure.

4. Results

A. Numerical Simulations (Opsin Models)

The authors tested the framework on Channelrhodopsin (opsin) models, which describe light-induced ion currents. They compared 3-state, 4-state, and 6-state Markov models.

• Standard vs. Optimal Control: Using standard "box" (step) inputs, the 4-state model could overfit the 3-state reference data, making them indistinguishable. However, the optimal control signals generated by the framework created trajectories where the 3-state model failed to match the reference, while the 4-state model succeeded (and vice versa).
• Complex Scenarios: The framework successfully identified the correct model even when the reference system was a stochastic (noisy) version of a specific model, or when the reference system was a complex model not exactly represented by the candidates (selecting the best approximation).

B. Laboratory Experiments

The method was applied to a real-world electrophysiology experiment using mammalian cells expressing ChR2 (Channelrhodopsin-2).

• Setup: Cells were voltage-clamped, and light intensity was controlled via an LED driven by the algorithm's output.
• Discrimination:
  • 6-state vs. 4-state: The algorithm ran for 100 iterations. The 4-state model showed faster error decay and converged, while the 6-state model struggled. The 4-state model was selected as it included a direct closed-to-open transition, which better explained the rapid response to the optimized light inputs.
  • 3-state vs. 4-state: The experiment ran for 47 iterations before the cell was lost. The 4-state model consistently outperformed the 3-state model in predicting the photocurrent dynamics.
• Outcome: In both cases, the framework successfully identified the 4-state model as the most adequate predictor of the experimental data.

5. Significance and Future Work

• Practical Applicability: This is one of the first demonstrations of a closed-loop optimal control framework for model discrimination applied to a live biological experiment. It proves that complex model selection can be automated in real-time.
• Efficiency: By optimizing the information content of measurements, the method reduces the need for extensive, time-consuming data collection, which is crucial for fragile biological samples (e.g., patch-clamp recordings).
• Generalizability: While demonstrated on optogenetics, the framework is applicable to any system where inputs can be controlled and outputs measured (e.g., financial markets, opinion dynamics, other electrophysiological systems).
• Future Directions:
  • Extending the framework to discriminate more than two models simultaneously (currently uses a pairwise tournament).
  • Integrating Bayesian frameworks to handle parameter uncertainty and measurement noise more rigorously.
  • Combining with "Scientific Machine Learning" (e.g., SINDy) to discover model equations from scratch rather than just selecting from a pre-defined set.

In conclusion, the paper presents a robust, iterative methodology that transforms model selection from a passive statistical exercise into an active, control-theoretic process, successfully distinguishing between complex biological models in both simulation and live experiments.