Fitness functions for population pharmacokinetic modeling

Technical Summary: Fitness Functions for Population Pharmacokinetic Modeling

Problem Statement
Automated population pharmacokinetic (PopPK) model building requires predefined, objective criteria to evaluate and rank candidate models within a search space. Conventional manual modeling relies on expert judgment to balance goodness-of-fit, parameter precision, and physiological plausibility. However, algorithm-driven processes lack this human-in-the-loop flexibility and require a rigid "fitness function" to guide model selection. Existing approaches often rely solely on information criteria (e.g., AIC, BIC), which trade off fit against complexity but ignore parameter precision and plausibility. Alternatively, penalty-based methods have been used, but they frequently employ binary thresholds (pass/fail) that can bias selection toward conservative outcomes or lack systematic validation. There is a critical need for a fitness function that comprehensively evaluates multiple dimensions of model quality while balancing trade-offs, specifically validated against ground-truth data to ensure the recovery of the true data-generating structure.

Methodology
The study developed a systematic framework to design and evaluate fitness functions using 48 simulated datasets derived from optimal design principles. These datasets were generated from 24 intravenous and 24 oral nonlinear mixed-effect (NLME) models, varying in compartment number (1–3), inter-individual variability (IIV) configurations, and residual error structures.

An exhaustive search was performed across a predefined model space containing 711 candidate models for intravenous routes and 1,422 for oral routes. The search space included variations in compartment number, IIV specifications (on clearance, volumes, and absorption rates), covariance structures, elimination kinetics (first-order vs. Michaelis-Menten), and residual error models.

The study compared two penalty design strategies:

1. Binary Penalty: A fixed penalty is applied if a diagnostic metric crosses a single threshold (e.g., RSE > 20%).
2. Step Penalty: Penalties are graded based on the severity of the violation, distinguishing between "no penalty," "mild violation," and "severe violation" using multiple cutoffs.

Key diagnostic metrics evaluated included:

• Relative Standard Error (RSE) of fixed-effect parameters.
• Eta-shrinkage of IIV estimates.
• Omega values (IIV magnitude).
• Residual error parameters (additive and proportional).
• Correlation coefficients between IIV parameters.
• Covariance step success.

The performance of various fitness functions (combining BIC, AIC, or Objective Function Value with different penalty sets and magnitudes) was assessed by calculating the "retrieval rate"—the ability of the algorithm to correctly identify the original model structure and components across the 48 datasets.

Key Contributions

• Systematic Assessment: The study provides the first systematic evaluation of fitness function designs using exhaustive searches over a complete model space with ground-truth simulated data, rather than relying on limited clinical datasets.
• Identification of Key Metrics: The analysis identified Relative Standard Error (RSE), eta-shrinkage, and omega values as the three most influential metrics for accurately identifying the true model structure.
• Step Penalty Superiority: The study demonstrated that a step-penalty design is more robust than a binary penalty design. Specifically, the step penalty maintained high structural identification rates (avoiding the decline seen with binary penalties) while effectively filtering out unreliable IIV terms.
• Proposed Fitness Function: A specific, implementable fitness function was derived, combining the Bayesian Information Criterion (BIC) with step-based penalties for RSE, shrinkage, omega values, residual errors, and correlations.

Results

• Metric Impact: Under a BIC-based fitness function, RSE penalties significantly improved structural model identification in oral datasets (increasing retrieval from 88% to 92%), while shrinkage and omega penalties were crucial for correctly identifying IIV structures.
• Binary vs. Step: The binary penalty design showed a trade-off where increasing penalty strength improved IIV retrieval but degraded structural model identification (dropping from 92% to 81% in oral data). In contrast, the step penalty design maintained structural identification stability at approximately 90% even at high penalty scores, while achieving comparable IIV retrieval rates (82% vs. 81% for binary).
• Optimal Configuration: The final proposed fitness function uses BIC as the baseline with step penalties. Key cutoffs identified include:
  • RSE: Mild (20%), Severe (40%).
  • Shrinkage: Mild (30%), Severe (50%).
  • Omega (IIV): Severe cutoff at 10% (to penalize implausibly low variability).
  • Correlations: Mild (0.1/0.8), Severe (0.05/0.95).
• Performance: Using the proposed BIC-based step-penalty function, the full model retrieval rate (correct identification of structure, IIV, residual error, and correlation) reached 56% overall, outperforming AIC and OFV-based functions (46%). Specifically, it achieved 71% retrieval for intravenous data and 42% for oral data.
• Component Accuracy: The method correctly identified one-compartment models in 100% of cases and two-compartment models in 94% of cases. However, residual error model identification was less robust, particularly for additive models, which were frequently misclassified as combined models (38% success rate).

Significance and Claims
The authors claim that the proposed BIC-based fitness function with step penalties offers a robust, interpretable, and objective framework for automated PopPK model building. By integrating parameter precision and plausibility constraints directly into the optimization objective, the method effectively recovers true data-generating structures without requiring expert intervention. The study highlights that this approach addresses the limitations of purely information-criterion-based selection and overly rigid binary penalties.

The authors note that while the method demonstrated high accuracy in simulated, optimal-design datasets, its generalizability to complex real-world clinical data (with missing observations, measurement errors, and heterogeneous populations) requires further validation. Additionally, the current search space was limited to linear absorption processes, and future work should extend these evaluations to more complex pharmacokinetic contexts. The proposed framework is integrated into the nlmixr2auto R package, providing a practical tool for automated model selection.