Technical Summary: Shapley Regression for Rare Disease Diagnosis Support (APDS Case Study)

1. Problem Statement

The diagnosis of rare primary immunodeficiencies, specifically Activated PI3Kδ Syndrome (APDS), faces a "small data, high complexity" dilemma. APDS is characterized by low prevalence (estimated 1–2 per million), highly heterogeneous symptoms, and significant overlap with common immune disorders, leading to frequent underdiagnosis and delayed recognition.

From a machine learning perspective, existing approaches struggle to balance predictive power with clinical interpretability:

• Traditional Linear Models (e.g., Logistic Regression): Offer stability on small datasets but assume additive feature contributions, failing to capture synergistic or redundant interactions between symptoms.
• High-Capacity Non-Linear Models (e.g., Neural Networks, XGBoost): Can capture complex interactions but are prone to overfitting on small cohorts and lack the transparency required for clinical adoption.

The challenge is to develop a model that captures non-linear symptom interactions (crucial for biological systems) while maintaining the interpretability and stability necessary for rare disease diagnosis.

2. Methodology: Shapley Regression (SR)

The authors propose Shapley Regression, a game-theoretic extension of logistic regression that replaces the standard affine predictor with a $k$-additive cooperative game.

Core Formulation

The probability of the positive class (APDS) is modeled as:
$$P(y = 1 | \mathbf{x}) = \sigma\left(\beta + \sum_{A \subseteq F, A \neq \emptyset} I(A) \cdot \phi_A(\mathbf{x})\right)$$
Where:

• $\mathbf{x}$ is the normalized feature vector.
• $\sigma$ is the sigmoid function.
• $I(A)$ are learnable coefficients representing Shapley Interaction Indices for feature subsets $A$.
• $\phi_A(\mathbf{x})$ are basis functions constructed to allow direct game-theoretic interpretation (based on the Choquet integral).

Key Architectural Choices

1. 2-Additive Complexity: To prevent the "curse of dimensionality" inherent in small medical datasets, the model is constrained to $k=2$. This limits learning to:
   • Main Effects: Independent contributions of single features ($|A|=1$).
   • Pairwise Interactions: Synergistic ($I > 0$) or redundant ($I < 0$) interactions between feature pairs ($|A|=2$).
   • Higher-order interactions are forced to zero, reducing the parameter space from exponential to quadratic ($O(n^2)$).
2. Regularization: The model employs $\ell_2$ regularization (Ridge) on the Shapley indices. Theoretical analysis (using Structural Risk Minimization and Algorithmic Stability) demonstrates that $\ell_2$ regularization provides a bound on the generalization gap and ensures the stability of the interaction explanations, unlike unregularized models which suffer from high variance in high-dimensional settings.
3. Training: The model is trained by minimizing the regularized negative log-likelihood on an expanded feature space containing the basis functions for singletons and pairs.

3. Key Contributions

1. Novel Model Architecture: Introduction of Shapley Regression, which integrates cooperative game theory into logistic regression to explicitly model feature interactions while preserving the convexity and transparency of linear models.
2. Theoretical Guarantees: The paper provides theoretical bounds on generalization and, crucially, on the stability of explanations. It proves that $\ell_2$ regularization imposes a hard ceiling on the variance of Shapley interaction indices, ensuring that the model's interpretability is robust to noise.
3. Empirical Validation:
   • Benchmarking: Validated on eight public biomedical datasets (including Diabetes and COVID-19), confirming that a 2-additive model with $\ell_2$ regularization achieves an optimal trade-off between predictive power and noise robustness.
   • APDS Case Study: Applied to a real-world cohort of 222 patients (29 APDS cases, 193 controls) from the Necker Hospital data warehouse.
4. Clinical Interpretability: The method outputs direct Shapley Interaction Indices, allowing clinicians to distinguish between independent symptoms, synergistic pairs, and redundant features without post-hoc approximation.

4. Results

Predictive Performance

In a 5-nested cross-validation on the APDS cohort, the 2-additive Shapley Regression model achieved:

• Balanced Accuracy: 0.941 (outperforming Linear LR at 0.907 and XGBoost at 0.884).
• Sensitivity (Recall): 0.933, the highest among all models, indicating superior ability to detect rare APDS cases.
• Specificity: 0.948, maintaining high accuracy in identifying controls.
• Efficiency: The model is computationally lightweight, running efficiently on standard CPU hardware, making it feasible for clinical deployment without specialized GPUs.

Interaction Analysis

The model successfully mapped the landscape of symptom interactions:

• Validation of Known Phenotypes: The model identified core APDS manifestations (e.g., immunodeficiency, splenomegaly, lymphopenia) as strong positive predictors.
• Interaction Insights:
  • Redundancy: Core immunological features showed negative interactions (redundancy) with other phenotypes, indicating they are highly informative individually and add little incremental value when combined.
  • Contextual Importance: Symptoms like pain and anesthesia showed widespread positive interactions with other features, suggesting their diagnostic value is context-dependent rather than standalone.
  • Counter-Intuitive Findings: Some phenotypes (e.g., pleural effusion, allergy) appeared negatively associated with APDS in isolation but showed positive interactions with non-traditional features, highlighting the model's ability to disentangle complex co-occurrence patterns.
• Expert Review: A clinical expert validated the top 30 interacting phenotypes, confirming that 17 were core clinical manifestations of APDS.

5. Significance and Claims

The paper claims that Shapley Regression offers a pragmatic solution to the "small data, high complexity" problem in rare disease diagnosis. Its significance lies in three areas:

1. Bridging the Gap: It successfully bridges the gap between the expressiveness of non-linear models and the interpretability of linear models, a critical requirement for clinical trust.
2. Stability of Explanations: Unlike post-hoc explanation methods (e.g., LIME, SHAP applied to black boxes), SR's parameters are the explanations. The authors claim theoretical proof that $\ell_2$ regularization guarantees these explanations are stable and noise-invariant.
3. Clinical Utility: The framework not only aids in detection (high sensitivity) but facilitates the exploration of pairwise symptom interactions, providing a complementary perspective on disease mechanisms that traditional additive models miss.

The authors remain modest regarding the current cohort size (222 patients), noting that while the results are promising, larger cohorts are needed to robustly validate the identified interactions and improve predictive confidence. They position SR as a useful exploratory tool for understanding complex phenotypic interactions in rare diseases.