Differentially Private One-Shot Federated Inference for Linear Mixed Models via Lossless Likelihood Reconstruction

1. Problem Statement

The paper addresses a critical tension in One-Shot Federated Learning (FL) for biomedical research: the trade-off between statistical efficiency and privacy.

• The Context: In multi-site clinical studies (e.g., across hospitals), sharing individual-level patient data (IPD) is often prohibited due to privacy regulations. Federated learning allows sites to share only summary statistics.
• The Specific Challenge: While sharing summary statistics (like the Gram matrix $X^\top X$) enables exact likelihood reconstruction for Linear Mixed Models (LMMs), these summaries are vulnerable to reconstruction attacks.
  • When a site has a small sample size and discrete covariates (e.g., binary variables like Sex or Test Result), the Gram matrix can uniquely determine the original individual-level data (up to row permutation) by solving a 0–1 integer programming problem.
  • Existing one-shot methods for LMMs (e.g., Luo et al., 2022) achieve lossless reconstruction but offer no formal privacy guarantees.
  • Existing privacy-preserving FL methods often rely on iterative communication (burdensome in clinical settings) or require large within-site sample sizes to ensure validity, which is unrealistic for many multi-center studies where some sites contribute very few patients.

Goal: Develop a one-shot (single communication round), differentially private (DP) inference framework for LMMs that:

1. Prevents the reconstruction of individual-level data from shared summaries.
2. Maintains exact likelihood reconstruction capabilities.
3. Provides valid statistical inference under a multi-site asymptotic regime (where the number of sites $K \to \infty$, but within-site sample sizes $n_k$ may remain small/bounded).

2. Methodology

The proposed framework consists of three main components:

A. Lossless Likelihood Reconstruction

The authors build upon the work of Luo et al. (2022). For a Linear Mixed Model with a random-intercept working covariance:
$$y = X\beta + Zb + \epsilon$$
Each site $k$ computes two sufficient statistics matrices based on local data $(y_k, X_k)$:

1. $S_k$: Standard quadratic forms ($y_k^\top y_k, y_k^\top X_k, X_k^\top X_k$).
2. $T_k$: Quadratic forms involving the site-membership vector ($y_k^\top \mathbf{1}\mathbf{1}^\top y_k$, etc.).

These matrices allow the exact reconstruction of the pooled log-likelihood function $l_{ML}$ and the Restricted Maximum Likelihood (REML) function without ever accessing IPD.

B. Differentially Private Perturbation (Gaussian Mechanism)

To prevent reconstruction attacks, the authors apply a Gaussian mechanism to the site-level summaries before sharing them.

• Adjacency: Two datasets are adjacent if they differ by exactly one individual record.
• Sensitivity: The global sensitivity $\Delta_F$ is measured using the Frobenius norm. For binary covariates, $\Delta_F \leq 2p$ (where $p$ is the number of covariates).
• Perturbation: Each site releases noisy summaries:
  $$\tilde{S}_k = S_k + U_{k1}, \quad \tilde{T}_k = T_k + U_{k2}$$
  where $U$ contains i.i.d. Gaussian noise calibrated to achieve $(\epsilon, \delta)$-differential privacy.
• Symmetrization: Post-processing is applied to ensure the noisy matrices remain symmetric without violating DP guarantees.

C. Estimation and Inference

• Estimator: The DP-perturbed Maximum Likelihood Estimator (MLE) $\hat{\theta}_{DP}$ is obtained by maximizing the perturbed log-likelihood $l_{DP}$ constructed from $\{\tilde{S}_k, \tilde{T}_k\}$.
• Variance Estimation: A Cluster-Robust Variance Estimator (CR0) is derived that can be computed solely from the perturbed summaries. This is crucial because it provides valid uncertainty quantification even if the working covariance structure (random intercept) is misspecified.
• Small-Sample Correction: The authors discuss applying scalar corrections (CR1, CR1p) to the variance estimator to mitigate downward bias when the number of sites $K$ is small.

3. Key Contributions

1. Formal Privacy for One-Shot LMMs: The study is the first to integrate formal $(\epsilon, \delta)$-differential privacy into a lossless one-shot likelihood reconstruction framework for LMMs. It bridges the gap between exact statistical reconstruction and privacy protection.
2. Theoretical Guarantees under Multi-Site Asymptotics:
   • Proved consistency and asymptotic normality of the DP estimator as the number of sites $K \to \infty$, even when within-site sample sizes are bounded.
   • Derived the leading-order statistical cost of privacy, showing that the estimation error scales as $O(\sigma_{DP}^2 / K)$. This implies that increasing the number of sites mitigates the noise introduced by privacy.
3. Cluster-Robust Inference: Developed a method to compute cluster-robust (sandwich) variance estimators directly from privatized summaries, ensuring valid inference under model misspecification.
4. Dimension-Adjusted Privacy Calibration: Proposed a strategy to adjust privacy parameters based on the dimensionality of the covariates to balance reconstruction risk and estimation accuracy.

4. Results

Simulation Studies

• Reconstruction Risk: In scenarios with small sample sizes ($n < 20$) and binary covariates, sharing unperturbed Gram matrices allowed for near-perfect reconstruction of individual data. Introducing moderate Gaussian noise (e.g., $\epsilon_0 \leq 8$) reduced the reconstruction rate to near zero.
• Estimation Accuracy:
  • As the privacy budget $\epsilon$ increases (less noise), the DP estimator converges to the IPD-based estimator.
  • As the number of sites $K$ increases, the impact of privacy noise diminishes ($O(K^{-1})$).
  • Trade-off: Strong privacy settings (low $\epsilon$) with a small number of sites ($K=20$) led to unstable standard errors (SE inflation). However, with moderate $K$ (e.g., $K=100+$) and moderate privacy, the method maintained competitive accuracy.
• Robustness: The cluster-robust variance estimator remained valid even when the random-intercept model was misspecified (e.g., data generated with random slopes).

Real Data Application

• Dataset: Multi-site COVID-19 testing data from 88 clinics (Children's Hospital of Pennsylvania).
• Findings: Even under a stringent privacy setting ($\epsilon_0 = 4$), the L2 privacy cost (deviation from the true pooled estimate) remained below 0.05, and the SE inflation factor did not exceed 1.5.
• Conclusion: Meaningful privacy protection can be achieved with only a modest loss of statistical efficiency in realistic multi-site clinical settings.

5. Significance and Impact

• Enabling Small-Site Collaboration: This method enables valid statistical inference in multi-center studies where many sites contribute very few patients (a common scenario in rare disease research or early-phase trials), a setting where traditional asymptotic methods (relying on large $n_k$) fail.
• Operational Efficiency: By being "one-shot," it avoids the operational burden of iterative communication, making it feasible for clinical environments with limited bandwidth or administrative overhead.
• Privacy-Utility Balance: It provides a principled framework for researchers to quantify the "cost of privacy." The results suggest that with a sufficient number of participating sites, the statistical penalty for strong privacy guarantees is negligible, while the protection against reconstruction attacks is substantial.
• Future Directions: The authors note that future work could focus on improved privacy accounting, extending the framework to more complex covariance structures (beyond random intercepts), and developing principled guidelines for selecting $(\epsilon, \delta)$ based on specific study utility targets.

In summary, this paper provides a statistically rigorous and practically viable solution for conducting privacy-preserving, multi-site clinical research using Linear Mixed Models, effectively solving the "small sample, discrete covariate" reconstruction vulnerability inherent in current federated learning approaches.