Induced replication and the assessment of models

1. Problem Statement

The paper addresses the challenge of assessing the adequacy of semiparametric and highly-parametrised models (e.g., models with infinite-dimensional nuisance components like baseline hazards or unknown smooth functions).

• Current Limitations: Traditional assessment methods typically involve:
  1. Residual analysis: Fitting the model, estimating nuisance parameters (often via kernel smoothing or basis expansion), and checking residuals. This requires tuning parameter selection, which blurs the line between estimation and assessment.
  2. Out-of-sample prediction: Splitting data to assess predictive performance, which reduces effective sample size and introduces issues with post-selection inference.
• The Core Issue: In semiparametric settings, the effective parameter space is often constrained by smoothness assumptions to be smaller than the sample size $n$, but not necessarily finite. The authors argue that standard approaches fail to separate the estimation of nuisance components from the assessment of the model structure itself, often relying on asymptotic approximations that may not hold in finite samples or under complex misspecification.

2. Methodology: Induced Internal Replication

The authors propose a framework based on Fisherian sufficiency and co-sufficiency to assess models without estimating high-dimensional nuisance parameters. The core concept is Induced Internal Replication.

• The Principle: The method seeks to construct a set of statistics $U_1(\psi_0), \dots, U_m(\psi_0)$ for a candidate interest parameter $\psi_0$ such that:
  • If the postulated model is correct and $\psi_0 = \psi^*$ (the true value), the $U_j$ are independent and identically distributed (i.i.d.) standard uniform random variables.
  • If the model is misspecified or $\psi_0 \neq \psi^*$, the $U_j$ deviate from standard uniformity.
• Mechanism: Instead of splitting the sample, the method uses the structure of the model to induce "internal" replication. This is achieved by:
  1. Conditioning on Sufficient Statistics: Eliminating nuisance parameters by conditioning on statistics sufficient for them (co-sufficiency).
  2. Transformation: Transforming data into variables whose distribution is free of nuisance parameters (e.g., ratios of outcomes).
  3. Randomization/Synthesis: Using randomization schemes (e.g., in high-dimensional regression) to generate synthetic replicates that follow a known distribution under the null model.
• Statistical Test:
  • The authors use Fisher's method to combine p-values derived from the $U_j$ statistics.
  • The test statistic is $R = -2 \sum_{j=1}^m \log U_j(\hat{\psi})$, where $\hat{\psi}$ is a consistent estimator of the interest parameter.
  • Under the null hypothesis (correct model), $R$ follows a $\chi^2_{2m}$ distribution.
  • Proposition 2.1 formally justifies replacing the true parameter $\psi^*$ with a consistent estimator $\hat{\psi}$, showing that the asymptotic uniformity of $U_j(\hat{\psi})$ is preserved.

3. Key Contributions and Examples

The paper demonstrates the versatility of this framework through four distinct inferential structures and multiple examples:

A. Inferential Structures

The authors categorize methods for inducing internal replication into four structures:

1. Structure 1 (Co-sufficiency): Conditioning on a statistic $S$ sufficient for nuisance parameters $\gamma$, where $S$ does not depend on $\psi$.
2. Structure 2 (Interest-dependent Co-sufficiency): Conditioning on a statistic $S(\psi)$ that depends on the interest parameter (e.g., Example 1.1).
3. Structure 3 (Transformation): Transforming data $Z = t(Y)$ such that the distribution of $Z$ is free of nuisance parameters.
4. Structure 4 (Interest-dependent Transformation): Transformation $Z(\psi)$ that depends on $\psi$.

B. Specific Applications

1. Matched-Pair Examples (Exponential/Weibull):
   • In a matched-pair design with nuisance rates $\gamma_j$, the ratio $Z_j = Y_{j1}/Y_{j0}$ eliminates $\gamma_j$. Under a proportional rates model, $Z_j$ follows a known distribution free of $\gamma_j$.
   • The paper shows that even if the true distribution is Weibull (misspecified), the test has power to detect the deviation, provided the shape parameter differs from 1.
2. Time-Dependent Poisson Processes:
   • For a process with intensity $\lambda_i(t) = e^{\gamma_i + \beta t}$, the nuisance parameters $\gamma_i$ are eliminated by conditioning on the total number of events $m_i$.
   • The conditional distribution of the sum of event times is derived (via Laplace transforms), allowing for model assessment without estimating the individual $\gamma_i$.
3. Proportional Hazards Model:
   • The partial likelihood for Cox's model is reinterpreted through conditional co-sufficiency. The indices of failure times, conditioned on the history of failure times, follow a distribution free of the baseline hazard $h_0$.
   • The authors propose splitting the sample into blocks to generate independent replicates of the partial likelihood scores, enabling model assessment without estimating $h_0$.
4. Sparse Regression Confidence Sets:
   • Extending work by Battey, Rasines & Tang (2025), the paper constructs confidence sets for sparse regression models.
   • By projecting data onto the orthogonal complement of the model space, the resulting statistics are uniformly distributed on a hypersphere under the correct model.
   • Synthetic replicates are generated to create multiple independent statistics, allowing for the assessment of model adequacy without sample splitting.

4. Results

Numerical simulations confirm the theoretical properties of the proposed methods:

• Nominal Error Rates: Under the postulated model, the methods correctly recover nominal error rates (e.g., 5% rejection rate for a 5% test), even when using estimated parameters ($\hat{\psi}$) instead of true values.
• High Sensitivity to Misspecification: The methods demonstrate high power to detect departures from the model in the direction of semiparametric alternatives (e.g., Weibull vs. Exponential, Power-law vs. Linear trend in Poisson processes).
• Robustness: The approach avoids the "tuning parameter" dilemma. Unlike cross-validation, which requires selecting smoothing parameters that affect both estimation and assessment, this method relies on the structural properties of the model.
• Local Power: In cases where the true model is a local perturbation of the postulated model (e.g., Weibull shape parameter close to 1), the test remains sensitive, particularly when using the estimator-based statistic $R$ (Eq. 8) rather than testing over a grid of $\psi_0$.

5. Significance and Conclusion

• Theoretical Unification: The paper unifies diverse model assessment techniques (residual analysis, partial likelihood, randomization tests) under a single framework of induced internal replication. It reframes model assessment as a problem of checking for "within-sample" exchangeability rather than out-of-sample prediction.
• Avoiding Nuisance Estimation: A major contribution is the demonstration that one can assess complex semiparametric models without estimating the infinite-dimensional nuisance components. This bypasses the difficulties of bandwidth selection, basis expansion, and regularization tuning.
• Post-Selection/Post-Reduction Inference: The framework provides a rigorous foundation for inference after model selection or dimension reduction, ensuring that the assessment of the final model is not contaminated by the selection process.
• Open Challenges: The authors conclude by noting that while specific structures allow for exact internal replication, a general theory for approximate internal replication (approximate exchangeability) in settings where exact elimination is impossible remains an open and critical area for future research.

In summary, Battey and Reid offer a foundational shift in how semiparametric models are validated, moving away from residual diagnostics and cross-validation toward a structural approach based on sufficiency and induced replication, thereby separating the estimation of interest parameters from the assessment of model adequacy.