📊 statistics

Goodness-of-Fit Tests for Censored and Truncated Data: Maximum Mean Discrepancy Over Regular Functionals

This paper proposes a new omnibus goodness-of-fit testing framework for parametric models with censored or truncated data by using a Neyman-orthogonal score process aggregated over a reproducing kernel Hilbert space, providing a computationally stable and asymptotically valid method for complex incomplete-data designs.

Original authors: Juan Carlos Escanciano, Jacobo de Uña-Álvarez

Published 2026-02-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Juan Carlos Escanciano, Jacobo de Uña-Álvarez

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out the "true nature" of a mysterious crowd of people. You want to know if everyone in this city follows a specific pattern—for example, if their heights follow a perfect "Bell Curve."

However, there is a massive problem: you can’t see everyone.

The Problem: The "Incomplete Picture"

In the real world, data is often "broken" or "hidden" in two main ways:

Censoring (The "Early Exit" Problem): Imagine you are tracking how long a lightbulb lasts. You turn them on today, but the study ends next week. Some bulbs might last for years, but you only know they lasted at least a week. You don't know their true "death date."
Truncation (The "VIP Club" Problem): Imagine you are studying the wealth of people in a city, but you can only interview people who walk into a specific luxury hotel. You aren't seeing the poor, and you aren't seeing the ultra-rich who stay in private mansions. Your "window" of observation is restricted.

Because of these gaps, if you try to use standard math to check if the data fits your "Bell Curve" theory, the math breaks. It’s like trying to solve a jigsaw puzzle when half the pieces are missing and the other half are from a different box.

The Solution: The "Smart Filter" Approach

The authors of this paper, Escanciano and de Uña-Álvarez, have invented a new way to test if a theory is correct, even when the data is messy and incomplete. Here is how their method works, using three metaphors:

1. The "Neyman-Orthogonal" Score (The Noise-Canceling Headphones)

When you have missing data, your estimates for the "missing parts" are often slightly wrong. In traditional math, these small errors "leak" into your main results and ruin your conclusion.

The authors use something called Neyman-orthogonality. Think of this like noise-canceling headphones. Even if there is a lot of "background noise" (the errors caused by the missing data), these mathematical "headphones" filter that noise out, allowing you to hear the "true signal" (the actual pattern of the data) clearly.

2. The RKHS/MMD Aggregation (The "Master Impressionist")

Usually, when scientists test a theory, they look for one specific type of error (e.g., "Is the average too high?"). But what if the error is weird? What if the data is too "wiggly" or has strange bumps?

The authors use a method called Maximum Mean Discrepancy (MMD). Instead of looking for one specific mistake, they use a "mathematical net" (called an RKHS) that can catch any kind of shape or wiggle.

Imagine you are comparing a real mountain range to a drawing of a mountain range. A standard test might only check if the height is correct. The authors' method is like an expert artist who looks at the shadows, the slopes, the ridges, and the valleys all at once. If anything looks "off," the test catches it.

3. The Multiplier Bootstrap (The "Replay Button")

To know if their result is a "real discovery" or just a "lucky guess," they use a technique called the Multiplier Bootstrap.

Think of this as a high-tech replay button. They take their data, shake it up slightly in thousands of different ways, and see how often a "lucky guess" would happen by accident. If their original result is much stronger than all those "shaken up" versions, they can confidently say, "Yes, the theory is wrong!"

Why does this matter?

The paper proves this works for very difficult scenarios, specifically "Double Truncation" (where you are stuck in a tiny window of observation, like looking at a city through a narrow mail slot).

In short: They have created a "super-lens" for scientists. It allows them to look at incomplete, biased, and "broken" data and say with mathematical certainty: "Yes, this pattern is real," or "No, your theory doesn't fit this reality."

Technical Summary: Goodness-of-Fit Tests for Censored and Truncated Data

Authors: Juan Carlos Escanciano and Jacobo de Uña-Álvarez (2026)

1. The Problem: Challenges in Incomplete-Data Inference

In survival analysis, econometrics, and epidemiology, the variable of interest ( $X$ ) is often only partially observed due to censoring (the event is not seen within the study period) or truncation (the subject is only included if the event occurs within a specific window).

The authors identify three critical failures in classical Goodness-of-Fit (GOF) testing for these settings:

NPMLE Instability: Traditional tests compare a parametric model to a Nonparametric Maximum Likelihood Estimator (NPMLE). However, in complex designs (like double truncation), the NPMLE may not exist, may be non-unique, or may converge too slowly to allow for reliable statistical calibration.
Nuisance Contamination: In incomplete data, the observed distribution depends on unknown "nuisance" features of the observation mechanism (the laws of censoring/truncation times). Naive tests often fail to be insensitive to these nuisances.
Computational Burden: Bootstrapping NPMLE-based statistics requires re-estimating the nonparametric model in every resample, which is computationally expensive and theoretically delicate under composite null hypotheses.

2. Methodology: The Orthogonal RKHS-MMD Approach

The authors propose a systematic, "NPMLE-free" framework that shifts the focus from estimating the distribution function to characterizing the null hypothesis through regular functionals.

A. Neyman-Orthogonal Score Processes
Instead of comparing distributions, the authors define the null hypothesis $H_0$ via a family of moment restrictions: $E[g_\phi(Z; \theta_0, G)] = 0$ . To handle composite nulls (where $\theta_0$ is unknown) and nuisance parameters ( $G$ ), they construct a Neyman-orthogonal score process $\Pi_\perp \phi(Z)$ . This process is designed to be locally insensitive to errors in the estimation of $\theta$ and $G$ , meaning the first-order influence of the nuisance parameters on the score is zero.

B. RKHS Aggregation (Maximum Mean Discrepancy)
To create an omnibus test (one that can detect many different types of deviations from the null), they aggregate these scores over a Reproducing Kernel Hilbert Space (RKHS) unit ball. This results in a statistic $\hat{Q}_K$ that resembles the Maximum Mean Discrepancy (MMD).

C. Quadratic-Form Representation and Bootstrap
A major technical breakthrough is the derivation of a simple quadratic-form $V$ -statistic representation:
$\hat{Q}_K = \frac{1}{n^2} \sum_{i=1}^n \sum_{j=1}^n \hat{K}_\perp(Z_i, Z_j)$
This allows for efficient computation. Furthermore, because of the Neyman-orthogonality, they can use a multiplier bootstrap that keeps the nuisance estimates fixed, avoiding the need to re-estimate the model in every bootstrap iteration.

3. Key Contributions

A General Template: A unified methodology for GOF testing under various incomplete-data schemes using pathwise differentiable functionals.
First Omnibus Test for Double Truncation: To the authors' knowledge, this is the first omnibus GOF test for a parametric family under random double truncation in the composite-hypothesis case.
Theoretical Rigor: They establish the asymptotic validity of the test under both the null hypothesis and local (Pitman-type) alternatives.
Computational Efficiency: By providing an induced kernel $\hat{K}_\perp$ , they transform a complex supremum problem into a scalable $V$ -statistic.

4. Results and Empirical Evidence

Asymptotic Theory: The authors prove that $n\hat{Q}_K$ converges to the squared norm of a Gaussian process, ensuring the test's size (Type I error) is well-controlled. They also prove the test is consistent against fixed alternatives.
Simulation (Double Truncation): Simulations demonstrate that the test maintains correct size control across varying levels of "truncation severity" (weak vs. strong truncation) and shows increasing power as sample size increases.
Empirical Illustration (Quasar Data): Applying the test to astronomical data (log luminosity of quasars), which is subject to double truncation, the test successfully identified a lack of fit to the exponential model, providing a $p$ -value ($0.0661$) that suggests the model is borderline, consistent with previous qualitative observations.

5. Significance

This paper provides a robust mathematical bridge between semiparametric theory (Neyman orthogonality) and kernel methods (MMD). By bypassing the NPMLE, the authors offer a tool that is theoretically more stable and computationally more practical for researchers working with complex survival and truncation data, where traditional methods often break down.