Inference Functionals and Observation Operators for Distributional Statistical Models

Technical Summary: Inference Functionals and Observation Operators for Distributional Statistical Models

Problem Statement
Classical statistical inference relies heavily on the existence of probability densities and finite moments to define likelihood functions and estimating equations. However, many statistical models—particularly those involving heavy-tailed distributions (e.g., Cauchy, stable laws) or non-point observations (e.g., interval-censored, blurred, or transform-based data)—lack classical densities or finite moments. While Godambe's (1960) theory of inference functions generalizes maximum likelihood estimation by focusing on the structural properties of estimating equations (unbiasedness, sensitivity, variability) rather than likelihood maximization, it traditionally assumes point-valued observations and classical integrability.

The central problem addressed in this paper is the lack of a unified framework for statistical inference in distributional statistical models, where probability measures are represented by distribution–kernel pairs $(T_\theta, \phi) \in \mathcal{S}'(\mathbb{R}) \times \mathcal{S}(\mathbb{R})$. In such models, classical point evaluations are often physically unrealistic or mathematically ill-defined. The paper seeks to generalize the concept of inference functions to accommodate these distributional representations and to formalize the role of the observation mechanism itself as a mathematical operator.

Methodology
The paper constructs a general operator-theoretic framework for statistical inference, proceeding through three conceptual layers:

1. Distributional Representations and Weak Expectations:
   Building on companion work [12], the paper defines a probability measure via a tempered distribution $T_\theta$ and a fixed Schwartz kernel $\phi$. The classical expectation $E[\psi(X)]$ is replaced by the weak expectation $\langle T_\theta, \psi\phi \rangle$. This pairing is well-defined for all polynomials and exponentials, ensuring that "weak moments" and "weak characteristic functions" exist even when classical moments do not.

2. Observation Operators:
   The paper introduces observation operators $O: \mathcal{S}'(\mathbb{R}) \to \mathcal{Y}$, which map the latent distributional model to an observation space $\mathcal{Y}$. This generalizes the classical notion of observation (point evaluation) to include:

   • Kernel observations: Local averages weighted by a response profile.
   • Interval observations: Integration over a set (censored data).
   • Convolutional observations: Smoothing or blurring of the distribution.
   • Transform observations: Evaluation of the weak characteristic function at specific frequencies.

   The operator $O$ induces a pushforward model $\{P^O_\theta\}$ on the observation space. The paper distinguishes between deterministic linear operators (Level 1), deterministic nonlinear operators (Level 2), and stochastic operators (Level 3), focusing the asymptotic theory primarily on the first two.

3. Inference Functionals:
   An inference functional $\Psi: \mathcal{Y} \times \Theta \to \mathbb{R}^p$ is defined as a measurable function acting on the observed data $Y = O(T_X)$. The estimator $\hat{\theta}_n$ is the root of the empirical estimating equation $\frac{1}{n}\sum \Psi(Y_i, \theta) = 0$. The paper categorizes these into three layers:

   • Layer I: Operator-valued equations (e.g., interval-censored data).
   • Layer II: Transform-based equations using point data but distributional identities (e.g., sinusoidal inference for heavy tails).
   • Layer III: Kernel-weighted classical equations.

Key Contributions and Results

• Asymptotic Theory: The paper establishes consistency and asymptotic normality for estimators derived from regular inference functionals. Under mild conditions (Glivenko–Cantelli property, identification, and differentiability), the estimator satisfies:
  $$\sqrt{n}(\hat{\theta}_n - \theta_0) \xrightarrow{D} N(0, S(\theta_0)^{-1} V(\theta_0) S(\theta_0)^{-\top})$$
  where $S$ is the sensitivity matrix and $V$ is the variability matrix. The Godambe information is defined as $G_\Psi = S^\top V^{-1} S$.

• The Information Hierarchy and Convolution Theorem:
  A central theoretical result is the derivation of a strict hierarchy of information bounds via the Hájek–Le Cam convolution theorem. For a regular estimator based on a pushforward model with Fisher information $I_O(\theta)$, the asymptotic distribution is a convolution of a Gaussian component (variance $I_O^{-1}$) and a noise distribution $M$:
  $$\sqrt{n}(\hat{\theta}_n - \theta_0) \xrightarrow{D} N(0, I_O(\theta)^{-1}) * M$$
  This leads to the inequality:
  $$I_{\text{classical}}(\theta) \geq I_O(\theta) \geq G_\Psi(\theta)$$

  • $I_{\text{classical}} - I_O$ quantifies information lost due to the observation mechanism (e.g., censoring or blurring).
  • $I_O - G_\Psi$ quantifies information lost due to the choice of inference functional.

  The paper argues that maximizing Godambe information is not merely an arbitrary optimality criterion but a structural necessity: it is equivalent to minimizing the non-Gaussian noise $M$ in the limit distribution.

• Nuisance Parameters and Orthogonality:
  The framework is extended to nuisance parameters using a two-stage construction of the nuisance tangent space (distributional derivatives followed by Hilbert-space projection). The paper demonstrates that certain inference functionals (e.g., sinusoidal functions in symmetric location-scale models) achieve automatic orthogonality to the nuisance tangent space, eliminating the need for explicit Bhapkar–Godambe projections.

• Concrete Examples:
  The theory is applied to:

  • Heavy-tailed models: Sinusoidal inference functions for Student $t$ and Cauchy distributions, which require no density evaluation and remain stable where the sample mean fails.
  • Interval-censored data: Formulating location inference directly from interval indicators.
  • Multivariate elliptically contoured distributions: Deriving Godambe information for linear functionals of the location parameter.
  • Simulation: A study comparing the sinusoidal estimator to the mean and median in Student $t$ models, showing the sinusoidal estimator's stability under heavy tails.

Significance and Claims
The paper claims to place estimating-function methodology on a foundation that does not require classical densities or finite moments. Its primary significance lies in:

1. Conceptual Unification: It unifies diverse estimation strategies (point observations, interval censoring, transform methods) under a single operator-theoretic framework.
2. Structural Justification: It provides a rigorous justification for the Godambe optimality criterion via the convolution theorem, resolving long-standing tensions regarding the asymptotic validity of inference functions.
3. Handling Ill-Posed Models: It offers a mathematically rigorous approach to inference for models where classical likelihoods are undefined or where observations are inherently non-pointwise (distributional).

The authors position this work as part of a broader series: companion papers [12] and [13] establish the probabilistic foundations and weak moment estimation, respectively, while this paper provides the general operator-theoretic framework and asymptotic theory. The paper acknowledges that the development of weak empirical process theory and a fully rigorous semiparametric theory for distributional nuisance parameters remains future work.