Unified formulas for conditional quantities and transportation functionals

Technical Summary: Unified Formulas for Conditional Quantities and Transportation Functionals

Problem Statement
The paper addresses the fragmentation in probability theory where conditional expectations, conditional distributions, hazard functions, and transportation functionals (such as Wasserstein distances) are often treated through distinct analytical frameworks depending on whether the underlying random variables are absolutely continuous, discrete, or mixed. Furthermore, there is a need for a unified approach to derive sharp bounds for transportation-type functionals and dispersion measures (like Gini indices) under fixed marginal distributions without relying on specific dependence structures. The authors aim to establish a common probabilistic structure that links these concepts through a unified framework.

Methodology
The authors develop a framework based on distributional derivatives and Dirac delta representations.

1. Distributional Tools: The paper utilizes weak derivatives and the Dirac delta distribution ($\delta_t$) to handle conditioning on events of probability zero (e.g., $Y=t$) in a unified manner. This approach treats the conditioning operation as a localization mechanism.
2. Copula Theory and Extremal Bounds: To analyze transportation functionals, the authors combine the distributional framework with copula theory. They utilize the Fréchet–Hoeffding bounds ($W_2$ and $M_2$) and the concept of $\Delta$-antitonic functions. A function $f$ is $\Delta$-antitonic if its mixed finite differences are non-negative. The authors leverage the monotonicity of expectations over the class of copulas to derive sharp bounds for moments of the form $E|X - Y|^r$.
3. Quantile and Survival Representations: By applying the sifting property of the Dirac delta to the Fréchet–Hoeffding bounds, the authors derive explicit integral representations involving quantile functions ($F^{-1}$) and survival functions ($\bar{F}$).

Key Contributions and Results

• Unified Formulas for Statistical Quantities:

  • The paper establishes general identities for conditional expectations, conditional laws, hazard functions, and improper distributions that are valid for absolutely continuous, discrete, and mixed random variables.
  • Substitution Principle: A key result is the interpretation of conditioning on $Y=t$ as a substitution operation where the randomness of $Y$ is replaced by the deterministic value $t$. This is formalized via limits of conditional probabilities over shrinking intervals, yielding unified expressions for $E[g(X,Y)|Y=t]$ and the conditional law $P((X,Y) \in B | Y=t)$.
  • Hazard Functions: The hazard function is expressed as a localized rate of occurrence using Dirac deltas, providing a common expression for both continuous and discrete lifetime models.
  • Improper Distributions: The paper reframes improper (defective) distributions as proper distributions on an extended state space including infinity, offering a unified stochastic representation for mixture models with a "cure fraction."

• Transportation Functionals and Moment Bounds:

  • Sharp Bounds: For a $\Delta$-antitonic function $f$, the expectation $E[f(X,Y)]$ is bounded by expectations under the lower ($W_2$) and upper ($M_2$) Fréchet–Hoeffding bounds. Specifically, for $f(x,y) = -|x-y|^r$ (where $r \ge 1$), the authors derive sharp bounds for the absolute difference moments $E|X-Y|^r$.
  • Quantile Representations:
    • The Wasserstein distance of order $r$ ($W_r$) is shown to equal the $L^r$ distance between quantile functions: $W_r^r(F_X, F_Y) = \int_0^1 |F_X^{-1}(u) - F_Y^{-1}(u)|^r du$.
    • A corresponding upper transportation functional ($\bar{W}_r$) is derived, representing the maximum possible $E|X-Y|^r$ under fixed marginals: $\bar{W}_r^r(F_X, F_Y) = \int_0^1 |F_X^{-1}(u) - F_Y^{-1}(1-u)|^r du$.
  • Survival Function Bounds: The paper provides equivalent bounds expressed solely in terms of marginal survival functions, offering alternative perspectives on dispersion.
  • Dependence Coefficient: A normalized dependence coefficient $\rho_r$ is introduced to measure the relative position of $E|X-Y|^r$ between its minimum and maximum attainable values.

• Applications to Specific Distributions:

  • The derived quantile representations are applied to the normal approximation of standardized counting distributions. Explicit formulas are provided for Poisson, Binomial, and Negative Binomial models.
  • For the quadratic case ($r=2$), the Wasserstein distance and upper transportation functional are shown to be determined entirely by a quantile correlation coefficient between the standardized sum and the standard normal distribution.

• Gini Measures:

  • As a particular case ($r=1$), the results yield explicit bounds and survival-function representations for the bivariate Gini mean difference and the associated bivariate Gini index, linking these inequality measures directly to optimal transport costs.

Significance and Claims
The paper claims to establish explicit links among conditional structures, dependence modeling, dispersion measures, normal approximation, and optimal transport within a single unified probabilistic framework.

• Unification: The primary significance lies in revealing a "common localization mechanism" underlying classical concepts like conditional expectations and hazard functions, valid across different distribution types (continuous, discrete, mixed).
• Conciseness: The approach yields concise derivations of classical results (such as the quantile representation of the Wasserstein distance) and new representations for transportation functionals without requiring complex transportation problem solutions.
• Interpretability: By connecting Gini-type dispersion measures to extremal transportation costs, the paper strengthens the theoretical connection between inequality measures and optimal transport theory.
• Computability: The derived formulas for counting distributions provide computable measures of deviation from normality that rely only on the distribution functions of the underlying models.

The authors position these results as providing a unified perspective on fundamental constructions in probability and mathematical statistics, highlighting the central role of distributional representations and extremal dependence structures.