Generative Augmented Inference

1. Problem Formulation

The paper addresses a critical bottleneck in data-driven operations management: the high cost and scarcity of human-generated labels (e.g., survey responses, expert annotations, purchase decisions) required to estimate statistical models. While Large Language Models (LLMs) and AI systems offer abundant, low-cost auxiliary data, integrating them into statistical inference presents two fundamental challenges:

1. Systematic Bias: AI outputs often differ systematically from human judgments (e.g., an LLM predicting a 30% purchase rate when the true rate is 44%). Treating AI outputs as direct "noisy proxies" for true labels introduces bias that can overwhelm efficiency gains.
2. Structural Mismatch: AI outputs are often structured auxiliary representations (e.g., reasoning traces, high-dimensional embeddings, confidence scores, persona descriptions) rather than simple labels. Conventional methods like Prediction-Powered Inference (PPI) assume AI outputs are surrogates for the outcome, failing when the AI output is categorically different from the target variable or depends on latent information not present in the observed covariates.

The Core Question: How can AI-generated signals be incorporated into statistical estimation to improve efficiency without requiring them to serve as accurate surrogates for the outcome?

2. Methodology: Generative Augmented Inference (GAI)

The authors propose GAI, a framework that shifts the paradigm from treating AI outputs as surrogate labels to treating them as informative features for predicting human outcomes.

Conceptual Shift

• Proxy View (Existing): AI output $z \approx y$. If biased, it must be corrected.
• Feature View (GAI): AI output $z$ is a feature that helps predict $y$. Even a biased $z$ contains information about $y$ (e.g., a biased prediction shifts the probability distribution).

The Estimator

GAI estimates parameters $\beta^*$ in a Generalized Linear Model (GLM) framework (potentially misspecified) using a Neyman-orthogonal score function.

1. Data Structure:

   • $D_P$: Primary sample with human labels ($y$) and AI features ($z$).
   • $D_A$: Auxiliary sample with only AI features ($z$) and no human labels.
   • $w$: Indicator variable for whether $y$ is observed.

2. Nuisance Function Estimation:
   The method estimates two nuisance functions using flexible machine learning (e.g., Random Forests, Neural Networks) on the labeled subset:

   • Outcome Prediction ($g$): $g(X, z) = E[y | X, z]$. This captures the predictive power of AI features.
   • Propensity Score ($e$): $e(X, z) = P(w=1 | X, z)$. This models the probability of a human label being observed.

3. Orthogonal Score Construction:
   The estimator solves for $\beta$ using the following score function $\psi$:
   $$\psi(\Xi; e, g; \beta) = X^\top \left[ \nabla b(X\beta) - g(X,z) + \frac{w}{e(X,z)}(g(X,z) - y) \right]$$

   • The term $g(X,z)$ imputes the missing $y$ for unlabeled data.
   • The term $\frac{w}{e(X,z)}(g(X,z) - y)$ corrects for the bias in imputation using Inverse Probability Weighting (IPW).
   • Neyman Orthogonality: This construction ensures that estimation errors in the nuisance functions ($g$ and $e$) affect the final parameter estimate only at the second order. This allows the use of complex, data-adaptive ML methods for nuisance estimation without invalidating the asymptotic inference.

4. Implementation:
   The algorithm uses cross-fitting (splitting data into $K$ folds) to ensure that nuisance functions are estimated on data independent of the observations used for the final score calculation, preventing overfitting and ensuring valid confidence intervals.

3. Key Contributions

Theoretical Contributions

• Asymptotic Normality: The GAI estimator is proven to be $\sqrt{n}$-consistent and asymptotically normal with a closed-form variance expression, enabling standard hypothesis testing.
• "Safe Default" Property: Under random labeling (where $w$ is independent of $y$ and $z$ given $X$), GAI is guaranteed to weakly dominate estimators using only human data. If the AI outputs contain any predictive information (even if biased), GAI strictly improves efficiency. It never hurts performance by including AI data.
• Variance Decomposition: The authors decompose the efficiency gains into three sources:
  1. Sample Expansion: Leveraging the larger size of the auxiliary dataset.
  2. Representational Power: AI features ($z$) may provide better representations of covariates $X$ (e.g., via embeddings) that help approximate complex conditional expectations, even if $y \perp z | X$.
  3. Extra Predictive Information: AI features contain signal about $y$ that is not present in $X$ (e.g., latent persona traits in digital twins).

Methodological Contribution

• GAI generalizes beyond the "surrogate label" assumption of PPI. It handles high-dimensional, unstructured, and stochastic AI outputs (like text embeddings) that PPI cannot utilize.

4. Empirical Results

The authors validate GAI across three diverse business applications, comparing it against Primary-only, Naive pooling, PPI, and PPI++.

Key Findings:

• Robustness: GAI consistently outperforms benchmarks regardless of whether AI outputs are biased, unstructured, or highly accurate.
• Inference Quality: Unlike PPI, which often undercovers (confidence intervals miss the true value) or produces overly wide intervals, GAI maintains nominal coverage (95%) with comparable or narrower interval widths.
• Decision Accuracy: GAI significantly reduces decision errors (e.g., incorrectly inferring the sign of a price elasticity) compared to all other methods.

5. Significance

• Operational Impact: GAI provides a principled, scalable solution to the "data scarcity" problem in operations management. It allows researchers and practitioners to achieve the statistical precision of massive datasets while collecting only a fraction of the expensive human labels.
• Theoretical Advancement: It bridges the gap between causal inference (Neyman orthogonality, double machine learning) and the emerging field of AI-augmented data, offering a rigorous framework for the "Sim-to-Real" problem where AI simulations are used to inform real-world decisions.
• Practical Utility: The "Safe Default" property eliminates the risk of using AI data; if the AI is useless, GAI performs as well as ignoring it, but if the AI is useful (even slightly), GAI strictly improves results. This makes it a robust tool for integrating AI into decision-making pipelines across supply chain, healthcare, and marketing.