Technical Summary: The Limits of AI-Driven Allocation under Aleatoric Uncertainty

1. Problem Statement

In high-stakes humanitarian and social policy contexts, decision-makers must allocate scarce resources (e.g., cash transfers, medical treatment, demining teams) to a population with heterogeneous, latent vulnerability. While universal screening (physical verification of eligibility) is ideal, it is often prohibitively expensive due to logistical constraints. Consequently, there has been a shift toward algorithmic targeting, where resources are allocated based on predicted risk scores ($\mu(X) = P(Y=1|X)$) derived from observable covariates.

However, the paper argues that algorithmic targeting alone cannot eliminate misallocation due to aleatoric uncertainty. Even with a Bayes-optimal risk model, the true vulnerability status $Y$ of an individual with a given risk score $\mu(X) \in (0,1)$ remains stochastic. This irreducible randomness means that purely algorithmic strategies inevitably misallocate some resources to non-vulnerable units or miss vulnerable ones. The central problem addressed is how to optimally combine screening (direct observation of $Y$ for a subset of units) with algorithmic targeting to maximize allocation efficiency under fixed budget constraints for both screening ($\alpha$) and resource allocation ($\beta$).

2. Methodology and Framework

The Two-Stage Allocation Framework

The authors formalize a two-stage decision process:

1. Screening Stage: A decision-maker selects a subset of units $S(X)=1$ to observe their true vulnerability status $Y$, subject to a screening budget $\alpha$.
2. Allocation Stage: Resources are distributed subject to an allocation budget $\beta$.
   • Screened units with $Y=1$ receive the resource with certainty.
   • Screened units with $Y=0$ are excluded.
   • Unscreened units are allocated based solely on the risk score $\mu(X)$.

The objective is to maximize allocation efficiency, defined as the probability of correctly allocating to vulnerable units: $E[\pi(X,Y)Y]$, subject to the constraints $E[S(X)] \le \alpha$ and the total allocation constraint.

Theoretical Derivation

The paper derives the optimal policy by analyzing the interaction between the screening and allocation budgets:

• Optimal Screening Set: The authors prove that the optimal screening strategy does not target the highest-risk or lowest-risk units. Instead, it targets an interval of risk scores $[q_\alpha, q_\beta]$ located at the margin of algorithmic allocation.
  • Units with risk scores above $q_\beta$ are directly allocated without screening (high risk justifies direct allocation).
  • Units with risk scores below $q_\alpha$ are neither screened nor allocated (low probability of $Y=1$ makes screening inefficient).
  • Units in the intermediate band $[q_\alpha, q_\beta]$ are screened to resolve aleatoric uncertainty before allocation.
• Fixed-Point Algorithm: Since the thresholds $q_\alpha$ and $q_\beta$ generally require solving a nonlinear integral equation without a closed-form solution, the authors develop a fixed-point algorithm. This algorithm iteratively adjusts the screening band based on the average risk within the band and the resulting "saved" allocation budget from excluding confirmed non-vulnerable units.

Key Theoretical Properties

• Distribution-Free Characterization: The structure of the optimal policy (an interval at the margin) holds regardless of the underlying risk distribution $F$, provided the risk distribution has full support.
• Concavity and Diminishing Returns: The value of screening is shown to be concave with respect to the screening budget $\alpha$. This establishes a formal law of diminishing returns for physical verification.
• Complementarity: The paper characterizes screening and algorithmic targeting as complements. Screening gains are maximized when aleatoric uncertainty is high, whereas algorithmic targeting acts as a near-perfect substitute for screening only when risk scores are highly informative (low aleatoric uncertainty).

3. Key Contributions

1. Optimal Screening Characterization: The paper provides a rigorous characterization of the optimal screening strategy in a two-stage framework, proving that the optimal set is an interval targeting units at the margin of algorithmic allocation.
2. Closed-Form and Algorithmic Solutions: The authors derive the optimal screening set in closed form for uniform risk distributions and provide a fixed-point algorithm to recover the screening band for arbitrary distributions.
3. Distribution-Free Value of Screening: They establish that the efficiency gain from screening is concave in the screening budget, regardless of the underlying risk distribution, formalizing the law of diminishing returns.
4. Empirical Relationship with Aleatoric Uncertainty: The work empirically demonstrates that the marginal gains from screening grow monotonically as the population-level aleatoric uncertainty increases.
5. Real-World Validation: The framework is applied to two real-world datasets:
   • Humanitarian Demining in Colombia: Predicting landmine contamination to prioritize demining teams.
   • Income-Based Social Programs (US): Identifying low-income individuals for social protection using Census data.

4. Results

Synthetic Experiments

Using synthetic populations with risk scores drawn from a Beta distribution family ($\mu \sim \text{Beta}(t, t)$), the authors varied the level of aleatoric uncertainty:

• Low Uncertainty ($t \to 0$): Risk scores are highly informative (bimodal distribution). Algorithmic targeting alone achieves near-perfect precision; screening yields negligible gains.
• High Uncertainty ($t \to \infty$): Risk scores are uninformative (converging to $\delta_{0.5}$). Screening yields large efficiency gains even though the risk scores carry no information, as screening resolves the inherent randomness.
• Intermediate Uncertainty: The study identifies an "S-shaped" curve where the value of screening grows steeply. Real-world risk distributions typically fall in this steep region, where screening acts as a powerful complement to algorithmic targeting.

Real-World Applications

• Demining (Colombia): Using logistic regression, random forest, and gradient boosting models on 63 features, the optimal screening framework improved detection precision from 39% (no screening) to 70% (with optimal screening at $\alpha = \beta$).
• Social Programs (US): Using ACS data to predict income below $50,000, the framework improved precision from 65% to 80%.
• Comparison: In both cases, optimal screening significantly outperformed random screening. Random screening yielded negligible gains, confirming that the location of the screening (targeting the margin) is critical, not just the volume of screening.
• Robustness: Despite using imperfect risk estimates (epistemic uncertainty), the framework delivered substantial gains, suggesting robustness in practical settings.

5. Significance and Claims

The paper claims to define the limits of AI-driven allocation under aleatoric uncertainty. Its primary significance lies in shifting the paradigm from viewing screening and algorithmic targeting as substitutes to viewing them as complements.

• Policy Implication: Decision-makers should not eliminate screening in favor of AI, nor rely solely on screening. Instead, they should use AI to identify the highest-risk units for direct allocation and reserve screening resources for the "marginal" units where uncertainty is highest.
• Budget Calibration: The concavity of the screening value function provides a principled basis for calibrating screening budgets, justifying targeted screening over universal screening.
• Operational Relevance: By demonstrating substantial efficiency gains in humanitarian demining and social protection, the paper argues that integrating human verification (screening) with algorithmic targeting is essential for realizing the full potential of AI-assisted resource allocation.

The authors explicitly state that their work assumes access to Bayes-optimal risk estimates for the theoretical derivation, acknowledging that in practice, estimation error (epistemic uncertainty) may shift the optimal screening set. However, their empirical results suggest the framework remains effective even with imperfect models.