Tilted sensitivity analysis in matched observational studies

Problem
In matched observational studies, researchers pair treated and control units with similar observed covariates to estimate treatment effects. However, unobserved confounders may still induce hidden bias, where units with larger responses are more likely to receive treatment. Sensitivity analysis, following the model of Rosenbaum (1987), quantifies the robustness of findings to such bias by determining the magnitude of unmeasured confounding ($\Gamma$) required to invalidate the study's conclusions.

While sensitivity analysis for paired studies (one treated, one control) is well-established with closed-form solutions for worst-case p-values and design sensitivity, extending these methods to studies with multiple controls (matched sets with $n_i > 2$) presents significant technical hurdles. The prevailing approach, the "separable algorithm" (Gastwirth et al., 2000), requires solving $I$ optimization problems to find the worst-case pattern of unmeasured confounding. Crucially, for $n_i > 2$, this algorithm does not yield a closed-form expression for the worst-case expectation, and the worst-case pattern changes as the sensitivity parameter $\Gamma$ varies. This lack of closed-form solutions hinders the analytic calculation of "design sensitivity" (Rosenbaum, 2004)—a metric describing a test statistic's ability to distinguish treatment effects from bias—forcing reliance on Monte Carlo simulations for theoretical comparisons.

Methodology
The paper introduces a "tilted sensitivity analysis" procedure. For any candidate test statistic $T = \sum Z_{ij}q_{ij}$, the method defines a modified, "tilted" statistic $T_{\Gamma i}$ for each matched set $i$:
$$T_{\Gamma i} = T_i - \bar{q}_i - \left(\frac{\Gamma - 1}{1 + \Gamma}\right)|T_i - \bar{q}_i|$$
where $T_i$ is the original statistic for set $i$, and $\bar{q}_i$ is the mean of the scores $q_{ij}$ within that set.

This modification is constructed such that the worst-case pattern of unmeasured confounding (the vector $u$ maximizing the expectation under the sharp null) is both available in closed form and invariant to the value of $\Gamma$. Specifically, the worst-case confounder assigns $u_{ij}=1$ if $q_{ij} > \bar{q}_i$ and $u_{ij}=0$ otherwise. Under this specific pattern, the expectation of the tilted statistic is zero, and its variance is given by a closed-form expression.

The paper demonstrates that this approach unifies existing methods:

1. Sign-score statistics: For statistics taking only two values (e.g., in paired designs or Mantel-Haenszel tests), the tilted approach reproduces the conventional sensitivity analysis exactly.
2. Inverse Probability Weighting (IPW): The tilted statistic is shown to be equivalent to a minimax-centered IPW statistic, differing only by a matched-set weight.

Key Contributions

1. Closed-Form Worst-Case Distribution: Unlike the separable algorithm, the tilted approach provides a closed-form expression for the worst-case expectation and variance for any matched structure, including those with multiple controls. This eliminates the need for iterative optimization in the sensitivity analysis step.
2. Closed-Form Design Sensitivity: The paper derives a closed-form expression for the design sensitivity of the tilted statistic for any matched structure:
   $$\tilde{\Gamma}_{tilt} = \frac{\eta + \theta}{\eta - \theta}$$
   where $\theta$ is the probability limit of the average treatment effect contribution and $\eta$ is the probability limit of the average absolute contribution. This allows for analytic comparisons of competing test statistics, which was previously infeasible for multi-control designs without simulation.
3. Adaptive Combination: The paper proposes an adaptive procedure that combines the tilted and conventional approaches. Using a multivariate one-sided testing framework (Cohen et al., 2020), this method adaptively weights the two statistics to achieve a design sensitivity no smaller than the maximum of the two individual approaches, guarding against a priori selection of a suboptimal method.

Results

• Theoretical Comparisons: Through analytic derivations and Monte Carlo simulations, the paper shows that the tilted approach generally improves design sensitivity over the conventional approach for symmetric and right-skewed error distributions. Conversely, for left-skewed distributions, the conventional approach often retains superior design sensitivity.
• Effect of Controls: For $m$-statistics (a class including difference-in-means and rank tests), the design sensitivity of the tilted approach is non-decreasing in the number of matched controls $J$, and strictly increasing under mild conditions.
• Real Data Applications: In five real-data examples (covering smoking, fish consumption, education, alcohol, and light drinking), the tilted approach frequently yielded higher sensitivity values (indicating greater robustness to hidden bias) than the conventional approach, particularly in datasets with right-skewed outcomes or specific test statistics. The adaptive approach generally performed as well as the best individual method and significantly better than the worst.

Significance
The paper claims that the tilted sensitivity analysis offers a practical and theoretically superior alternative for matched observational studies with multiple controls. By providing closed-form solutions for both the worst-case distribution and design sensitivity, it removes the computational and theoretical barriers that have limited the development of robust inference in multi-control settings. The ability to analytically compare design sensitivities facilitates better study design and the selection of more robust test statistics. Furthermore, the adaptive combination ensures that researchers need not choose between the tilted and conventional methods a priori, providing a safeguard against poor methodological choices while maintaining or improving the reported robustness of observational findings.