Can National Administrative Data Identify Optimal Nurse Staffing? Evidence of Estimand Instability and Identification Challenges in CMS Home Health Panel Data

Technical Summary: Identification Challenges in Estimating Optimal Nurse Staffing in Home Health Care

Problem Statement
The study addresses a critical gap in health services research: the inability to empirically identify an optimal registered nurse (RN) staffing level in the U.S. home health sector using national administrative data. While production-function theory posits an interior optimum where marginal quality benefits equal marginal labor costs ($Q'(N^*) = C'(N^*)$), no national-scale study has successfully estimated this optimum using U.S. Medicare data. The research investigates whether the currently available CMS Home Health Compare panel data (FY2016–2018) possesses the necessary structural variation and identification power to reliably estimate the staffing-quality production function, or if the data infrastructure itself precludes such estimation.

Methodology
The study employs a feasibility and identification design using a merged panel of 16,828 agency-years (analytic subpanel $n \approx 10,496$ for quality monetization). The analysis integrates three primary data sources: CMS Home Health Compare public use files, CMS Home Health Agency Cost Reports (Form CMS-1728-94), and CMS Inpatient Prospective Payment System (IPPS) reports.

• Variables:
  • Dose: RN Full-Time Equivalent (FTE) per 100 skilled-nursing patients.
  • Quality: A T-standardized composite of four CMS measures (Average 4Ts), including process/outcome ratings, patient satisfaction (HHCAHPS), Medicare spending ratios, and community discharge rates.
  • Cost: Skilled nursing labor costs per patient.
• Estimators: Three linear estimators were applied to assess dose–quality and dose–cost relationships:
  1. Within-CRE (Correlated Random Effects): Mathematically equivalent to a within-agency fixed-effects model via the Mundlak–Chamberlain theorem.
  2. Pooled: Exploits both within- and between-agency variation.
  3. Between-Agency: Pure cross-sectional comparison of agency means.
• Validation & Sensitivity:
  • Nonparametric Validation: A Wide & Deep Bayesian Neural Network (BNN) with Monte Carlo Dropout was used to test the marginal predictive contribution of staffing dose.
  • Robustness Checks: Leverage analysis (removing high-influence observations), Huber robust regression, and Equivalence Testing (TOST) against clinically meaningful benchmarks.
  • Break-Even Analysis: Calculated the theoretical quality gradient required to offset observed cost increases.
  • Natural Experiment: Attempted to utilize the Home Health Value-Based Purchasing (HHVBP) rollout in nine states as an exogenous shock, though pre-period data limitations prevented a formal difference-in-differences design.

Key Results
The study finds that the staffing-quality relationship is not stably identified in the available data, exhibiting significant estimand instability depending on the estimator chosen:

1. Cost Function ($C(N)$): The relationship between staffing intensity and labor cost is robustly positive across all estimators ($\beta \approx +212$ to $+447$ per patient per dose unit, $p < 0.001$). This aligns with an accounting identity where labor costs rise with FTEs.
2. Quality Function ($Q(N)$):
   • Within-CRE: Produced a null result ($\beta = +0.014, p=0.50$) that was highly leverage-fragile; removing the top 0.1% of observations reversed the sign to negative.
   • Pooled & Between-Agency: Both yielded statistically significant negative associations ($\beta \approx -0.036$ to $-0.037, p \approx 0.02–0.03$), suggesting that higher staffing correlates with lower quality in these specifications, likely due to unresolved confounding.
   • BNN Validation: Removing the staffing dose from the model resulted in a negligible change in validation loss ($-0.17\%$), indicating staffing intensity carries no marginal predictive information for quality beyond covariates.
   • Equivalence Testing: TOST confirmed the quality effect is practically equivalent to zero relative to clinically meaningful benchmarks ($p \approx 4 \times 10^{-140}$).
3. Break-Even Analysis: Under the most optimistic assumption (using the upper bound of the within-CRE estimate), the observed quality response is approximately 365 times smaller than the threshold required to offset the marginal cost of additional staffing.
4. Data Artifacts: The study identified two specific data quality failure modes:
   • A 2017 reporting convention shift where ~20% of agencies changed from fractional FTE reporting to a minimum unit of 1.00, creating spurious within-agency variation.
   • A "shared-denominator artifact" where quality and cost measures derived from the same patient count are mechanically correlated with the staffing dose.

Key Contributions

• Identification Diagnostic: The paper provides a diagnostic map demonstrating that current CMS administrative panel data cannot support the identification of an optimal staffing level. It clarifies that this failure is due to data structure and endogeneity, not necessarily the falsification of production-function theory.
• Methodological Toolkit: It introduces and operationalizes specific diagnostic procedures for CMS data, including raw-numerator correlation testing, N-control ablation, and alternative-denominator cross-validation to detect shared-denominator artifacts.
• Economic Translation: The break-even framework translates econometric null results into policy-relevant quantitative claims, showing that even under optimistic assumptions, the economic return on incremental RN staffing is undetectable in the current data.
• Data Quality Protocol: The study documents a multi-stage screening protocol that identifies previously unreported failure modes in CMS Cost Report data, specifically regarding FTE reporting shifts and implausible cost-per-patient ratios.

Significance and Claims
The paper explicitly states that its findings do not imply that nurse staffing is inconsequential to home health quality or that an optimal staffing level does not exist. Rather, the study delineates the specific data conditions required for future identification efforts.

The authors claim that the current data infrastructure lacks the exogenous variation necessary to disentangle staffing decisions from agency characteristics (patient case mix, organizational capability). Consequently, the sign and significance of the staffing-quality relationship are contingent on estimator choice, rendering any point estimate of an "optimum" unreliable.

The significance of the work lies in redirecting future research agendas. The authors argue that reliable identification will require:

1. Patient-level linkage of staffing episodes to individual outcomes.
2. Pre-period data (pre-2016) to enable difference-in-differences designs exploiting HHVBP rollout.
3. Credible instruments (e.g., local labor supply shocks or regulatory discontinuities).
4. State-level variation in minimum staffing standards for regression discontinuity designs.

In summary, the paper concludes that while the theoretical framework for optimal staffing remains valid, the empirical verification of this optimum in home health care is currently precluded by the limitations of national administrative panel data.