Eligibility for shingles vaccination and hospital-coded dementia in England and Wales: a regression discontinuity analysis in England

Technical Summary: Eligibility for Shingles Vaccination and Hospital-Coded Dementia in England and Wales

Problem and Background
There is substantial interest in whether existing licensed interventions, specifically the live-attenuated shingles vaccine (Zostavax), might reduce dementia incidence through mechanisms beyond their primary indication, such as preventing neurotropic herpesvirus reactivation or inducing trained innate immunity. Previous quasi-experimental studies, notably a regression discontinuity design (RDD) using the Secure Anonymised Information Linkage (SAIL) databank in Wales, reported a large but imprecisely estimated protective effect of shingles vaccination on dementia (approximately a 20% relative risk reduction). Similar findings were reported in Canada and Australia. However, these studies utilized modest sample sizes, and the biological mechanisms remain unresolved. The authors sought to independently replicate these findings in a substantially larger dataset to interrogate the reliability of the reported effects and provide more precise effect estimation.

Methodology
The study employed a regression discontinuity design (RDD) centered on the September 2013 age-based rollout of the live-attenuated shingles vaccine in England.

• Data Source: The primary analysis utilized Hospital Episode Statistics (HES), an administrative dataset covering all NHS inpatient and outpatient activity in England. The dataset included 6,340,214 individuals born within eight years of the September 1933 eligibility threshold (September 1925 to September 1941).
• Running Variable: Due to data access limitations (year and month of birth available, but not exact day), the running variable was defined as the month of birth relative to September 1933. Individuals born in or after September 1933 were eligible for the catch-up program; those born before were not.
• Outcomes: The study estimated the intention-to-treat (ITT) effect of vaccine eligibility on two outcomes: hospital-coded shingles (ICD-10 code B02) and hospital-coded dementia.
• Analytical Approach: The authors used the rdrobust R package to select mean-squared-error (MSE) optimal bandwidths for local linear regression. Primary analyses used HES hospital-activity denominators, with Office for National Statistics (ONS) population denominators used for sensitivity checks.
• Validation and Sensitivity: To address discrepancies with the Welsh SAIL study, the authors conducted a series of robustness checks:
  • SAIL Replication: They accessed the SAIL Databank to reproduce the Welsh analysis and then imposed HES-like restrictions (e.g., hospital-only ascertainment, exclusion of prevalent disease) to test if specific dataset features could explain a null result.
  • Negative Controls: Analyses were performed on top 20 ICD-10 codes (e.g., Type 2 diabetes) to detect spurious discontinuities.
  • Temporal Analysis: Follow-up periods were varied (1 to 8 years), and grace periods were introduced to account for the delay between primary care diagnosis and hospital coding.
  • Secondary Analysis: An independent team (SAHSU) analyzed a separate HES extract with a longer pre-programme lookback (2003–2023) and week-of-birth resolution.

Key Results

• Shingles Diagnosis: The study confirmed the validity of the RDD approach by demonstrating a clear, robust protective effect of vaccine eligibility on hospital-coded shingles. The RD estimate was a reduction of 0.12 percentage points (95% CI -0.153 to -0.079; p = 5.9 x 10⁻¹⁰), consistent with randomized controlled trial data.
• Dementia Diagnosis: In contrast, there was no evidence of a corresponding reduction in hospital-coded dementia diagnoses. The RD estimate was -0.06 percentage points (95% CI -0.40 to 0.27; p = 0.72), indicating an effect close to null.
• Sensitivity and Robustness:
  • Results remained null across various denominator definitions (HES vs. ONS), diagnostic code specifications, and estimator choices.
  • Sex-stratified analyses showed no protective effect in either men or women.
  • A nominal protective association observed at the 1-year follow-up was attributed to prevalent cases recorded after the index date and the play of chance, as it attenuated to null by two years and was not specific to the true cutoff in placebo analyses.
  • SAIL Comparison: When applying HES-like restrictions to the SAIL data, the protective effect observed in Wales persisted. The authors concluded that differences in sampling frames (hospital-only vs. all-source), denominators, or the exclusion of prevalent disease in HES were insufficient to explain the discrepancy between the null English HES results and the positive Welsh SAIL results.
  • Birth-Month Structure: The analysis identified strong birth-month seasonal structure in certain outcomes (notably Type 2 diabetes), which could generate apparent discontinuities. However, adjusting for this structure did not alter the null finding for dementia.

Significance and Claims
The paper claims that its findings do not support a detectable intention-to-treat effect of live-attenuated shingles vaccine eligibility on hospital-coded dementia in England.

• Replication Failure: Despite an 18-fold increase in sample size compared to the original Welsh study, the authors failed to replicate the large protective effect on dementia.
• Dataset Bias: The study systematically ruled out major dataset-specific biases (incomplete prevalent disease exclusion, hospital-contact sampling frames, and secondary care coding reliance) as explanations for the discrepancy between the English and Welsh results.
• Scope of Evidence: The authors emphasize that these results relate specifically to the live-attenuated vaccine, which has largely been replaced by the recombinant vaccine (Shingrix). They note that while their data does not support a large effect for the live vaccine in this setting, it does not settle the wider question regarding the recombinant vaccine or effects in other data sources.
• Future Directions: The authors suggest that further epidemiological investigations using primary care data (e.g., OpenSAFELY, CPRD) and randomized controlled trials (currently planned in Finland and Denmark) are necessary to definitively resolve the question of shingles vaccination and dementia prevention.

The study concludes that the English HES-based evidence is more compatible with no large effect on dementia than with a strong protective effect, highlighting the need for caution when interpreting quasi-experimental estimates that vary significantly across datasets and analytical choices.