📊 statistics

Joint Bayesian models for validating spatial health-event databases against a gold standard: separating global and local discrepancies

This paper proposes a novel Bayesian framework for rigorously validating spatial health-event databases against a gold standard by simultaneously quantifying global shifts and detecting local discrepancies, demonstrating its effectiveness through simulation and real-world application to Crohn's disease data.

Original authors: Mathias Brugel, Florine Kempf, Camille Ternynck, Marta Blangiardo, Michaël Génin

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mathias Brugel, Florine Kempf, Camille Ternynck, Marta Blangiardo, Michaël Génin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have two different maps of the same city. One is the "Gold Standard" map, drawn by expert cartographers who have spent decades measuring every street and building with perfect precision. The other is a "Candidate" map, created quickly using a new, cheaper method (like crowd-sourced data or a different satellite).

You want to know: Is the new map good enough to use?

This paper proposes a new, smart way to compare these two maps, specifically for health data (like where diseases occur). The authors argue that simply looking at the maps side-by-side isn't enough. You need to ask two specific questions:

The Global Question: Is the whole new map just "smaller" or "bigger" than the real one? (Like if the new map shrank the whole city by 10%?)
The Local Question: Are there specific neighborhoods where the new map is completely wrong, even if the rest of the city looks okay?

The Problem with Old Methods

Previously, statisticians had tools to check if two maps looked similar, but they were like a blunt instrument. They couldn't easily tell the difference between a map that was consistently off by a little bit everywhere (a global shift) versus a map that was wildly wrong in just a few specific spots (local errors). They also struggled to treat one map as the "truth" and the other as the "student" being graded.

The New Solution: A Bayesian "Twin" System

The authors created a statistical framework (a set of mathematical rules) that acts like a detective. They use three different "detective personas" (models) to solve the case:

The Random Detective (REM): Assumes that if the new map is wrong, the errors are just random noise, like static on a radio.
The Pattern Detective (SEM): Assumes that if the new map is wrong, the errors might follow a pattern (like a whole district being mislabeled).
The Shared-Root Detective (SCM): Assumes both maps are trying to draw the same underlying picture but have their own unique "quirks."

How They Solve the Mystery

1. Catching the Global Shift (The "Volume Knob")
The system first checks the "volume knob" of the new map. It calculates a number called $RR_{global}$ .

If the number is 1, the new map is the same size as the real one.
If the number is 0.93, the new map is consistently about 7% "quieter" (lower) than the real one across the entire territory.
Analogy: Imagine the Gold Standard map says there are 100 trees in a park. If the new map says there are 93 trees everywhere, the "Global Shift" detector catches this immediately. It tells you, "Hey, the whole map is just scaled down."

2. Catching Local Errors (The "Spotlight")
Next, the system shines a spotlight on individual neighborhoods to see if they are outliers.

The Problem: Sometimes, if a map is shifted globally, the math can trick you into thinking a specific neighborhood is "wrong" just because of how the numbers balance out.
The Fix: The authors invented a new tool called RCEP (Robustly Centred Exceedance Probability). Instead of asking, "Is this neighborhood different from zero?" it asks, "Is this neighborhood different from the average error of the whole map?"
Analogy: If the whole class gets a 10% lower grade than usual, a student who got a 90% isn't "failing" just because they are below 100%. They are actually doing great relative to the class. The RCEP tool prevents the system from falsely accusing a neighborhood of being an error just because the whole map is slightly off.

The Simulation Test (The "Stress Test")

Before using this on real data, the authors tested their system with computer simulations. They created fake maps and intentionally messed them up in different ways:

Uniform mess: They shrunk the whole map.
Clustered mess: They messed up a specific cluster of neighborhoods.
Random mess: They messed up random spots.

The Results:

The Global Detector was perfect at spotting when the whole map was shifted.
The Local Detector (RCEP) was the best at finding specific bad neighborhoods without crying "wolf" when the whole map was just slightly off.
The "Pattern Detective" (SEM) and "Random Detective" (REM) performed very similarly, but the "Shared-Root Detective" (SCM) was a bit more cautious and conservative.

The Real-World Test: Crohn's Disease in France

The authors applied their system to real data about Crohn's disease (a type of bowel disease).

Source A (Gold Standard): A long-term, high-quality registry of patients in Northern France.
Source B (Candidate): A national hospital database (PMSI), which is cheaper and covers the whole country but might be less precise.

The Verdict:

Global: The hospital database was about 7% lower than the registry everywhere. It wasn't "wrong" in a broken way; it was just consistently undercounting.
Local: The system found zero specific neighborhoods where the hospital data was wildly different from the registry. The hospital data successfully copied the pattern of where the disease was high and low; it just had a lower overall number.

The Takeaway

This paper gives researchers a new, reliable toolkit. It teaches us that when comparing two maps, we shouldn't just look for "errors." We need to separate global shifts (is the whole thing off?) from local errors (is this specific spot broken?).

The authors conclude that the hospital data is actually quite good for seeing the geography of the disease (where it happens), but it needs a "recalibration" (a volume knob adjustment) to match the exact numbers of the gold-standard registry. This framework helps scientists decide when they can safely reuse old or cheaper data without getting misled.

Technical Summary: Joint Bayesian Models for Validating Spatial Health-Event Databases

Problem Statement
The secondary use of medico-administrative and synthetic spatial data offers a cost-effective alternative to population-based registries for spatial epidemiology. However, rigorous validation against a gold standard (GS) is required to ensure these candidate-for-reuse databases (CFRD) adequately reproduce the spatial signal of the original source without distorting geographical structures. Current methodological literature lacks a tool capable of spatially validating a CFRD against a GS in a manner that is both paired (same areal support) and asymmetric (one source is the reference, the other is the candidate). Existing methods, such as Moran's I, spatial scan statistics, or point process comparisons, are often descriptive, symmetric, or fail to explicitly model areal count uncertainty and decompose disagreement into global shifts versus local discrepancies.

Methodology
The authors propose a Bayesian hierarchical framework for map-to-map validation that distinguishes between two dimensions of disagreement: global (map-wide shifts) and local (area-specific deviations).

Modeling Framework:
- The framework assumes a Poisson sampling distribution for observed counts, with expected counts standardized to a common scale to ensure comparability.
- Three joint Bayesian models are developed to represent the relationship between the reference (Database 1) and the candidate (Database 2):
  - Random Error Model (REM): Database 2 is modeled as the reference plus a shared spatial field and a purely random (independent) error term.
  - Structured Error Model (SEM): Similar to REM, but the error term is modeled as a spatially structured field (using BYM2 parametrization), capturing residual spatial coherence in the discrepancies.
  - Shared Component Model (SCM): A symmetric benchmark where both databases are modeled as deviations from a common latent shared field, rather than one being a perturbation of the other.
Discrepancy Summaries:
- Global Discrepancy: Quantified by the difference in database-specific intercepts ( $\Delta = \alpha^{(2)} - \alpha^{(1)}$ ), transformed into a multiplicative global relative risk ( $RR_{global} = \exp(\Delta)$ ). This captures average map-wide shifts.
- Local Discrepancy: Assessed via posterior summaries of the source-specific error terms. Two decision rules are proposed:
  - Null-Referenced Exceedance Probability (NREP): The probability that the local error is positive ( $Pr(\psi_i > 0)$ ).
  - Robustly Centred Exceedance Probability (RCEP): The probability that the local error deviates from a robust global center ( $c^\star$ , the median of posterior medians) by more than a tolerance $\epsilon$ . This rule is designed to distinguish genuine local anomalies from broad shifts that may be artificially amplified by the sum-to-zero constraints of latent spatial models.

Simulation Study Results
The framework was evaluated using simulations on North Carolina county data under various perturbation scenarios (null, uniform, clustered, and random).

Global Detection: The intercept contrast ( $RR_{global}$ ) accurately recovered map-wide shifts across all models and scenarios. Under uniform perturbations, it correctly identified the shift; under localized perturbations, it remained close to 1, correctly indicating the absence of a global shift.
Local Detection:
- Model Performance: The asymmetric error models (REM and SEM) generally outperformed the symmetric SCM in local detection, particularly in clustered and sparse random scenarios. SEM and REM showed similar performance, suggesting that the assumption of spatial structure in the error term does not drastically alter conclusions.
- Decision Rules: The RCEP rule demonstrated superior operating characteristics compared to NREP. RCEP provided a better trade-off between sensitivity and false-positive control, especially in clustered settings where NREP tended to be overly liberal and generate high false discovery rates. A threshold of 0.9 for RCEP was identified as offering the best balance.
- Sensitivity: Local detection performance deteriorated significantly in scenarios with rare events (low incidence) or when 50% of the territory was modified in the same direction (transforming local anomalies into diffuse shifts).

Application to Crohn's Disease Data
The framework was applied to compare the EPIMAD registry (reference) with the French national hospital discharge database (PMSI, candidate) for Crohn's disease in northern France.

Global Findings: All three models consistently estimated a global discrepancy, with the PMSI signal being approximately 7% lower than the EPIMAD signal ( $RR_{global} \approx 0.93$ ).
Local Findings: No spatial unit exceeded the local RCEP detection threshold (0.9) in any of the models.
Conclusion: The PMSI database reproduces the broad geographical organization of Crohn's disease observed in the registry but underestimates the absolute level of the signal. It is suitable for large-scale spatial analysis but requires recalibration for absolute incidence estimation.

Significance and Claims
The paper claims to provide the first principled Bayesian framework specifically designed for the asymmetric validation of spatial health-event databases. Its key contributions include:

Decomposition of Disagreement: It explicitly separates global calibration errors (intercept shifts) from local spatial discrepancies, addressing a gap where previous methods often conflated the two or failed to model them jointly.
Operational Decision Rules: The introduction of the RCEP rule addresses a specific statistical artifact in centered latent models, offering a more robust method for identifying atypical local areas compared to standard null-referenced thresholds.
Practical Utility: The application demonstrates that a database can be "validated" for spatial structure (preserving gradients) even if it is not validated for absolute magnitude, providing a nuanced basis for data reuse decisions.

The authors note limitations, including the reliance on a single areal configuration in simulations and the fact that RCEP treats the robust center as fixed (plug-in) rather than fully integrating its uncertainty. They suggest future extensions to other outcome distributions (e.g., binomial, Gaussian) and spatio-temporal models as natural next steps.