Where species distribution models fail under occurrence-data contamination: calibration error concentrates at stream-network headwaters

Technical Summary: Calibration Failure at Stream-Network Headwaters in Species Distribution Models

Problem Statement
Species distribution models (SDMs) are critical for conservation and biosecurity decisions in freshwater systems, particularly where field surveys are costly. While ensemble SDMs are standard for generating prediction intervals, their reliability depends on the assumption that across-replicate variance captures systematic error. This paper identifies a critical failure mode: when training data are contaminated with low-accuracy occurrence records (common in citizen-science datasets like GBIF), the resulting systematic bias is shared across all ensemble members. Consequently, the ensemble replicates do not disagree about the error, leading to narrow, overconfident prediction intervals that fail to capture the true uncertainty.

The central question is whether this calibration failure is spatially uniform or concentrated in specific ecological subregions. The authors hypothesize that failure concentrates at stream-network headwaters. In dendritic freshwater systems, environmental predictors are often aggregated from upstream catchments. Headwater segments, by definition, lack upstream catchments, rendering upstream-aggregated predictors structurally undefined. The authors posit that contamination-induced bias is exacerbated in these topological "edges" where the predictor space is qualitatively distinct.

Methodology
The study utilizes a panel of eight European freshwater crayfish entities (including native species Astacus astacus, Austropotamobius torrentium, and invasive species Pacifastacus leniusculus, Faxonius limosus). The analysis focuses on four entities with substantial headwater representation (35–45% of accessible pixels).

1. Data and Contamination Protocol: The authors used pre-generated prediction surfaces from a deterministic benchmark (clean data) and applied a contamination protocol injecting low-accuracy records at 3%, 10%, and 20% levels. Two ensemble protocols were tested:
   • Protocol A (Replicate Ensembles): 30 bootstrap replicates of Random Forest and XGBoost.
   • Protocol B (Consensus Ensembles): A consensus of four algorithms (GLM, GAM, RF, XGBoost).
2. Headwater Identification: Headwaters were defined structurally by the absence of upstream-aggregated predictors (missingness in upstream features), rather than by topological order alone.
3. Calibration Analysis:
   • Error Decomposition: Per-pixel bias (difference between ensemble mean and benchmark) and interval width were calculated to distinguish between prediction shift and variance underestimation.
   • Conformal Calibration: The study applied split conformal prediction with Leave-One-Basin-Out (LOBO) spatial folds to correct coverage. Two variants were compared:
     • Standard LOBO: Computes a single pooled calibration quantile for all pixels.
     • Mondrian LOBO: Computes separate calibration quantiles for headwater and non-headwater populations (group-conditional calibration).
4. Robustness Checks: The authors controlled for feature-space sparsity using local density estimates and k-nearest neighbor distances to ensure the headwater effect was not merely an artifact of undersampled regions.

Key Results

1. Spatial Concentration of Failure: Calibration failure is strongly spatially structured. As contamination severity increases, empirical coverage drops significantly more in headwaters than in non-headwater segments. For example, at 20% contamination with XGBoost, headwater coverage fell to 0.467, while non-headwater coverage remained at 0.663.
2. Mechanism of Failure: The failure is driven by upward prediction bias, not by intervals failing to widen. Contaminated ensembles systematically overpredict suitability in headwaters. Because this bias is shared across all replicates (due to shared training data), the ensemble variance remains low, and the intervals do not widen to reflect the error.
3. Limitations of Standard Calibration: Standard LOBO conformal calibration restores marginal (panel-wide) coverage to near-nominal levels (approx. 95%) but systematically undercovers headwaters (remaining below 90%). This occurs because the pooled calibration quantile is dominated by the larger, lower-error non-headwater population, leaving the high-error headwater population under-corrected. Furthermore, standard calibration misallocates width, inflating intervals excessively for non-headwaters while insufficiently correcting headwaters.
4. Efficacy of Mondrian Calibration: Group-conditional (Mondrian) LOBO calibration, which stratifies by headwater status, successfully restores nominal coverage for both populations simultaneously. It reallocates the "width-inflation budget," increasing interval width for headwaters (where needed) and decreasing it for non-headwaters. This approach resulted in narrower total intervals overall while achieving honest coverage.
5. Robustness: The headwater concentration of error persists even when controlling for local feature-space density, confirming the failure is due to the structural distinctness of the headwater predictor space, not sparsity. The pattern replicated across both ensemble protocols (replicate and consensus) and multiple species.

Significance and Claims
The paper claims to provide the first empirical characterization of the spatial structure of ensemble SDM miscalibration under realistic data contamination. Its primary contribution is demonstrating that standard conformal calibration, while effective for marginal coverage, leaves a "blind spot" at the topological edges of stream networks where ecological stakes are highest.

The authors argue that this blind spot is conservation-relevant because headwaters serve as thermal refugia for threatened native crayfish and are the front lines for invasive species spread. A model that silently overpredicts suitability in these areas misdirects management efforts. The study concludes that for SDMs operating on dendritic networks with upstream-aggregated predictors, network-position-stratified (Mondrian) calibration should be the default, as it repairs conditional miscoverage without sacrificing marginal performance or incurring extra computational costs. The authors remain modest regarding generalizability, noting that while the mechanism is likely general to network-aggregated predictors, the empirical magnitude is system-specific and requires validation in other taxa or terrestrial systems with analogous topological edges.