Increasing spatial approximation complexity can degrade prediction quality in distribution models

This study demonstrates that increasing spatial approximation complexity in distribution models does not necessarily improve prediction quality and can actually degrade predictive performance by producing poorly calibrated uncertainty estimates, thereby highlighting the need for practitioners to carefully select appropriate mesh resolution rather than assuming higher complexity always yields better results.

The Gist

Imagine you are trying to draw a map of the ocean floor to find where fish are hiding. You have a bunch of data points from trawlers that have caught fish in specific spots. To make a complete picture, you need to connect the dots.

This paper is about how many dots you need to connect to get the best map, and why using too many dots can actually make your map worse.

Here is the story of the paper, broken down with some everyday analogies:

1. The Old Belief: "More Detail is Always Better"

For a long time, scientists thought that if they wanted to predict where fish were, they should use the most detailed map possible. They thought, "If I use a grid with tiny squares (high resolution), I'll capture every little bump and dip in the ocean, and my predictions will be perfect."

In technical terms, they were using "finer meshes" (more triangles in their computer model) to approximate the data.

2. The Surprise Discovery: The "Goldilocks" Zone

The authors of this paper tested this idea using real data from the West Coast of the US (fish and ocean temperature). They built maps with:

• Coarse grids: Big, chunky triangles (low detail).
• Medium grids: Just right.
• Fine grids: Tiny, intricate triangles (high detail).

The Result?

• On the data they already had (In-Sample): The fine grids looked amazing. They hugged every single data point perfectly. It was like a tailor making a suit that fits a mannequin perfectly because they measured every inch.
• On new, unseen data (Out-of-Sample): The fine grids failed. They were terrible at predicting where fish would be in a new spot.

The Analogy:
Imagine you are trying to memorize a song to sing it at a party.

• Coarse Grid: You only remember the chorus. You can't sing the whole song, but you don't make mistakes.
• Fine Grid: You memorize the song perfectly, including every breath the singer took, every cough, and every background noise. You are a human recording.
• The Problem: When you get to the party and the song changes slightly (a new verse, a different singer), your "perfect" memory fails because you memorized the noise, not the music. You can't adapt.

The paper found that medium-resolution grids were the "Goldilocks" zone. They were detailed enough to catch the real patterns (the music) but smooth enough to ignore the random noise (the coughs).

3. Why Does This Happen? (The "Overfitting" Trap)

The paper explains that when you use a grid that is too fine, the computer model starts to get confused about what is a "real pattern" and what is just "random error."

• The Scenario: Imagine you are measuring the temperature of the ocean. Sometimes your thermometer is slightly off (random error).
• The Fine Grid: The computer thinks, "Oh, this tiny temperature spike isn't an error; it's a real, tiny island of cold water!" It builds a complex shape to explain that one spike.
• The Consequence: The model becomes overconfident. It thinks it knows exactly where the cold water is, but it's actually just reacting to a measurement glitch. When it tries to predict the future, it fails because that "tiny island" wasn't real.

This is called overfitting. The model is so busy trying to explain the noise that it forgets the big picture.

4. What About Fish Counts and Management?

The researchers also looked at how this affects real-world decisions, like counting fish populations to decide how many can be caught (fishing quotas).

• For most fish: The choice of grid didn't matter much. The total number of fish looked roughly the same whether the map was coarse or fine.
• For some specific fish (Rockfish): The grid choice changed the numbers significantly. If you picked the wrong grid, you might think a fish population is booming when it's actually crashing, or vice versa. This could lead to bad fishing rules that hurt the fish or the fishermen.

5. The Takeaway: Don't Just Guess; Test It!

The authors aren't saying "stop using detailed maps." They are saying: Don't assume the most detailed map is the best.

Instead, they suggest a simple recipe for scientists:

1. Try a few different levels of detail (coarse, medium, fine).
2. Test them by hiding some data and seeing which map predicts the hidden spots best.
3. Pick the one that works best, even if it's not the most detailed one.

Summary

Think of spatial modeling like taking a photo.

• Low Resolution: The photo is blurry. You miss the details.
• High Resolution: The photo is so sharp you can see the dust on the lens and the pores on the subject's skin. It looks "real," but it's actually full of distracting noise.
• Just Right: The photo is sharp enough to see the subject clearly, but smooth enough that the background looks nice and the subject stands out.

The main lesson: In science, more complexity doesn't always mean better answers. Sometimes, a slightly simpler, smoother model is actually the most accurate predictor of the future.

Technical Summary

1. Problem Statement

Spatial and spatiotemporal models (e.g., Species Distribution Models or SDMs) are critical for ecological inference and conservation management. A common assumption in the field, particularly when using Stochastic Partial Differential Equation (SPDE) approaches to approximate Gaussian Random Fields (GRF), is that increasing the resolution of the spatial mesh (i.e., using more knots/vertices) universally improves predictive accuracy.

However, this paper challenges that assumption. The authors investigate whether higher spatial approximation complexity (finer meshes) actually degrades the quality of probabilistic predictions, particularly out-of-sample performance, and how these choices affect derived management indices (e.g., biomass trends).

2. Methodology

The study utilizes a combination of real-world case studies and simulation experiments to evaluate the relationship between mesh resolution and model performance.

• Data Sources:

  • West Coast Groundfish Bottom Trawl Survey (WCGBTS): A 21-year dataset (2003–2023) from the California Current ecosystem.
  • Case Study 1 (Temperature): A spatial model of ocean bottom temperature (2018, $n=701$) to test mesh effects on a continuous Gaussian response.
  • Case Study 2 (Species Density): Spatial models for three groundfish species (Sablefish, Petrale Sole, Arrowtooth Flounder) with non-Gaussian responses (Tweedie distribution) and high observation error.
  • Case Study 3 (Biomass Indices): Spatiotemporal models for 27 species to generate area-weighted biomass indices used in stock assessments.
  • Simulation: Synthetic data generated from spatial Gaussian random fields with varying spatial ranges, spatial variances, and observation error levels.

• Modeling Framework:

  • SPDE Approach: Approximates continuous spatial fields using a triangulated mesh and Gaussian Markov Random Fields (GMRF).
  • Software: Primarily sdmTMB (Template Model Builder with SPDE), with validation using INLA/inlabru and mgcv (GAMs).
  • Variable: The primary manipulated variable was mesh resolution, controlled by the "cutoff distance" (the distance below which points are treated as a single vertex). This ranged from coarse (few vertices) to fine (hundreds/thousands of vertices).
  • Evaluation Metrics:
    • Log Predictive Density (LPD): The primary metric for probabilistic prediction quality (out-of-sample).
    • Root Mean Squared Error (RMSE): For simulated data to assess point estimate accuracy.
    • Cross-Validation: 10-fold k-fold cross-validation (repeated 5 times) to assess out-of-sample performance.

3. Key Contributions

• Refutation of the "Finer is Better" Heuristic: The paper provides empirical evidence that increasing mesh resolution does not universally improve predictions. Instead, predictive performance often peaks at intermediate resolutions and declines at the finest scales.
• Mechanism Identification: The authors identify that the degradation in performance is driven by overfitting the latent spatial process, which leads to an underestimation of observation error variance. When the mesh is too fine, the spatial field absorbs variability that should be attributed to observation noise, resulting in overconfident (and thus penalized) probabilistic predictions.
• Impact on Management Indices: The study demonstrates that while biomass trends are often robust to mesh choice, the scale and uncertainty of absolute biomass indices can vary significantly for certain species, potentially impacting stock assessment decisions.
• Practical Guidance: The authors advocate for cross-validation as a mandatory step in model specification to select the optimal mesh complexity for a specific dataset, rather than relying on default rules of thumb.

4. Key Results

A. Predictive Performance (Log Predictive Density)

• In-sample fit: Consistently improved as mesh resolution increased (more vertices = better fit to training data).
• Out-of-sample fit: Followed an inverted-U shape. Performance improved up to an intermediate number of vertices (e.g., ~400 for temperature, ~200–600 for fish species) and then declined as resolution became too fine.
• Simulation Findings: The decline in LPD was most pronounced when:
  1. Spatial range was small (high "wiggliness").
  2. Spatial variance was high.
  3. Observation error was low.
• RMSE vs. LPD: Interestingly, while LPD declined at high resolutions, the RMSE (point prediction accuracy) often continued to improve. This indicates that the model was predicting the mean well but failing to correctly quantify the uncertainty (predictive distribution).

B. Parameter Estimation

• Spatial Parameters: As mesh resolution increased, the estimated spatial range decreased, and spatial variance increased. The model attempted to explain fine-scale variation by shrinking the spatial range.
• Observation Error: At very high resolutions, the estimated observation error variance dropped to near zero, causing the model to become overconfident.
• Fixed Effects: In the temperature case study, increasing mesh complexity altered the estimated coefficients for fixed effects (e.g., day-of-year seasonality), suggesting that overly complex spatial fields can "absorb" covariate effects (spatial confounding).

C. Biomass Indices

• For most of the 27 species, mesh resolution had minimal impact on the trend of biomass estimates.
• However, for several commercially important rockfish species, different mesh resolutions resulted in substantial differences in the magnitude (scale) and uncertainty of the biomass indices.

5. Significance and Implications

• Ecological Inference: The choice of spatial resolution is not just a computational detail; it fundamentally alters ecological inference. Overly complex meshes can mask the true effects of environmental covariates and lead to biased estimates of spatial parameters.
• Conservation Management: Stock assessments rely on biomass indices. If mesh resolution choices arbitrarily alter the scale or uncertainty of these indices, management decisions (e.g., catch limits) could be suboptimal or based on spurious precision.
• Methodological Shift: The paper argues that the community should move away from defaulting to the "finest computationally feasible mesh." Instead, analysts should:
  1. Fit models across a range of mesh complexities.
  2. Use out-of-sample cross-validation (specifically Log Predictive Density) to select the optimal resolution.
  3. Be aware that Penalized Complexity (PC) priors help stabilize parameters but do not fully prevent the degradation of predictive performance at extreme resolutions.

In conclusion, Ward and Anderson demonstrate that spatial model specification is a critical, non-trivial step in ecological modeling. Optimal prediction requires balancing model flexibility with the risk of overfitting the latent spatial process, a balance best found through rigorous cross-validation rather than theoretical assumptions.