📊 statistics

A review of regularised estimation methods and cross-validation in spatiotemporal statistics

This review article examines regularised estimation procedures and cross-validation techniques for geostatistical and spatial econometric models, highlighting their utility in handling big geospatial data through dimensionality reduction, model selection, and the analysis of multivariate spatiotemporal processes.

Original authors: Philipp Otto, Alessandro Fassò, Paolo Maranzano

Published 2026-04-30

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Philipp Otto, Alessandro Fassò, Paolo Maranzano

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand the weather across a whole continent, or perhaps tracking how a disease spreads through a country. You have a massive amount of data: temperature readings from thousands of sensors, income levels for every town, or infection rates for every neighborhood. This is the world of spatiotemporal statistics—data that changes over both space (where you are) and time (when you are).

The problem? The data is so huge and messy that traditional math tools break down. They get stuck in "computational traffic jams" or try to find patterns that aren't really there (overfitting).

This paper is a guidebook for a set of tools called Regularised Estimation. Think of these tools as a "smart filter" or a "digital pruning shears" that helps statisticians cut through the noise to find the real signal.

Here is a breakdown of the paper's main ideas using everyday analogies:

1. The Problem: The "Too Many Ingredients" Soup

Imagine you are a chef trying to recreate a complex soup. You have 1,000 possible ingredients (variables) and a recipe that changes every hour (time) and in every kitchen (space).

The Challenge: If you try to use all 1,000 ingredients, the pot explodes (computational complexity), and the soup tastes like nothing (overfitting). You don't know which ingredients actually matter.
The Solution (Regularisation): This is like having a strict rule: "You can only use the top 10 ingredients." Regularisation forces the model to ignore the useless ingredients and focus only on the ones that truly affect the taste.

2. The Two Main Ways to Cook (The Models)

The paper discusses two main ways statisticians model this data, which are like two different cooking styles:

Geostatistics (The Smooth Blanket):
- Analogy: Imagine a smooth, stretchy blanket covering the whole continent. If you pull the blanket up at one point, the whole blanket moves slightly.
- How it works: This method assumes that things close to each other are similar. It uses a "distance rule" (if two sensors are 1 mile apart, their data is very similar; if they are 100 miles apart, they are less similar).
- The Regularisation Twist: Sometimes the blanket is too thick or has too many wrinkles. Regularisation helps "smooth out" the blanket by finding the simplest version that still fits the data, or by figuring out which parts of the blanket are actually independent (not connected).
Spatial Autoregression (The Neighborhood Chat):
- Analogy: Imagine a row of houses. The temperature in your house isn't just about the weather; it's also influenced by your immediate neighbors. If your neighbor turns on their AC, your house might get cooler.
- How it works: This method explicitly links a location to its neighbors using a "Weight Matrix" (a map of who talks to whom).
- The Regularisation Twist: Usually, we assume we know exactly who the neighbors are. But what if we don't? Regularisation acts like a detective, trying to figure out the actual neighbor connections by testing different maps and keeping only the ones that make sense, while ignoring the fake connections.

3. The "Pruning" Tools (LASSO and Shrinkage)

The paper focuses heavily on a specific technique called LASSO (Least Absolute Shrinkage and Selection Operator).

The Metaphor: Imagine a garden with 1,000 plants. You want to keep only the 50 that are healthy.
How it works: LASSO applies a "penalty" to every plant. If a plant (a variable) isn't contributing much to the garden's beauty, the penalty forces it to shrink until it disappears completely (becomes zero).
The Result: You are left with a clean, manageable garden of only the most important plants. This helps in Variable Selection (picking the right ingredients) and Dimension Reduction (making the math faster).

4. The Danger of "Cheating" (Cross-Validation)

To know if your pruning shears are working, you need to test them. You can't just look at the plants you already cut; you need to see how well the model predicts new plants. This is called Cross-Validation.

The Trap: In normal data, you can pick random plants to test. But in space and time, plants are connected. If you test a plant right next to one you used for training, you are cheating. It's like taking a test where you peek at your neighbor's answers because you both live in the same house.
The Fix: The paper explains that you must use Block Cross-Validation.
- Time: Don't test tomorrow's weather using today's data if you are trying to predict next week. You must leave a "buffer zone" of time between training and testing.
- Space: Don't test a city using data from the city right next door. You must leave a "buffer zone" of empty space between your training map and your test map.
- The Goal: This ensures the model is actually learning the rules of the world, not just memorizing the neighborhood gossip.

5. The Future: Making the Map Itself

One of the most interesting parts of the paper is a suggestion for the future. Usually, we assume we know the "neighborhood map" (the weight matrix) beforehand.

The Idea: What if we use these pruning tools to draw the map for us?
The Analogy: Instead of assuming a specific road connects two towns, we let the data tell us which roads exist. If the data shows no connection, the tool "prunes" that road away. This helps us discover hidden relationships in the data that we didn't know existed.

Summary

This paper is a review of smart filters for massive, messy maps of data. It teaches us how to:

Cut out the noise (remove useless variables).
Simplify the math (make big datasets manageable).
Test fairly (avoid cheating by using proper "buffer zones" in time and space).

The authors argue that while deep learning (AI) is powerful, it is often a "black box" we can't understand. These regularised methods are like a clear window: they give us powerful predictions while still letting us see why the model made those predictions, which is crucial for science, economics, and policy.

1. Problem Statement

The analysis of geospatial and spatiotemporal data has become increasingly complex due to the explosion of "big geospatial data" (e.g., remote sensing, LiDAR, crowd-sourced data). This growth presents two primary statistical challenges:

Computational Complexity: As the number of spatial locations ( $N$ ) and variables ( $q$ ) increases, the dimensionality of the observation covariance matrix grows to $qNT \times qNT$ . Inverting this matrix has a cubic computational cost ( $O(N^3)$ ), rendering standard maximum likelihood estimation infeasible for large datasets.
Model Selection and Interpretability: Traditional step-wise selection methods are computationally inefficient and struggle with high-dimensional settings. Furthermore, deep learning models, while powerful, often lack interpretability ("black-box" nature), whereas geostatistical and spatial econometric models require interpretable structures to understand spatial dependencies and variable relationships.

The paper addresses the need for regularised estimation procedures to handle dimensionality reduction, variable selection, and the estimation of complex spatial dependence structures while maintaining computational efficiency and model interpretability.

2. Methodological Framework

The authors structure their review around a mixed-effects spatiotemporal model framework:
$Y_t(s) = \mu_t(s) + \omega_t(s) + \varepsilon_t(s)$
Where:

$\mu_t(s)$ : Fixed-effects component (mean behavior, exogenous variables).
$\omega_t(s)$ : Random-effects component (spatial/temporal dependence, heterogeneity).
$\varepsilon_t(s)$ : Stochastic error (white noise).

The review categorizes regularisation techniques based on which component they target and the type of spatial model used:

A. Geostatistical Models (Continuous Space)

These models use covariance functions based on distance (e.g., Euclidean, great-circle).

Fixed-Effects Regularisation: Focuses on variable selection (predictor selection) using LASSO (Least Absolute Shrinkage and Selection Operator) and its extensions (Adaptive LASSO, Group LASSO).
- Challenge: Standard LASSO fails with correlated predictors. Adaptive LASSO is highlighted as the solution to ensure selection consistency even with cross-correlated covariates.
- Techniques: Penalised Maximum Likelihood Estimation (PMLE), one-step sparse estimators, and covariance tapering to approximate likelihoods.
Random-Effects Regularisation: Focuses on the covariance/precision matrix structure.
- Sparse Precision Matrices: Using graphical LASSO to induce zeros in the inverse covariance matrix, implying conditional independence between distant locations.
- Low-Rank Approximations: Using penalised estimation to select a subset of local basis functions (e.g., via Cholesky decomposition) to reduce the rank of the covariance matrix, thereby lowering computational cost.

B. Spatial Autoregression Models (Discrete/Lattice Data)

These models (SAR, CAR) explicitly correlate the response variable with adjacent observations via a weight matrix $W$ .

Mean Model Regularisation: Applying LASSO to select relevant regressors in the presence of spatial lags.
Structure Regularisation:
- Weight Matrix Selection: Instead of assuming a fixed $W$ , regularisation is used to select the optimal weight matrix from a set of candidates (e.g., directional weights) or to estimate the full matrix $W$ (often requiring structural constraints like triangularity or block-diagonality for identifiability).
- Joint Estimation: Methods that simultaneously estimate regression coefficients, autoregressive parameters, and the spatial weight matrix.

C. Bayesian Approaches

The paper reviews Bayesian penalisation, which treats the penalty parameter $\lambda$ as a hyperparameter.

Shrinkage Priors: Using priors like the Laplace distribution (for LASSO) or Normal distribution (for Ridge) to shrink coefficients.
Hierarchical Models: Using Integrated Nested Laplace Approximations (INLA) for efficient inference in large datasets.
Model Selection: Techniques like Spike-and-Slab priors and Zellner's $g$ -prior for variable selection in spatiotemporal contexts.

3. Key Contributions: Cross-Validation (CV) under Dependence

A significant portion of the paper is dedicated to Cross-Validation (CV) strategies, arguing that standard random CV fails for spatiotemporal data due to autocorrelation (leading to over-optimistic performance estimates).

The authors propose Target-Oriented CV schemes that respect the data structure:

Temporal CV: Uses blocking (e.g., $k$ -fold blocked subsets, forward validation) to preserve time order and prevent data leakage from future to past.
Spatial CV: Uses spatial buffering or blocking to ensure training and test sets are spatially independent.
- Methods: Spatial Leave-One-Location-Out (SLOO), Spatial $k$ -fold, and buffering based on variogram ranges.
- Concept: Introduction of the "Area of Applicability" (AOA) to define valid prediction zones.
Spatiotemporal CV: Combines spatial and temporal blocking.
- LTO (Leave-Time-Out): Iterates over time blocks.
- LLO (Leave-Location-Out): Iterates over spatial blocks.
- LLTO (Leave-Location-and-Time-Out): Iterates over spatiotemporal cells.
- NNDM (Nearest Neighbour Distance Matching): A variant ensuring the distance distribution between training and test sets matches the prediction scenario.

4. Results and Findings

Efficiency: Regularised methods (PMLE, LASSO) significantly reduce computational burden by inducing sparsity in covariance/precision matrices and selecting relevant variables, making large-scale analysis feasible.
Selection Consistency: Adaptive LASSO and Bayesian shrinkage priors provide consistent model selection even when predictors are highly correlated, a common issue in geospatial data.
Bias-Variance Trade-off: Regularisation introduces bias but reduces variance, leading to better out-of-sample predictive performance compared to unpenalised models, provided the penalty parameter $\lambda$ is tuned correctly.
Importance of CV Strategy: The review demonstrates that ignoring spatiotemporal dependence in CV leads to severe overfitting. Target-oriented schemes (LLO, LTO, LLTO) are essential for obtaining unbiased estimates of predictive accuracy.
Software Ecosystem: The paper catalogs R packages (sperrorest, blockCV, CAST, mlr3spatiotempcv) and MATLAB tools that implement these specific CV and regularisation techniques.

5. Significance and Future Directions

Interpretability vs. Black Boxes: The paper advocates for regularised statistical models over deep learning in scenarios where understanding the underlying spatial mechanisms (e.g., identifying specific drivers of pollution or disease) is crucial.
Bridging the Gap: It provides a unified framework connecting geostatistics (continuous) and spatial econometrics (discrete) through the lens of regularisation.
Future Research:
- Covariance Function Estimation: Developing regularised splines to estimate spatial covariance functions automatically while ensuring positive-definiteness.
- Weight Matrix Estimation: Moving beyond fixed weight matrices in spatial autoregression to fully estimate $W$ using regularisation, potentially using distance-based matrices as shrinkage targets to maintain geographical interpretability.
- Scalability: Further improving computational efficiency for massive datasets (e.g., via scalable resampling or distributed computing).

In conclusion, this review serves as a comprehensive guide for statisticians and data scientists, establishing regularisation and specialized cross-validation as the standard toolkit for analyzing modern, high-dimensional spatiotemporal data.