📊 statistics

Irregularly and incompletely sampled random fields in the Earth sciences: Analysis and synthesis of parameterized covariance models

This paper presents an asymptotically unbiased spectral maximum-likelihood estimation method that incorporates discrete sampling geometry to analyze irregularly sampled random fields in Earth sciences, demonstrating that growing-domain sampling strategies minimize estimator bias and variance while providing tools to assess model assumptions like stationarity and Gaussianity.

Original authors: Olivia L. Walbert, Frederik J. Simons, Arthur P. Guillaumin, Sofia C. Olhede

Published 2026-04-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Olivia L. Walbert, Frederik J. Simons, Arthur P. Guillaumin, Sofia C. Olhede

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand the weather patterns of an entire continent, but you only have a few scattered thermometers, some placed on mountain tops, some in valleys, and many missing entirely because of clouds or broken equipment. You want to know: How does the temperature change from one spot to another? Is it smooth and gradual, or does it jump around wildly?

This paper is a guidebook for scientists (specifically geophysicists) on how to answer those questions even when their data is messy, incomplete, and irregularly spaced.

Here is the breakdown of their work using simple analogies:

1. The Problem: The "Puzzle with Missing Pieces"

Usually, scientists love data that comes in neat, perfect grids (like a chessboard). But in the real world—whether mapping the ocean floor, the moon's surface, or land temperature—data is rarely perfect.

The Reality: You might have a dense cluster of measurements in one area (like a city) and huge gaps in another (like the ocean). Or, you might have a long, thin strip of data from a ship's path, but nothing in between.
The Risk: If you try to analyze this messy data using standard tools designed for perfect grids, you get the wrong answer. It's like trying to guess the shape of a whole elephant by only looking at a few scattered hairs. You might think the elephant is a fuzzy ball instead of a giant creature with a trunk.

2. The Solution: The "Smart Filter" (Debiased Whittle Likelihood)

The authors developed a new mathematical "filter" (a method called Debiased Whittle Maximum-Likelihood Estimation) that acts like a smart detective.

How it works: Instead of ignoring the gaps or pretending the data is perfect, this method knows exactly where the gaps are. It looks at the "shape" of your data collection (the sampling pattern) and mathematically corrects for the holes.
The Analogy: Imagine you are listening to a song, but there is static and missing beats. A normal listener might get confused and think the song is broken. This new method is like a super-smart audio engineer who knows exactly which beats are missing and can reconstruct the melody perfectly, telling you exactly how confident they are in their reconstruction.

3. The Tool: The "Matérn Swiss Army Knife"

To describe how things change over space (like temperature or elevation), the authors use a mathematical model called the Matérn covariance. Think of this as a "Swiss Army Knife" for describing smoothness.

The Handle (Variance): How big are the swings? (Is the temperature difference between two spots huge or tiny?)
The Blade (Range): How far does the influence stretch? (If it's hot here, how far away does it stay hot?)
The Screwdriver (Smoothness): How jagged is the terrain? (Is it a gentle rolling hill, or a jagged mountain range?)

The paper shows that while scientists often pick just one specific "blade" (a simplified model) because it's easier, the full "Swiss Army Knife" (the general Matérn model) is much better. If you force a simplified model onto complex data, you get biased results—like trying to cut a steak with a butter knife.

4. The Big Discovery: "Grow the Map, Don't Just Zoom In"

One of the most practical findings in the paper is about how to collect data.

The Old Way (Infill): If you want better data, you might think, "I'll just put more sensors right next to the ones I already have." This is like zooming in on a low-resolution photo; you just get a bigger, blurrier version of the same small area.
The New Way (Growing Domain): The authors found that it is much better to expand the area you cover. If you have a small map, don't just add more dots to it; draw a bigger map.
The Analogy: If you want to understand the traffic flow of a whole country, putting 1,000 cameras on one single street corner won't help you. It's better to put 10 cameras on 100 different streets across the country. The paper proves mathematically that spreading your samples out reduces errors and gives you a truer picture of the whole system.

5. Real-World Examples

The authors tested their method on real-world "messy" data:

The Moon: They analyzed the crater Tycho on the Moon. The data came from different satellites with different resolutions. Their method successfully separated the "noise" from the actual geological features, even though the data was a patchwork of different sources.
The Ocean: They looked at the ocean floor, where ships only measure in thin lines (tracks). Their method could predict what the ocean floor looks like in the gaps between the tracks without getting confused by the missing data.
Clouds vs. Land: They analyzed satellite images where clouds blocked the view of the land. Their method could tell the difference between the "smooth" pattern of clouds and the "rough" pattern of the land underneath, even when the data was fragmented.

Summary: Why This Matters

This paper gives scientists a way to stop worrying about "imperfect" data. It provides a rigorous way to say:

Here is the best guess for how the Earth (or Moon) behaves.
Here is exactly how confident we are in that guess, based on where we took the measurements.
Here is how to design future experiments to get the best results (hint: spread out your sensors!).

It turns the frustration of missing data into a solvable math problem, allowing us to understand our planet (and others) more clearly, even when we can't see every single inch of it.

1. Problem Statement

Geophysical data (e.g., bathymetry, lunar topography, satellite reflectance) are rarely collected on perfect, regular grids. Instead, they are often:

Irregularly bounded: Data exists only within specific, non-rectangular regions (e.g., ship tracks, crater interiors).
Incompletely sampled: Data contains "missingness" due to instrument gaps, cloud cover, or voids.
Merged from disparate sources: Datasets often combine direct measurements with indirect models (e.g., satellite altimetry), creating non-stationary fields.

Standard statistical methods often assume regular, full grids. Applying these to irregular data leads to biased estimates of statistical properties (covariance parameters) and incorrect uncertainty quantification. The paper addresses the challenge of characterizing the covariance structure of spatial random fields under these realistic, imperfect sampling conditions while rigorously quantifying the resulting estimator uncertainty.

2. Methodology

The authors extend a spectral-domain maximum-likelihood estimation framework to handle arbitrary sampling geometries.

A. Theoretical Framework: Debiased Whittle Likelihood

The core method is the Debiased Whittle Maximum Likelihood Estimation (DWMLE).

Model: The data is modeled as a realization of a stationary, isotropic, Gaussian process characterized by the Matérn covariance function, defined by three parameters: Variance ( $\sigma^2$ ), Smoothness ( $\nu$ ), and Range ( $\rho$ ).
Blurring the Spectral Density: To account for irregular sampling, the method incorporates the observation window $w(x)$ $w (x)$ (a binary or tapered mask indicating where data exists).
- The theoretical spectral density $S_\theta(k)$ is "blurred" by the sampling window to create an expected periodogram $\bar{S}_\theta(k)$ .
- This blurring is calculated via the Fourier transform of the spatial covariance modified by the autocorrelation of the window $W(y)$ .
Likelihood Function: The estimator maximizes the debiased Whittle likelihood:
$\bar{L}(\theta) = -\frac{1}{K} \sum_k \left[ \ln \bar{S}_\theta(k) + \frac{|H(k)|^2}{\bar{S}_\theta(k)} \right]$
where $|H(k)|^2$ is the window-modified empirical periodogram. This formulation ensures the estimator is asymptotically unbiased even for irregular grids.

B. Uncertainty Quantification

A key innovation is the analytical prediction of estimator covariance.

Instead of relying solely on bootstrapping (which is computationally expensive), the authors derive the covariance of the parameter estimates $\text{cov}(\hat{\theta})$ analytically using the blurred Fisher Information Matrix and the covariance of the blurred score.
This allows researchers to predict the precision of their estimates based only on the sampling geometry and the assumed model parameters, without needing a specific data realization.

C. Diagnostics and Model Selection

Residual Analysis: The ratio of the empirical periodogram to the blurred model spectrum is tested against a $\chi^2_2$ distribution. Deviations indicate model misspecification (e.g., non-stationarity, non-Gaussianity, or anisotropy).
Nested Models: The framework supports fixing the smoothness parameter $\nu$ to specific values (e.g., $\nu=1/2$ for Exponential, $\nu=1$ for Whittle) to test special cases of the Matérn class.

3. Key Contributions

Generalization to Irregular Domains: The paper provides a rigorous mathematical framework to estimate covariance parameters for fields with disjoint support, structured tracks, and random missingness, moving beyond the limitations of rectangular grid assumptions.
Analytical Uncertainty Calculation: It demonstrates that estimator uncertainty can be calculated exactly as a function of the sampling pattern geometry, enabling "design of experiments" for geophysical surveys.
Asymptotic Insights: Through synthetic experiments, the authors compare Growing-Domain (expanding the physical area) vs. Infill (densifying a fixed area) asymptotics. They find that growing-domain strategies generally reduce bias and variance more effectively than infill strategies, particularly for irregular sampling.
Handling Merged Processes: The study shows that analyzing spatially merged processes (e.g., two different geological regimes) as a single stationary field leads to severe bias. However, if the boundaries are known and accounted for via the sampling window, the processes can be estimated separately and accurately.
Software Suite: The authors provide an open-source software suite (MATLAB/Python) for simulation, estimation, and diagnostics of irregularly sampled spatial fields.

4. Results

Synthetic Validation: Extensive simulations (Figs. 6–13) confirm that the DWMLE produces unbiased estimates for $\sigma^2$ , $\nu$ , and $\rho$ even with 33% missing data or irregular boundaries. The analytically predicted uncertainties match empirical results from simulation ensembles.
Impact of Sampling Geometry:
- Random vs. Contiguous: Randomly dispersed missing data generally yields lower parameter uncertainty than contiguous patches of missing data (which introduce strong edge effects).
- Smoothness Bias: Incorrectly fixing the smoothness parameter $\nu$ (e.g., assuming an Exponential model when the true field is smoother) introduces significant bias in the estimates of variance and range. The authors recommend estimating the full 3-parameter Matérn model unless $\nu$ is known with high certainty.
Real Data Applications:
- Lunar Topography (Tycho Crater): Successfully modeled the interior of a crater, distinguishing deterministic rim structures from stochastic background noise.
- Ocean Bathymetry: Analyzed ship-track data, detecting multi-directional geometric anisotropy in residuals that a simple isotropic model could not capture.
- Satellite Reflectance (LANDSAT): Demonstrated how the method handles cloud masking. The "land" dataset showed poor fit (likely due to resolution limits relative to the correlation length), while the "cloud" dataset fit better but still showed deviations from Gaussianity.
Asymptotic Behavior:
- Growing Domain: Increasing the physical size of the grid (keeping spacing constant) consistently reduces bias and variance.
- Infill: Densifying a fixed grid improves the estimation of smoothness ( $\nu$ ) but has diminishing returns for variance ( $\sigma^2$ ) and range ( $\rho$ ), especially if missingness is present.

5. Significance

This paper bridges the gap between theoretical spatial statistics and practical geophysical data analysis.

For Experimental Design: It provides a tool for scientists to optimize sampling plans (e.g., where to place sensors or how to schedule satellite passes) to minimize parameter uncertainty before data collection.
For Data Interpretation: It offers a robust method to quantify how much "trust" to place in statistical parameters derived from messy, real-world data. It explicitly warns against the "common wisdom" that more data always equals better estimates if the sampling geometry is poor (e.g., infill vs. growing domain).
For Model Validity: By providing rigorous goodness-of-fit tests and residual diagnostics, it allows researchers to detect when their assumptions (stationarity, isotropy, Gaussianity) are violated, preventing the propagation of errors into downstream applications like simulation or risk assessment.

In summary, Walbert et al. provide a comprehensive, mathematically rigorous, and computationally practical toolkit for analyzing the statistical properties of Earth and planetary data, acknowledging that "data is a model" shaped by the sampling strategy used to acquire it.