The influence of data gaps and outliers on resilience indicators

This study mathematically demonstrates that data gaps and outliers significantly compromise the reliability of variance- and autocorrelation-based resilience indicators, with missing values weakening their agreement and outliers causing systematic overestimation of system stability.

The Gist

Imagine you are trying to figure out how "sturdy" a house is. If you push it gently, a sturdy house bounces back quickly. A house that is losing its strength (low resilience) will wobble for a long time before settling down. Scientists use this idea to study Earth's systems, like forests or the climate, to see if they are about to collapse into a new, worse state (like a rainforest turning into a desert).

To do this, they use two main "thermometers" to measure stability:

1. The Variance Thermometer: How much the system is shaking or wobbling around.
2. The Memory Thermometer: How much the system's current state depends on its past state (how long it "remembers" a wobble).

The paper argues that scientists often trust these two thermometers to agree with each other. If they both say the system is unstable, we assume the warning is real. However, this study reveals that these two thermometers are actually "glued together" by a hidden factor, and they are easily fooled by bad data.

Here is a simple breakdown of their findings:

1. The "First Step" Glue

The researchers discovered that these two thermometers aren't actually independent. They are mathematically linked in a way that depends heavily on the very first data point of the measurement.

• The Analogy: Imagine you are trying to measure the bounce of a ball. If you drop the ball from a specific height to start your test, that initial height dictates how the math works out for the rest of the test.
• The Finding: Even if the ball behaves perfectly normally afterward, the relationship between your two measurements is mostly determined by that single first drop. If you change that first number, the two thermometers will suddenly agree or disagree, even if the ball's actual stability hasn't changed at all. This means seeing them agree doesn't necessarily prove the system is unstable; it might just mean the starting number was "lucky."

2. The "Missing Puzzle Pieces" Problem

Real-world data (like satellite images of forests) often has holes. Clouds cover the camera, or sensors glitch, leaving "missing values."

• The Analogy: Imagine trying to solve a jigsaw puzzle, but someone has ripped out random pieces. If you try to figure out the picture's stability by looking at the remaining pieces, your calculation gets messy.
• The Finding: When data is missing, the two thermometers stop agreeing with each other. The more missing pieces there are, the less they match.
• The Real-World Twist: This is a big problem for forests. Tropical rainforests are often cloudy, so satellites miss a lot of data there. Deserts are clear, so satellites get perfect data. The study found that in cloudy, high-biomass forests, the two thermometers disagree not because the forest is behaving strangely, but simply because there are too many "missing puzzle pieces" (clouds) confusing the math.

3. The "Spiky" Outlier Problem

Sometimes data has "outliers"—weird, extreme numbers that don't fit the pattern. This could be a sensor glitch, a sudden shadow from a mountain, or a cloud that looks like a forest.

• The Analogy: Imagine a calm lake. Suddenly, someone throws a giant boulder in, creating a massive, fake wave. If you measure the "memory" of the water (how long ripples last), that one giant splash tricks you into thinking the water is very "sticky" or slow to settle, even though the lake is actually calm.
• The Finding: Outliers mess up the "Memory Thermometer" (autocorrelation) specifically. They make the system look like it has a longer memory than it really does.
• The Consequence: This leads to overestimating resilience. The math tells us the system is "sturdy" and will bounce back quickly, when in reality, the data was just corrupted by a glitch. This is dangerous because it might make us think a forest is safe when it's actually on the brink of collapse.

The Bottom Line

The paper concludes that we cannot blindly trust these "early warning" signals.

• The agreement between the two main indicators is often an illusion caused by the first data point.
• Missing data (like clouds) breaks the agreement between the indicators.
• Weird data spikes (outliers) trick us into thinking systems are stronger than they are.

To get a true reading of Earth's stability, scientists need to clean their data much more carefully and understand that these mathematical tools are sensitive to the quality of the data, not just the health of the planet.

Technical Summary

Technical Summary: The Influence of Data Gaps and Outliers on Resilience Indicators

Problem Statement
The resilience of Earth system components, particularly ecosystems, is increasingly threatened by anthropogenic pressures, necessitating reliable early warning signals for abrupt regime shifts. Data-driven resilience indicators based on "critical slowing down" (CSD)—specifically variance ($\lambda_{Var}$) and lag-one autocorrelation ($\lambda_{AC1}$)—are widely used to detect decreasing stability. However, the interpretation of these indicators is hindered by poorly understood statistical interdependencies and their susceptibility to common data imperfections, such as missing values and outliers. Previous studies have noted inconsistencies between $\lambda_{Var}$ and $\lambda_{AC1}$ across different land-cover types (e.g., lower agreement in high-biomass regions), but a formal mathematical explanation for these discrepancies and the mechanisms driving them has been absent.

Methodology
The authors develop a general analytical framework to characterize the statistical dependence between $\lambda_{Var}$ and $\lambda_{AC1}$.

1. Mathematical Derivation: Starting from the Ornstein-Uhlenbeck process discretized as an AR(1) model, the authors derive an exact functional relationship between the two indicators. This derivation reveals that the relationship is not solely determined by system dynamics but is fundamentally sensitive to the initial conditions of the time series, specifically the amplitude of the first data point relative to the total variance.
2. Synthetic Experiments: To isolate the effects of data quality, the authors generate 10,000 synthetic time series from AR(1) processes. They systematically introduce:
   • Missing Values: By randomly removing data points to simulate varying fractions of gaps ($r$).
   • Outliers: By injecting extreme values with controlled magnitudes (quantified by kurtosis) at random positions.
3. Empirical Validation: The framework is applied to global satellite-derived vegetation indices (NDVI, kNDVI, EVI, GPP, LAI) from MODIS products across ten distinct land-cover types. The study correlates the observed agreement between $\lambda_{Var}$ and $\lambda_{AC1}$ with the fraction of missing values and the magnitude of outliers (kurtosis) inherent in each land-cover dataset.

Key Results

• Initial Condition Sensitivity: For gap-free time series, the authors prove that the agreement between $\lambda_{Var}$ and $\lambda_{AC1}$ is largely determined by the relative amplitude of the first data point ($X_1^2 / \text{Var}[X]$). High consistency between indicators can arise purely from initial conditions rather than underlying CSD dynamics, challenging the assumption that such agreement provides independent confirmation of resilience changes.
• Impact of Missing Values: Missing values disrupt the mathematical relationship derived for gap-free series. As the fraction of missing data increases, the correlation between $\lambda_{Var}$ and $\lambda_{AC1}$ significantly weakens. This occurs because missing values affect the estimation of autocorrelation (which requires consecutive pairs) and variance (which uses individual points) differently, creating inconsistencies.
• Impact of Outliers: Outliers introduce systematic biases. While $\lambda_{Var}$ remains relatively stable, $\lambda_{AC1}$ decreases as outlier magnitude increases. This leads to a "biased pattern" where $\lambda_{AC1}$ is consistently lower than $\lambda_{Var}$, resulting in a systematic overestimation of resilience when relying on autocorrelation-based metrics.
• Empirical Correlation: Analysis of MODIS data reveals a strong negative correlation between the fraction of missing values and indicator agreement across land-cover types. High-biomass ecosystems (e.g., evergreen broadleaf forests) exhibit higher missing value fractions due to cloud cover, which explains the previously observed reduced agreement in these regions. Synthetic experiments replicating the specific missing value fractions and outlier magnitudes of different ecosystems successfully reproduce the empirical divergence in indicator agreement.

Significance and Claims
The paper establishes a rigorous mathematical foundation for understanding the statistical dependencies between widely used resilience indicators. Its primary contribution is demonstrating that data quality issues—specifically missing values and outliers—are not merely noise but systematically undermine the consistency and accuracy of resilience assessments.

• Reinterpretation of Previous Findings: The study suggests that the divergence in indicator agreement observed between high- and low-biomass regions is primarily driven by data quality (missing values) rather than inherent ecological differences.
• Caution in Interpretation: The authors caution that high consistency between $\lambda_{Var}$ and $\lambda_{AC1}$ should not be taken as sufficient evidence for the applicability of CSD analysis, as such agreement can be an artifact of initial conditions or data processing.
• Implications for Practice: The findings highlight the urgent need for robust preprocessing strategies and accuracy assessments in disciplines using real-world time series to infer system resilience. The authors note that traditional gap-filling methods may introduce further biases and that emerging AI-based reconstruction methods require careful validation to ensure they preserve underlying system dynamics.

The work bridges a critical gap between theoretical resilience frameworks and their empirical application, emphasizing that reliable resilience assessment is contingent upon continuous, high-quality datasets and a rigorous understanding of how data imperfections propagate into statistical estimators.