Technical Summary: Seasonal Trend Anomaly Detection (STAD)

Problem Statement

Detecting anomalies in time series data is a critical task across domains such as finance, energy, and healthcare. However, existing methods face significant limitations when applied to real-world data characterized by complex structures. Traditional approaches often assume a constant baseline or focus exclusively on point anomalies. While recent methods like CAPA (Collective And Point Anomalies) can detect both point and collective anomalies, they typically operate under the assumption of a constant baseline with Gaussian noise.

In practice, time series frequently exhibit polynomial trends and periodic seasonal patterns. When these components are present, standard decomposition methods (e.g., STL, RobustSTL) often fail to distinguish between genuine anomalies and structural variations. Specifically, collective anomalies (groups of consecutive anomalous points) can be absorbed into the estimated trend or seasonal components, leading to false negatives. Conversely, strong trends and seasonality can obscure anomalies, causing false positives in methods that do not account for them. There is a lack of robust statistical methods capable of simultaneously decomposing a time series into trend, seasonality, and anomaly components while accurately identifying both point and collective anomalies.

Methodology: Seasonal Trend Anomaly Detection (STAD)

The authors propose STAD, a novel approach that extends existing anomaly detection frameworks to handle polynomial trends and seasonal components. The method decomposes the observed time series $Y$ into four components:
$$Y = A + T + S + \varepsilon$$
where $A$ represents anomalies (both collective and point), $T$ is a polynomial trend, $S$ is a periodic seasonal component, and $\varepsilon$ is Gaussian noise.

The STAD algorithm proceeds through four iterative steps:

1. Initial Estimation:

   • Trend: To remove seasonality, the data is differenced with a lag $D$ that is a multiple of the seasonal period $P$. A robust trend estimator is applied to this differenced series to obtain an initial trend estimate $\hat{T}_0$.
   • Seasonality: The initial trend is subtracted from the original data. A robust seasonality estimator is then applied to the remainder to obtain an initial seasonal estimate $\hat{S}_0$.

2. Refined Trend Estimation:

   • The refined trend is estimated by applying a robust trend estimator to the remainder $Y - \hat{S}_0$.
   • Sampling Strategy: A key innovation is the systematic sampling strategy used to handle collective anomalies. The method divides the time series into disjoint segments and samples sub-segments from alternating sets (odd or even indexed segments). This ensures that at least one subsample is free of collective anomalies with high probability, allowing for a reliable estimation of the polynomial trend even in the presence of large anomalous blocks.
   • The method iterates this sampling process $J$ times and selects the trend estimate that minimizes a penalized cost function applied to the anomaly detection step.

3. Refined Seasonality Estimation:

   • Once the trend is refined, the seasonal component is re-estimated by applying the robust seasonality estimator to $Y - \hat{T}$, explicitly excluding the intervals identified as anomalous in the previous step.
   • If the seasonality is assumed smooth, spline smoothing is applied to the estimated seasonal coefficients.

4. Anomaly Detection:

   • Finally, the anomaly component $A$ is estimated by applying a modified version of the CAPA algorithm to the remainder $Y - \hat{T} - \hat{S}$.
   • This step utilizes a penalized cost approach (Generalized Functional Pruning Optimal Partitioning - GFPOP) to identify the number, locations, and magnitudes of both collective and point anomalies.

Key Contributions

• Novel Decomposition Framework: STAD is the first method to simultaneously estimate polynomial trends, seasonal components, and both point and collective anomalies within a unified statistical framework.
• Robust Sampling for Trend Estimation: The paper introduces a systematic sampling procedure that guarantees the existence of non-anomalous subsamples with high probability. This overcomes the limitation of standard robust estimators (like M-estimators) which can be biased by large collective anomalies.
• Theoretical Guarantees: The authors establish statistical theory proving that STAD:
  • Consistently estimates the number of anomalies.
  • Provides asymptotic $L_2$ bounds for the estimation errors of the anomaly, trend, and seasonal components.
  • Ensures that the estimated start and end points of anomalies are close to the true values (within a logarithmic error bound).
• Handling Non-Stationarity: Unlike CAPA, which assumes a constant baseline, STAD accommodates non-stationary data driven by polynomial trends and seasonality.

Results

The performance of STAD was evaluated through extensive simulation studies and a real-world application:

• Simulation Studies:

  • STAD was compared against "oracle" versions of itself (which know the true trend or seasonality) and standard methods.
  • Anomaly Length: STAD performs nearly as well as the oracles for anomalies up to approximately 25% of the time series length. Performance degrades only when anomalies become excessively long, preventing the identification of non-anomalous subsamples.
  • Seasonal Period: The method remains robust across various seasonal periods, provided there are at least five seasonal cycles in the data.
  • Multiple Anomalies: STAD successfully detects multiple collective anomalies with varying lengths and magnitudes, matching oracle performance in most scenarios.
  • Tuning: The study suggests default tuning parameters ($J=20$ for sampling iterations, $\lambda_{coll}=4$, $\lambda_{point}=3$) that offer a good trade-off between accuracy and computational cost.

• Real-World Application:

  • The method was applied to UK electricity export price data (Octopus Agile), which exhibits daily seasonality and long-term trends.
  • Comparison: When applied to this data, CAPA detected a large number of false positives attributed to the seasonal pattern, even with high penalty parameters.
  • STAD Performance: STAD successfully decomposed the data and identified three significant collective anomalies. These anomalies correlated with periods of high renewable energy generation, demonstrating the method's practical utility in distinguishing structural anomalies from typical seasonal fluctuations.

Significance and Claims

The paper claims that STAD addresses a critical gap in time series analysis by enabling the detection of anomalies in the presence of complex, non-stationary structures. The authors emphasize that current methods are ill-suited for data where trends and seasonality are pronounced, often leading to the masking of collective anomalies or the misidentification of seasonal patterns as anomalies.

By establishing rigorous statistical theory and demonstrating strong finite-sample performance, the authors position STAD as a robust tool for domains requiring high-fidelity anomaly detection, such as energy markets and financial monitoring. The work extends the theoretical foundations of the CAPA algorithm to a more general setting, providing a consistent estimator for the number and location of anomalies even when the underlying baseline is not constant. The authors note that while the method is restricted to polynomial trends and stationary seasonal components, it represents a significant step forward in handling the structural challenges of real-world time series data.