Technical Summary: GDformer for Multivariate Time Series Anomaly Detection

1. Problem Statement

Multivariate Time Series (MTS) anomaly detection is critical for maintaining the reliability of real-world systems (e.g., industrial sensors, server metrics). However, existing deep learning approaches face significant limitations due to the subsequence isolation strategy. Current methods typically divide long time series into non-overlapping, isolated subsequences to learn intra-subsequence temporal knowledge. This approach suffers from:

• Limited Horizon: Models lack access to series-level context, leading to representations that are not globally informative.
• Inconsistent Criteria: Due to heterogeneity across subsequences (fluctuations, anomaly density), point-wise anomaly scores derived from isolated segments are highly variable. Concatenating these scores often results in false positives and false negatives when establishing a unified detection threshold for the entire series.
• Scalability Issues: Directly applying self-attention to the entire series to capture global context results in $O(n^2)$ time and space complexity, which is computationally prohibitive for long sequences.

The paper frames MTS anomaly detection as an unsupervised learning problem, aiming to derive a compact, unified detection criterion without access to anomaly labels during training.

2. Methodology: GDformer

The authors propose GDformer (Global Dictionary-enhanced Transformer), a model designed to cultivate global representations shared by all normal points across the entire series, moving beyond subsequence isolation.

2.1 Core Architecture

GDformer replaces the standard self-attention mechanism with a Dictionary-Based Cross-Attention mechanism.

• Global Dictionary: Instead of deriving Keys ($K$) and Values ($V$) from the input sequence (which limits context to the subsequence), GDformer introduces learnable, global $K$ and $V$ vectors stored in a dictionary. These vectors are updated iteratively during training to capture the shared temporal representations of normal points across the entire dataset.
• Cross-Attention: The Query ($Q$) is derived from the input temporal tokens, while $K$ and $V$ come from the global dictionary. The attention map ($M$) reflects the correlation weights between a specific time point and the global normal representations.
• Efficiency: By using a dictionary of size $N$ (where $N \ll T$, the sequence length), the complexity is reduced from $O(T^2)$ to $O(TN)$, significantly improving memory and computational efficiency.

2.2 Similarity Evaluation and Prototypes

To derive a robust detection criterion, GDformer introduces Prototypes:

• Prototype Learning: A set of $P$ prototypes is introduced to capture the distribution patterns of the correlation weights between normal points and the global dictionary.
• Similarity Discrepancy: The model calculates the similarity between the attention map of a given point and these prototypes.
  • Normal Points: Their attention weight distributions align closely with the learned prototypes.
  • Anomalous Points: Their distributions deviate significantly from the prototypes.
• Loss Function: The training objective combines a reconstruction loss ($L_c$) and a similarity discrepancy loss ($L_s$). The total loss is formulated as:
  $$L_{total} = L_c - \lambda L_s$$
  where $L_s$ aggregates the similarity scores across all layers. The negative sign encourages the model to maximize the similarity between normal points and prototypes, thereby creating a compact decision boundary.

2.3 Inference

During inference, the anomaly score for a point is derived from the negative similarity to the prototypes. Points with low similarity (high discrepancy) are flagged as anomalies. A unified threshold is applied across the entire series, eliminating the need for subsequence-specific thresholds.

3. Key Contributions

1. Global Dictionary-Enhanced Transformer: The authors propose GDformer, which utilizes a dictionary-based cross-attention mechanism to learn global representations shared by all normal points, effectively overcoming the horizon limitations of subsequence isolation.
2. Prototype-Based Similarity Criterion: The introduction of prototypes to capture normal point-global correlation patterns enables a distinguishable similarity discrepancy between normal and abnormal points, providing a compact and unified detection boundary.
3. Efficiency and Scalability: The method significantly reduces computational complexity and memory footprint compared to standard Transformer-based approaches (e.g., AnomalyTrans, DCdetector) by decoupling the attention mechanism from the sequence length.
4. Transferability: The global dictionary and prototypes demonstrate transferability across different datasets, suggesting shared normal temporal patterns exist across domains.

4. Experimental Results

The authors evaluated GDformer on six popular benchmark datasets: MSL, SMAP, SWaT, PSM, GECCO, and ASD.

• Performance: GDformer achieved state-of-the-art (SOTA) performance, securing the highest F1 scores on five out of six datasets and the second-best on the remaining one. It outperformed strong baselines including AnomalyTrans, DCdetector, MEMTO, and ModernTCN across multiple metrics (Point-Adjustment F1, Range-AUC, and VUS metrics).
• Efficiency: GDformer demonstrated superior efficiency. Compared to AnomalyTrans and DCdetector, it achieved an average training time reduction of 88.8% and 94.7%, respectively, while maintaining a significantly lower memory footprint (e.g., ~4.5 GB vs. ~16 GB for AnomalyTrans).
• Ablation Studies:
  • Replacing self-attention with dictionary-based cross-attention improved F1 scores by ~5.87%.
  • Introducing the similarity-based criterion (vs. reconstruction error alone) improved F1 scores by ~9.11%.
  • Multi-layer combination of similarity loss yielded better results than single-layer approaches.
• Case Studies: Visualizations showed that while baselines (AnomalyTrans, DCdetector) failed to detect specific point anomalies or generated false positives due to subsequence heterogeneity, GDformer consistently detected anomalies using its unified series-level criterion.

5. Significance and Claims

The paper claims that GDformer addresses the fundamental limitation of existing deep learning methods: the reliance on isolated subsequences which prevents the learning of global, series-level knowledge. By shifting the paradigm to global dictionary-enhanced cross-attention, the model ensures a unified detection criterion that is consistent across the entire time series.

The authors emphasize that the similarity-based detection criterion, derived from the discrepancy between point-global correlations and learned prototypes, offers a more compact and effective decision boundary than traditional reconstruction error or statistical divergence methods (like KL/JS divergence). Furthermore, the observed transferability of the global dictionary suggests a pathway toward building foundation models for time series anomaly detection, capable of generalizing across different domains.

The paper concludes that GDformer is both effective and efficient, validating the hypothesis that cultivating global normal representations is superior to isolated subsequence analysis for MTS anomaly detection.