Technical Summary: ChronosAD

Problem Statement

Time series anomaly detection is critical across domains such as finance, healthcare, and industry, yet existing methods face significant challenges in generalization. Current approaches are often tailored to specific domains, relying on handcrafted features or domain-specific heuristics that limit scalability. Furthermore, while time series foundation models have emerged, they possess two primary limitations for anomaly detection tasks:

1. Univariate Focus: Most foundation models are designed for univariate settings, failing to explicitly model cross-channel (inter-variable) relationships essential for complex multivariate systems.
2. Forecasting Bias: These models are primarily optimized for signal forecasting, where their performance can be limited in challenging settings, rather than for the distinct task of anomaly detection.

Consequently, there is a need for an architecture that leverages the generalizable representations of foundation models while addressing the specific requirements of multivariate anomaly detection with minimal task-specific tuning.

Methodology: ChronosAD

The paper proposes ChronosAD, a novel two-stage architecture that utilizes a time series foundation model as a zero-shot feature extractor, followed by a custom refinement module.

1. Problem Formulation

The input is a multivariate time series $X \in \mathbb{R}^{C \times L_{total}}$, where $C$ is the number of channels and $L_{total}$ is the total length. The series is segmented into non-overlapping windows of length $l$. The goal is to classify each segment as normal ($0$) or abnormal ($1$) based on a scoring function $f$.

2. Two-Stage Pipeline

Stage 1: Zero-Shot Feature Extraction (Chronos Encoder)

• The model employs Chronos [10], a pre-trained time series foundation model, as a frozen encoder.
• Each univariate channel of the multivariate input is processed independently through the Chronos encoder to generate fixed-dimensional embedding vectors ($E_c$).
• Scaling Factor: Chronos provides a scaling factor $s$ for each instance, calculated as the mean of absolute values within the context window (weighted by an attention mask). This captures the natural scale of the data, aiding generalization across datasets with varying magnitudes.
• Preprocessing: Before entering the second stage, embeddings undergo L2 normalization to standardize magnitudes and SeLU activation to ensure stable gradient flow and mitigate vanishing gradients.

Stage 2: Temporal Refinement (Temporal Block)

• A custom Temporal Block processes the normalized embeddings to capture temporal dependencies and inter-channel interactions.
• BiLSTM: The sequence is processed through Bidirectional Long Short-Term Memory (BiLSTM) networks to encode forward and backward temporal features ($h_c$).
• Multi-Head Attention: The BiLSTM output is refined using a Multi-Head Attention mechanism. Queries ($Q$), Keys ($K$), and Values ($V$) are derived from the hidden states to compute attention weights, highlighting salient patterns.
• Aggregation: The attention outputs are averaged and summed to reduce dimensionality, followed by a Linear Layer to produce a refined embedding ($e_c$).
• Fusion: The scaling factor $s$ is concatenated to the refined embedding. Finally, embeddings from all $C$ channels are concatenated along the feature dimension and passed through a Fully Connected Network (FCN) to produce the final anomaly score.

Training: The model is trained in a supervised setting using a weighted Binary Cross-Entropy (WBCE) loss.

Key Contributions

1. Novel Architecture: ChronosAD introduces a two-stage pipeline that effectively adapts a time series foundation model (Chronos) for anomaly detection, treating it as a zero-shot feature extractor rather than a forecasting engine.
2. Multivariate Adaptation: Unlike standard univariate foundation models, ChronosAD employs a multi-path architecture with channel fusion, explicitly modeling cross-channel interactions alongside temporal dynamics.
3. Custom Temporal Block: The integration of BiLSTM and Multi-Head Attention allows the model to refine foundation model embeddings, capturing both bidirectional temporal dependencies and salient patterns specific to anomaly detection.
4. Minimal Tuning: The approach requires minimal task-specific tuning, leveraging the pre-trained knowledge of the foundation model for robust generalization.

Experimental Results

The authors evaluated ChronosAD on 11 diverse benchmarks, including UCR Archive datasets, real-world industrial (CWRU), medical (MIT-BIH), and cyber-physical (SWaT) datasets. The model was compared against 11 state-of-the-art methods (e.g., AnoGAN, Deep-SVDD, USAD, KalmanAE).

• Performance: ChronosAD outperformed existing methods by an average of 4.72% in AUC and 6.60% in AP.
• Specific Achievements:
  • Achieved near-perfect or perfect scores (AUC $\approx$ 100%) on datasets like TwoPatterns, FordA, and CWRU.
  • Demonstrated robustness on challenging multivariate datasets like SWaT and medical data like MIT-BIH.
• Ablation Study:
  • The Chronos encoder was identified as the most critical component; its removal caused a significant drop in performance (AUC $\downarrow$ 18.34%).
  • L2 normalization and Attention mechanisms were also crucial, with their removal leading to notable performance decreases.
  • Other components (SeLU, BiLSTM, Scaling Factor) provided incremental but measurable improvements in stability and accuracy.
• Latent Space Analysis: t-SNE visualizations showed that the model successfully separates normal and anomalous points into distinct clusters, with clear manifolds even in complex datasets.

Significance and Claims

The paper positions ChronosAD as a preliminary study demonstrating the viability of adapting time series foundation models for downstream anomaly detection tasks.

• Validation of Foundation Models: The results validate that foundation models, when paired with appropriate refinement layers, can serve as powerful, generalizable feature extractors for anomaly detection across diverse domains (industrial, medical, automotive, cyber-physical).
• Supervised Context: The authors explicitly frame the work within a supervised setting to facilitate initial evaluation. They acknowledge that anomaly detection is traditionally unsupervised and note that while labeled data is scarce, emerging technologies like digital twins are making acquisition more feasible.
• Future Path: The authors state that these results serve to validate their approach as a "viable path for a more rigorous future investigation," specifically suggesting that future work will focus on extending the approach to unsupervised settings to enable direct comparisons with the dominant unsupervised baselines.

The paper concludes that ChronosAD paves the way for future research into unsupervised anomaly detection using foundation models, offering a robust alternative to current domain-specific heuristics.