DySCo: Dynamic Semantic Compression for Effective Long-term Time Series Forecasting

The paper proposes DySCo, a universal plug-and-play framework for long-term time series forecasting that combines entropy-guided dynamic sampling, hierarchical frequency-enhanced decomposition, and a cross-scale interaction mixer to effectively filter noise, preserve critical details, and capture complex dependencies while reducing computational costs.

The Gist

Imagine you are trying to predict the weather for next week. You have a massive notebook containing the weather data for the last 10 years.

The Problem:
Intuitively, you might think, "The more history I have, the better my prediction will be!" But in reality, reading 10 years of data is overwhelming. Most of those pages are just boring, repetitive details (like "it was sunny on a Tuesday in 2018") that don't help you predict next week. This "noise" confuses your brain, and you end up making worse guesses than if you had just looked at the last few days.

This is the exact problem computers face with Time Series Forecasting (predicting future trends based on past data). They get "drowned" by too much history.

The Solution: DySCo (Dynamic Semantic Compression)
The authors of this paper created a smart tool called DySCo. Think of DySCo as a super-intelligent editor for your history notebook. Instead of reading every single page, it knows exactly which pages to keep and which to throw away, without losing the important story.

Here is how DySCo works, broken down into three simple steps using analogies:

1. The "Spectral Filter" (HFED): Separating the Music from the Noise

Imagine your history data is a song.

• The Bass (Low Frequency): This is the slow, steady beat—the long-term trends (like the economy growing over 10 years).
• The High Notes (High Frequency): These are the quick, sharp sounds—the sudden spikes, anomalies, or daily fluctuations.

Old methods tried to listen to the whole song at once, which gets messy. DySCo uses a multi-layered filter. It separates the deep, slow bass (the long trends) from the sharp, high notes (the sudden changes). This ensures the computer doesn't confuse a sudden spike with a long-term trend.

2. The "Smart Editor" (EGDS): The Entropy Guide

This is the magic part. Usually, computers just chop off old data because "it's too old." DySCo says, "Wait! Old data might be important!"

It uses a concept called Entropy (a fancy word for "surprise" or "information density").

• Boring Pages: If a chunk of history is just a smooth, predictable line (like a straight road), the editor says, "I don't need to read every inch of this. I'll just summarize it." This compresses the data, saving space.
• Exciting Pages: If a chunk of history has a sudden crash, a weird spike, or a complex pattern (high entropy), the editor says, "Whoa! This is important! I need to keep this in high definition."

The Analogy: Imagine you are summarizing a 500-page novel.

• Old Method: "I'll just read the first 50 pages and the last 50 pages." (You miss the plot twist in the middle).
• DySCo: It reads the boring parts quickly to get the gist, but when it hits the chapter where the hero gets shot, it slows down and reads every word carefully. It keeps the "high-value" moments and compresses the "boring" ones.

3. The "Team Captain" (CSIM): Mixing the Views

After the editor has summarized the history, the computer makes a prediction based on different "views" (some looking at the long trends, some at the short details).

Sometimes the long-term view is right; sometimes the short-term view is right. DySCo uses a Cross-Scale Mixer (a smart team captain) to decide how much to trust each view.

• If the weather is stable, the captain listens to the "Long-Term Trend" expert.
• If a storm is brewing, the captain listens to the "Short-Term Detail" expert.

It doesn't just average them out; it dynamically weighs them to get the best possible answer.

Why is this a big deal?

1. It's Smarter: It doesn't just throw away old data; it understands what is important in that data.
2. It's Faster: By compressing the boring parts, the computer has to do way less math. It's like driving a car with a turbocharger; you go faster using less fuel.
3. It's Plug-and-Play: You can take this "editor" and drop it into almost any existing forecasting model, and it instantly makes that model better at seeing the long-term future without getting confused.

In a nutshell:
DySCo teaches computers how to read a history book efficiently. Instead of trying to memorize every single word of the last 10 years, it learns to skim the boring parts and focus intensely on the dramatic moments that actually matter for predicting the future.

Technical Summary

1. Problem Statement

Time Series Forecasting (TSF) faces a fundamental paradox regarding the lookback window:

• Theoretical Expectation: Extending the lookback window should provide richer historical context, improving the capture of long-term dependencies.
• Practical Reality: Simply increasing the input length often degrades performance. Long sequences introduce information redundancy and noise accumulation, where the correlation between distant past data and current steps weakens.
• Limitations of Current Solutions:
  • Truncation: Blindly cutting off history loses critical long-term periodicities (e.g., weekly traffic patterns).
  • Fixed Heuristics: Existing sampling or compression methods rely on rigid rules (e.g., "older data is less important" or uniform downsampling), which fail to distinguish between valuable high-entropy signals and redundant trends.
  • Computational Cost: Processing extra-long sequences in Transformer-based models leads to quadratic complexity ($O(L^2)$), creating memory bottlenecks.

2. Methodology: The DySCo Framework

The authors propose DySCo (Dynamic Semantic Compression), a universal, plug-and-play module designed to distill long sequences into sparse, high-value representations. It replaces fixed heuristics with a learnable paradigm consisting of three core components:

A. Hierarchical Frequency-Enhanced Decomposition (HFED)

• Goal: To disentangle time series into multi-granularity representations based on the frequency-temporal trade-off.
• Mechanism:
  • The input sequence is decomposed into $N$ hierarchical scales (from short-term local details to long-term global trends).
  • Scale-Specific Filtering: Each scale $i$ is assigned a specific bandwidth $\sigma_i$.
    • Longer horizons use smaller bandwidths (larger kernel sizes) to act as low-pass filters, smoothing out high-frequency noise and extracting clean global trends.
    • Shorter horizons use larger bandwidths to preserve sharp local variations and anomalies.
• Outcome: A pyramid of representations that prevents aliasing and separates high-frequency anomalies from low-frequency patterns.

B. Entropy-Guided Dynamic Sampling (EGDS)

• Goal: To autonomously identify and retain high-entropy (information-rich) segments while compressing redundant trends, regardless of their temporal distance.
• Mechanism:
  • Learnable Scoring: A lightweight MLP-based scorer $G(\cdot)$ evaluates each segment of the decomposed sequence, outputting an importance score $\alpha_j \in [0, 1]$. High $\alpha_j$ indicates complex, unpredictable patterns; low $\alpha_j$ indicates predictable/redundant trends.
  • Dynamic Kernel Calculation: The sampling resolution (pooling kernel size $k_j$) is dynamically adjusted based on a competition between Temporal Bias (distance decay) and Semantic Importance:
    $$k_j = k_{base} \cdot \left(1 + \frac{j}{n}\right) \cdot (1 - \beta \cdot \alpha_j)$$
  • Logic: Older data is naturally penalized (larger kernel/compression). However, if an old segment has a high semantic score (high entropy), the penalty is overridden, forcing the model to use a smaller kernel (dense sampling) to preserve critical details.
• Outcome: A sparse representation that retains critical precursors from the distant past while aggressively compressing redundant history.

C. Cross-Scale Interaction Mixer (CSIM)

• Goal: To fuse predictions from different scales without losing contextual correlation.
• Mechanism:
  • Instead of simple linear summation, a Gating Network dynamically weights the raw predictions from each scale.
  • The weights are computed based on the current context, allowing the model to autonomously decide which scale (global trend vs. local detail) is most reliable for the current prediction step.
• Outcome: A final forecast that benefits from both the stability of long-term trends and the sensitivity of short-term details.

3. Key Contributions

1. Semantic Compression Paradigm: Shifts from fixed, rule-based sampling to a learnable, entropy-guided approach that dynamically prioritizes information density over temporal distance.
2. Universal Plug-and-Play Module: DySCo is designed to be integrated into various mainstream TSF architectures (both Linear and Transformer-based) without requiring architectural overhaul.
3. Computational Efficiency: By compressing the input sequence length from $L$ to a sparse $T$ ($T \ll L$) before feeding it into base predictors, DySCo drastically reduces parameter count and computational complexity.
   • For Linear models: Reduces complexity from $O(L \times O)$ to $O(N \times T \times O)$.
   • For Transformers: Reduces attention complexity from $O(L^2)$ to $O(N \times T^2)$.
4. Theoretical & Empirical Validation: Provides a rigorous analysis of complexity and demonstrates that DySCo can utilize extra-long lookback windows (e.g., $L=2440$) without overfitting, outperforming models tuned to their optimal (shorter) windows.

4. Experimental Results

• Datasets: Evaluated on 7 datasets across Transportation, Energy, and Climate domains.
• Baselines: Compared against PatchTST, iTransformer, TimeMixer, and Linear models.
• Performance:
  • DySCo-integrated models consistently achieved lower MSE (Mean Squared Error) than their vanilla counterparts.
  • Example: On the Electricity dataset, DySCo+TimeMixer achieved an MSE of 0.141 vs. 0.201 for the vanilla version.
  • DySCo with a fixed long window ($L=2440$) outperformed baseline models even when those baselines were tuned to their best-performing (shorter) windows.
• Ablation Studies: Confirmed that removing any component (HFED, EGDS, or CSIM) degraded performance, proving the necessity of the full framework.
• Efficiency:
  • Achieved a ~58.7% reduction in parameters for Linear models.
  • Achieved a ~94.3% reduction in attention-related computations for Transformer models.
  • Significantly reduced GPU memory consumption and training time.

5. Significance

DySCo addresses a critical bottleneck in long-term time series forecasting: the inability to effectively utilize long historical contexts due to noise and computational limits. By introducing dynamic semantic compression, it enables models to:

• Capture Long-Range Dependencies: Effectively utilize extra-long lookback windows to detect rare, long-term periodicities that short windows miss.
• Filter Noise: Automatically distinguish between critical signals and redundant trends.
• Scale Efficiently: Make long-context forecasting feasible for resource-constrained environments and large-scale Transformer models.

The paper concludes that DySCo offers a robust, efficient, and universal solution for next-generation time series forecasting, bridging the gap between theoretical context richness and practical model performance.