🤖 machine learning

Olivia: Harmonizing Time Series Foundation Models with Power Spectral Density

This paper introduces Olivia, a novel time series foundation model that leverages a Harmonizer module and HarmonicAttention mechanism to align datasets via normalized power spectral density, achieving state-of-the-art performance across zero-shot, few-shot, and full-shot forecasting scenarios.

Original authors: Jingru Fei, Kun Yi, Alex Xing Wang, Qingsong Wen, Xiangxiang Zhu, Wei Fan

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jingru Fei, Kun Yi, Alex Xing Wang, Qingsong Wen, Xiangxiang Zhu, Wei Fan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a single, super-smart chef how to cook dishes from every culture on Earth: Italian pasta, Japanese sushi, Mexican tacos, and Ethiopian injera.

The problem? Each cuisine has its own rhythm, ingredients, and cooking style. If you just throw all these recipes into a giant pot and say, "Learn everything," the chef gets confused. The sizzling of the wok clashes with the simmering of the soup. The result is a chef who is okay at everything but great at nothing, or one who gets stuck trying to force a sushi roll to taste like spaghetti.

This is exactly the challenge Olivia, a new AI model for time series data, solves. Time series data is just a record of things changing over time (like stock prices, weather, or traffic). Different sources (domains) have very different "rhythms."

Here is how the paper explains the solution, using simple analogies:

1. The Problem: A Clash of Rhythms

When AI models try to learn from many different datasets at once, they struggle because the "beat" of the data is different everywhere.

Energy data might have a strong daily rhythm (people wake up, use power, sleep).
Traffic data might have a weekly rhythm (rush hour on weekdays, quiet on weekends).
Health data might have irregular spikes.

Trying to learn all these at once is like trying to listen to a jazz band, a marching band, and a choir all playing at the same time. The AI gets lost in the noise.

2. The Secret Ingredient: The "Spectral Fingerprint" (PSD)

The authors realized that every time series has a hidden "fingerprint" called Power Spectral Density (PSD).

The Analogy: Imagine you have a smoothie. You can't see the individual fruits inside, but if you run it through a special machine, it tells you exactly how much strawberry, banana, and mango is in the mix.
The Science: PSD does this for time. It breaks the data down to show how much "energy" is in fast changes (high frequency) versus slow changes (low frequency). It reveals the underlying rhythm of the data, ignoring the specific details like whether the numbers are big or small.

3. The Solution: The "Harmonizer"

The paper introduces a special module called the Harmonizer.

The Analogy: Think of the Harmonizer as a universal translator for rhythms. Before the AI tries to learn, the Harmonizer takes the chaotic jazz, marching, and choir music and re-tunes them all to the same key.
How it works: It doesn't change the actual notes (the data values); it just rearranges the timing so that the "beat" of the energy data matches the "beat" of the traffic data. It forces all these different datasets to speak the same rhythmic language.
The Result: Once the data is "harmonized," the AI can finally see the common patterns underneath. It's like the chef realizing that, despite the different cuisines, they all rely on the same basic principles of heat and timing.

4. The Engine: "Harmonic Attention"

Once the data is harmonized, the AI needs a way to process it efficiently. The paper introduces Harmonic Attention.

The Analogy: Imagine a crowded room where everyone is trying to talk to everyone else at once. It's chaotic and slow (this is how standard AI works).
The Innovation: Instead of letting everyone shout at everyone, the Harmonic Attention puts a few "Resonators" (like megaphones or conductors) in the room. Everyone talks to the conductors, and the conductors talk to each other.
Why it helps: This is much faster and clearer. Because the data was already harmonized by the Harmonizer, these few conductors can capture the most important information without needing to listen to every single whisper in the room.

5. The Result: Olivia

The final model, named Olivia, combines the Harmonizer and the Harmonic Attention.

The Test: The researchers tested Olivia on massive amounts of data from different worlds (weather, electricity, traffic, health).
The Outcome: Olivia became a master chef. It could look at a new dataset it had never seen before (Zero-Shot) and predict the future with high accuracy. It outperformed other big models, even though it was smaller and used fewer resources.
Key Takeaway: By first "tuning" the different data sources to the same rhythm, the model learned to generalize much better. It didn't just memorize the data; it learned the universal language of time.

Summary

The paper claims that by using a mathematical tool (PSD) to align the "rhythms" of different datasets before training, and then using a smart, efficient way to process that aligned data (Harmonic Attention), you can build a time-series AI that is faster, more accurate, and better at handling new, unseen data than previous models.

What the paper does NOT claim:

It does not claim this will cure diseases or predict specific stock market crashes for profit.
It does not claim the model works for every type of data without limits (it notes that very large datasets can sometimes confuse the model if not managed well).
It focuses strictly on the mathematical and architectural improvements to make the AI better at forecasting time-based data.

Technical Summary: Olivia: Harmonizing Time Series Foundation Models with Power Spectral Density

1. Problem Statement

Time series foundation models aim to learn unified, general-purpose representations from diverse datasets across multiple domains (e.g., energy, traffic, health) to enable effective transfer to downstream forecasting tasks. However, the heterogeneity of temporal patterns across these domains presents a significant challenge. Datasets originating from different domains exhibit distinct periodic structures and long-term dependencies due to varying physical dynamics, operational cycles, and measurement settings.

From an optimization perspective, jointly training on datasets with disparate temporal characteristics often leads to slower convergence and suboptimal behavior. From a representation learning perspective, the model struggles to form unified, transferable representations when forced to simultaneously accommodate incompatible temporal structures. While existing approaches attempt to mitigate cross-domain discrepancies through architectural modularization or capacity-based specialization, they often fail to explicitly address the fundamental differences in temporal distributions, particularly in the spectral domain.

2. Methodology

The paper proposes Olivia, a novel time series foundation model that addresses cross-domain heterogeneity by harmonizing datasets via Power Spectral Density (PSD) in the spectral domain. The core hypothesis is that normalizing PSDs can reduce mismatches between datasets and enhance pretraining efficacy.

2.1. The Harmonizer: Bidirectional PSD Reparameterization

Directly minimizing spectral discrepancies (e.g., via Jensen–Shannon divergence) across datasets is computationally intractable for large-scale pretraining due to the mismatch between dataset-level objectives and instance-level optimization. To overcome this, the authors propose the Harmonizer, a structural module that implicitly harmonizes PSDs by reorganizing second-order temporal correlations.

The Harmonizer consists of two bidirectional submodules:

Aligner: Projects raw time series data into a shared, PSD-harmonized latent subspace. It utilizes a learnable orthogonal matrix $Q$ , parameterized via a sequence of Householder reflections ( $Q = \prod H_k$ ), to ensure strict orthogonality. This transformation reorganizes temporal dynamics such that the resulting moment matrix becomes block-diagonal, separating shared subspace dynamics from dataset-specific variations.
Restorer: Performs the inverse isometric mapping to project the processed representations back to the original domain-specific temporal space after decoding, ensuring the output preserves the original temporal structure.

Theoretically, this approach corresponds to a shared reparameterization of second-order temporal correlations, inducing a common structural form across diverse datasets without the instability of direct divergence optimization.

2.2. HarmonicFormer and HarmonicAttention

Building on the harmonized representations, the authors introduce HarmonicFormer, a Transformer-style encoder-decoder backbone. The core innovation is HarmonicAttention, which replaces dense all-to-all token interactions with a low-dimensional interaction space mediated by a compact set of resonators.

Token-to-Resonator Aggregation: Global information is aggregated from tokens into a small set of $M$ latent resonators ( $M \ll L$ , where $L$ is the number of tokens).
Cross-Resonator Interaction: Self-attention is performed within the compact $M \times M$ resonator space rather than the full $L \times L$ token space.
Global Resonator Projection: The interacted resonator information is redistributed back to the tokens.

This design reduces computational complexity from $O(L^2P)$ to $O(LMP + M^2P)$ , enabling linear scalability for long sequences while modeling global dependencies through the harmonized spectral subspace.

2.3. Training Pipeline

Olivia follows a three-stage pipeline:

Pretraining: Trained on large-scale heterogeneous datasets (e.g., UTSD) using a reconstruction objective. The Aligner and Restorer are jointly optimized to learn a shared temporal organization that reduces cross-domain spectral discrepancies.
Tuning: The model adapts to specific forecasting tasks. The Harmonizer's Aligner remains learnable to refine temporal reparameterization, while the encoder and decoder weights are frozen to preserve the pre-trained representations.
Inference: The fully trained model performs forecasting with all parameters frozen.

3. Key Contributions

Principled Harmonization Approach: The paper introduces a method to address pretraining dataset diversity by implicitly harmonizing PSDs through the reorganization of second-order temporal correlations, moving beyond unstructured domain gap treatments.
HarmonicAttention Mechanism: A novel attention mechanism that enables low-dimensional token interactions mediated by a compact set of resonators, facilitating efficient foundational modeling via the HarmonicFormer architecture.
Olivia Model: A new time series foundation model integrating these mechanisms, demonstrating superior performance across zero-shot, few-shot, and full-shot scenarios.

4. Experimental Results

Extensive experiments were conducted on two large-scale benchmarks (TSLib and GIFT-Eval) and six additional datasets from GluonTS.

Zero-Shot Performance: Olivia consistently outperformed state-of-the-art foundation models (including SEMPO, Time-MoE, Timer, Moirai, Chronos, and TimesFM) and task-specific baselines. On the TSLib benchmark, it achieved significant reductions in MSE and MAE compared to large-scale models like Time-MoE (average MSE reduction of 26.3%).
Few-Shot and Full-Shot: Olivia demonstrated strong performance in low-data regimes (few-shot) and full-shot settings, consistently achieving lower average MSE and MAE than both pretrained foundation models and specialized forecasting models (e.g., iTransformer, DLinear).
Efficiency: Despite the added overhead of the Harmonizer, Olivia (5.1M parameters) maintained a compact footprint and inference speeds significantly faster than large-scale models like Time-MoE (113M–2.4B parameters) and Moirai, while outperforming lightweight baselines like SEMPO in accuracy.
Ablation Studies: Removing the Harmonizer or replacing HarmonicAttention with standard Full Attention, Linear Attention, or Nyström Attention resulted in consistent performance degradation, validating the necessity of spectral harmonization and low-dimensional resonator interactions.

5. Significance and Claims

The paper claims that Olivia demonstrates the value of structurally harmonizing temporal dynamics to build scalable and generalizable time series foundation models. By aligning disparate spectral signatures into a unified subspace, the model learns universal temporal primitives that are invariant to domain-specific shifts.

The authors position their work as a principled remedy to the "unstructured domain gap" problem in time series pretraining. They argue that explicitly addressing second-order temporal structures via PSD-guided reparameterization allows for more effective transfer learning than previous methods relying solely on architectural modularization. The results suggest that PSD-consistent representations enable a favorable trade-off between model efficiency, accuracy, and generalization capability across heterogeneous real-world datasets.

The paper acknowledges a limitation regarding the additional inference overhead introduced by the sequential Householder reflections in the Harmonizer, identifying more efficient orthogonal parameterization strategies as a direction for future work. However, it maintains that the current design offers a robust balance between architectural rigor and practical speed.