🤖 machine learning

OATS: Online Data Augmentation for Time Series Foundation Models

The paper proposes OATS, a principled online data augmentation framework that dynamically generates high-quality synthetic time series data tailored to specific training stages using diffusion models and an explore-exploit mechanism, significantly outperforming static augmentation baselines in enhancing Time Series Foundation Models.

Original authors: Junwei Deng, Chang Xu, Jiaqi W. Ma, Ming Jin, Chenghao Liu, Xu Zhang, Li Zhao, Jiang Bian

Published 2026-08-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Junwei Deng, Chang Xu, Jiaqi W. Ma, Ming Jin, Chenghao Liu, Xu Zhang, Li Zhao, Jiang Bian

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a super-smart robot how to predict the future, like guessing tomorrow's weather or next month's electricity usage. To do this, the robot needs to study massive amounts of history, but real-world data is messy. It has holes, it's inconsistent, and sometimes there just isn't enough of it. In the world of computer science, this is called "Time Series," and the robots we build to understand it are called "Foundation Models." To help these robots learn better, scientists often use a trick called "Data Augmentation." Think of this like a cooking class where the teacher doesn't just give you the original recipe; they also hand you a bunch of fake, made-up ingredients that look and taste just like the real thing, so you can practice more.

For a long time, the way scientists made these fake ingredients was a bit like using a cookie cutter. They would take a real piece of data and apply a simple, static rule to it—like shaking it a little bit (adding noise) or mixing it with another piece (blending). It was a one-size-fits-all approach: the same cookie cutter was used for every single step of the robot's training, no matter how much the robot had already learned. But what if the robot needs different kinds of practice at different times? What if the "best" fake data changes as the robot gets smarter? This paper asks a big question: Can we stop using the cookie cutter and start baking fresh, custom-made practice data for every single moment of the robot's training?

The authors of this paper say "yes" and introduce a new method called OATS (Online Data Augmentation for Time Series Foundation Models). Instead of using a static rule, OATS acts like a highly attentive coach who watches the robot learn in real-time. Here is how it works:

First, the coach needs to know which practice problems are actually helpful. OATS uses a clever scoring system called "Time-Series Influence Scores." Imagine the robot is taking a test. The coach looks at every single practice problem and asks, "If I make the robot solve this specific problem right now, will it get better at the final exam?" If a problem helps the robot improve, it gets a high score. These high-scoring problems become the "guiding signals."

Next, the coach uses these signals to bake new, synthetic data. Instead of just copying and pasting, OATS uses a sophisticated tool called a diffusion model. Think of this as a magical artist. The coach hands the artist the "best" practice problems (the guiding signals) and says, "Create something new that feels exactly like these, but is unique." The artist then generates realistic, brand-new time series data that fits perfectly with what the robot needs to learn at that exact moment.

However, checking every single practice problem to see if it's helpful takes a lot of time and energy. To solve this, OATS uses a strategy called Explore-Exploit.

Explore: Sometimes, the coach stops and checks a fresh batch of problems to see if there are any new, hidden gems they missed. This is thorough but slow.
Exploit: Other times, the coach remembers the high-scoring problems they found earlier and uses those to generate new data immediately. This is fast and efficient.
OATS smartly switches between these two modes, balancing the need to find new information with the need to save time.

The paper tested this idea on six different real-world datasets, including electricity usage and weather patterns, using two different types of robot architectures. The results showed that OATS consistently performed better than both the standard training method (no extra data) and the old "cookie cutter" methods (like TSMixup or Jitter). In fact, OATS helped the robots learn faster and make more accurate predictions. The authors found that while checking every single problem (always "exploring") was the most thorough, it wasn't always the best; mixing in the "exploit" steps (using cached scores) saved a huge amount of computing power without hurting the results.

In short, OATS suggests that the best way to train these powerful time-series robots isn't to throw random fake data at them, but to carefully curate and generate high-quality, custom-made practice data that evolves alongside the robot's learning journey. It turns the training process from a static drill into a dynamic, responsive conversation between the data and the model.

Technical Summary: OATS (Online Data Augmentation for Time Series Foundation Models)

Problem Statement

Time Series Foundation Models (TSFMs) have advanced the field by enabling cross-domain learning and zero-shot generalization through large-scale datasets. However, their efficacy is constrained by data quality issues prevalent in time series, such as missing values, heterogeneous sampling rates, imbalanced domain distributions, and data duplication. While synthetic data augmentation is a widely adopted strategy to mitigate data scarcity and enhance robustness, existing methods suffer from two primary limitations:

Heuristic Reliance: Current approaches depend on manually designed patterns (e.g., sinusoidal waves, decomposed series) or static transformations (e.g., jittering, TSMixup). These heuristics are often task-agnostic and may fail to generalize across different time series tasks.
Static Paradigm: Existing augmentation strategies generate synthetic data once and incorporate it uniformly throughout the entire training process. This ignores the dynamic nature of model training, where the value and utility of specific data samples vary across different training stages.

Methodology: OATS

The authors propose OATS (Online Data Augmentation for Time Series Foundation Models), a principled strategy that dynamically generates synthetic data tailored to specific training steps. OATS operates through three core modules:

1. Time-Series Influence Scores (TSIS)

To address the lack of principled quality metrics for synthetic data, OATS utilizes data attribution to quantify the influence of individual training samples on model performance.

Mechanism: For a given training step $t$ with model parameters $w_t$ , the TSIS function $F_{D_{ref}}$ estimates the data attribution score of a sample $z_i$ with respect to a reference dataset $D_{ref}$ .
Formulation: The score is derived from a first-order Taylor approximation of the change in reference loss. It combines the influence score (gradient alignment between the sample and the reference set) with time-series-specific quality indicators, specifically a Signal-to-Noise Ratio (SNR) threshold.
Output: Samples with high TSIS scores are identified as "valuable" and serve as guiding signals for generation.

2. High-Quality Guided Data Generation

Rather than simply oversampling high-scoring instances, OATS uses them as conditioning signals to generate diverse, realistic synthetic data.

Architecture: A diffusion-based model is employed, conditioned on the selected high-quality samples ( $H_t$ ).
Conditioning: The model incorporates a Time-Series Semantic Prototype module and class-conditional guidance based on subset metadata (e.g., domain or source). The generation process models the conditional distribution $p(\hat{z}|H_t)$ , synthesizing new samples that capture diverse temporal patterns while adhering to the characteristics of the high-quality guides.

3. Explore-Exploit Mechanism

Calculating TSIS for the entire dataset at every step is computationally prohibitive. OATS introduces an Explore-Exploit mechanism to balance computational efficiency with data quality.

Explore Steps: The system stochastically samples new batches to calculate fresh TSIS values, updating a dynamic cache ( $\Phi_{D_l}$ ) for each data subset via an exponentially moving average. This allows the system to adapt to changing model dynamics.
Exploit Steps: The system leverages the cached TSIS values to sample high-quality batches directly, bypassing the expensive recalculation of influence scores.
Control: An $\epsilon$ -greedy strategy determines whether a step is an "explore" or "exploit" step, balancing the discovery of new high-value data against the efficiency of using known high-value data.

Key Contributions

Online Data Augmentation Paradigm: The paper proposes a shift from static to online augmentation, where synthetic data is generated dynamically based on the current state of the training process.
Principled Strategy (OATS): A novel framework integrating data attribution (TSIS) with diffusion-based generation. It uses high-influence samples as guiding signals to produce synthetic data that is both high-quality and diverse.
Efficiency Mechanism: The introduction of an explore-exploit mechanism that significantly reduces computational overhead while maintaining the ability to adapt to training dynamics.
Empirical Validation: Comprehensive evaluation across six datasets (ETTm1, ETTm2, ETTh1, ETTh2, Weather, Electricity) and two TSFM architectures (Encoder-only and Decoder-only Transformers).

Experimental Results

The authors evaluated OATS against regular training and two baseline augmentation methods: TSMixup (weighted summation of existing samples) and Jitter (adding Gaussian noise).

Performance: OATS consistently outperformed both regular training and baseline augmentation methods across nearly all datasets and metrics (Normalized Mean Absolute Percentage Error - MAPE, and Negative Log-Likelihood - NLL).
Robustness: OATS achieved the best results on five of the six datasets (ETTm1, ETTm2, ETTh2, Weather, Electricity) for both model architectures.
Convergence: Test loss curves indicated that OATS achieved faster reduction in NLL compared to baselines.
Hyperparameter Sensitivity: Experiments varying the explore-exploit ratio ( $\epsilon$ ) showed that a setting of $\epsilon=1$ (pure exploration) was not always optimal, suggesting that leveraging cached scores (exploitation) is crucial for efficiency. However, OATS outperformed regular training regardless of the $\epsilon$ setting.

Significance and Claims

The paper claims that OATS offers a principled approach to dynamic data augmentation, moving beyond the limitations of static, heuristic-driven methods. By quantifying data quality through influence scores and adapting generation to the training stage, OATS addresses the critical challenge of data scarcity and quality in TSFMs. The authors assert that this method not only improves model performance but also provides a scalable framework for training foundation models on complex, real-world time series data. The code is made available to facilitate further research in this domain.