🤖 machine learning

Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters

This paper presents a large-scale empirical study revealing that synthetic data augmentation for time series forecasting yields architecture-dependent results, significantly benefiting channel-mixing models while degrading channel-independent ones, with optimal gains observed only under specific conditions like gradual annealing and the use of Seasonal-Trend generators.

Original authors: Hugo Cazaux, Eyjólfur Ingi Ásgeirsson, Hlynur Stefánsson

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hugo Cazaux, Eyjólfur Ingi Ásgeirsson, Hlynur Stefánsson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to predict the weather. You have a huge library of real weather reports, but what if you could also generate millions of "fake" weather reports using a computer program to help the robot learn faster? This is the idea behind synthetic data.

This paper asks a simple but tricky question: Does feeding a robot fake weather data actually help it predict the real thing, or does it just confuse it?

The researchers ran a massive experiment (over 4,000 tests) to find out. Here is what they discovered, explained in everyday terms.

The Big Surprise: It Depends on the "Brain"

The most important finding is that synthetic data isn't a magic potion that works for everyone. Whether it helps or hurts depends entirely on how the robot's brain is built.

Think of the different robot brains (architectures) like two types of students:

The "Group Thinkers" (Channel-Mixing Models): These students (like TimesNet and iTransformer) look at all the weather variables (temperature, wind, humidity) together as a connected group. They can see how wind affects temperature.
- Result: When you give these students fake data, they get smarter. They use the extra practice to understand patterns better.
The "Solo Thinkers" (Channel-Independent Models): These students (like DLinear and PatchTST) look at each variable in isolation. They study temperature alone, then wind alone, never connecting the dots.
- Result: When you give these students fake data, they get dumber. The fake data doesn't match the real world perfectly, and because they can't see the connections between variables, the fake data just confuses them.

The Verdict: If you use a "Group Thinker" brain, synthetic data is a great tutor. If you use a "Solo Thinker" brain, synthetic data is a bad teacher that leads them astray.

When is Fake Data Most Useful?

The paper found that synthetic data shines brightest when you are short on real data.

Imagine you are trying to teach a student to predict the weather, but you only have 10% of the usual weather reports.

Without fake data: The student struggles and makes many mistakes.
With fake data: If you use a "Group Thinker" brain, the student can actually perform better than a student who had access to 100% of the real data but no fake data. The extra practice from the fake data fills in the gaps.

However, if you already have plenty of real data, adding fake data usually doesn't help much and might even slightly hurt the "Solo Thinkers."

The "Recipe" for Success

The researchers tested different ways to create and use this fake data. They found three golden rules:

Pick the Right "Flavor" of Fake Data:
They created four types of fake weather patterns:
- Seasonal-Trend: Smooth, predictable patterns (like summer being hot, winter being cold).
- Non-Stationary: Sudden, chaotic changes.
- Long Memory: Slow, lingering effects.
- Volatility: Sudden, sharp spikes (like a stock market crash).
- The Winner: The Seasonal-Trend flavor was the only one that consistently helped. The chaotic and volatile flavors were too different from the real weather data in their tests and confused the robots.
Don't Switch Gears Abruptly:
They tried a teaching method where they started with 100% fake data and suddenly switched to 100% real data halfway through. This was a disaster. It's like teaching a student with a cartoon book and then suddenly handing them a textbook on day 3; the student gets shocked and forgets everything.
The Fix: Use a gradual schedule. Start with some fake data and slowly mix in more real data over time. This "annealing" process lets the robot adjust smoothly.
Don't Overcomplicate the Connection:
Real weather variables are connected (wind and rain go together). The fake data generator tried to mimic this. They found that adding a moderate amount of connection between variables helped, but you don't need to over-engineer it.

What Didn't Work?

Hard Switches: Suddenly switching from fake to real data caused performance to crash by about 24%.
Wrong Brain Types: Using synthetic data with "Solo Thinker" models almost always made them worse.
Too Much Chaos: Using fake data that was too wild or unpredictable (like the "Volatility" or "Non-Stationary" types) didn't match the smooth, seasonal nature of the weather data they tested, so it didn't help.

Summary

If you want to use synthetic data to train a time-series predictor:

Check your model: Make sure it's a "Group Thinker" (like TimesNet or iTransformer). If it's a "Solo Thinker," skip the fake data.
Keep it simple: Use fake data that looks like smooth, seasonal trends.
Mix it slowly: Don't switch from fake to real all at once; blend them gradually.
Save for emergencies: It's most powerful when you don't have enough real data to begin with.

The paper concludes that synthetic data is a powerful tool, but only if you pick the right robot brain and the right recipe. If you get it wrong, it's like giving a student a textbook written in a language they don't speak—it just slows them down.

Technical Summary: Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters

Problem Statement

While synthetic data has revolutionized the training of Large Language Models (LLMs) and computer vision systems, its efficacy in time series forecasting remains poorly understood. Unlike text data, which can be scraped at scale, time series data is often locked behind paywalls or resides on private servers, creating a bottleneck for training foundational models. Although major forecasting models (e.g., Chronos, TimesFM, MOIRAI) have begun incorporating synthetic data, there is no consensus on how to use it effectively. Specifically, it is unclear which generator types transfer well, what quantities are beneficial, whether curriculum scheduling matters, and if synthetic augmentation helps when real data is abundant versus when it is scarce. This paper addresses the gap between the theoretical promise of synthetic data and the empirical reality of its impact on diverse deep learning architectures.

Methodology

The authors conducted a large-scale empirical study comprising 4,218 experimental runs organized into nine distinct experiment groups. The study evaluated five state-of-the-art architectures across seven real-world long-term forecasting datasets (including ETTh1/2, ETTm1/2, Weather, Electricity, and Traffic).

1. Synthetic Data Generator

The core of the methodology is a controllable, difficulty-conditioned synthetic time series generator with three main components:

Bundle System: Defines the temporal archetype of univariate channels. Four bundles were created to represent major real-world patterns:
- Seasonal-Trend (ST): Combines Fourier seasonality, deterministic trends, and autoregressive residuals.
- Non-stationary Regime (NR): Markov-switching AR processes with structural breaks.
- Long Memory (LM): Slowly decaying autocorrelations via ARFIMA or fractional Brownian motion.
- Volatility Events (VE): Financial dynamics modeled via GARCH and Hawkes processes.
Difficulty Conditioning: A scalar $d \in [0, 1]$ modulates complexity (e.g., number of harmonics, trend strength, noise levels).
Cross-Channel Correlation: A latent factor wrapper ( $p_{latent}$ ) transforms independently generated channels to introduce inter-variable correlations, mimicking real multivariate structures.
Variance Allocation: Components are mixed using Dirichlet-distributed weights to ensure controlled contributions to total output variance.

2. Experimental Design

The study systematically varied five factors across the nine groups:

Data Scarcity: Simulated by retaining only a fraction ( $s \in \{0.10, 0.25, 0.50\}$ ) of real training windows.
Augmentation Ratio: The volume of synthetic data added relative to the original dataset size ( $r_{synth} \in \{0.25, 0.50, 1.0, 2.0\}$ ).
Curriculum Scheduling: Tested "hard" (binary switch) vs. "gradual" (linear interpolation) annealing schedules, including synthetic-to-real and real-to-synthetic directions.
Bundle Selection: Isolated the impact of specific bundle types (ST, NR, LM, VE).
Latent Correlation: Varied the probability of applying cross-channel correlation.

3. Models and Metrics

Five architectures were tested: DLinear and PatchTST (channel-independent), SegRNN (RNN-based), and TimesNet and iTransformer (channel-mixing). Performance was measured using Mean Squared Error (MSE) on held-out real test sets for horizons $H=96$ and $H=336$ .

Key Contributions

A Controllable Generator: A novel synthetic time series generator featuring four difficulty-conditioned bundle types and Dirichlet variance allocation, capable of simulating seasonality, regime shifts, long memory, and volatility.
Large-Scale Empirical Evidence: A comprehensive study of 4,218 runs across nine experiment groups, five architectures, and seven datasets, providing the first systematic analysis of synthetic data in time series forecasting.
Architecture-Conditional Findings: The identification of model architecture as the primary determinant of success or failure, offering concrete guidelines for bundle selection, scheduling, and deployment.

Results

1. Architecture is the Deciding Factor

The effect of synthetic data is sharply conditional on the model architecture:

Channel-Mixing Models (Beneficial): TimesNet and iTransformer consistently improved with synthetic augmentation. TimesNet improved in 67% of trials, and iTransformer in 71%. These models benefit from the diverse spectral characteristics and inter-channel diversity provided by synthetic batches.
Channel-Independent Models (Detrimental): DLinear, PatchTST, and SegRNN were consistently degraded by synthetic data. DLinear, for instance, saw a mean MSE degradation of 19.2%. These models process channels in isolation, leading to distribution mismatches without structural compensation.
Aggregate Impact: Across all architectures and configurations, synthetic augmentation hurt performance in 67% of trials.

2. Low-Resource Settings

In scenarios with scarce real data ( $s \le 0.25$ ), synthetic augmentation can yield striking gains for receptive architectures:

Surpassing Full-Data Baselines: TimesNet trained on only 10% of the Weather dataset with synthetic augmentation achieved lower MSE (0.2841) than the full-data real-only baseline (0.2850).
Recovery: For iTransformer, augmentation at 10% sparsity recovered 42.8% of the performance gap between sparse and full-data models.

3. Ablation Insights

Bundle Type: The Seasonal-Trend (ST) bundle was the least harmful (and most beneficial) across benchmarks, aligning with the dominant seasonal-trend dynamics of the test datasets. NR, LM, and VE bundles caused greater degradation due to distribution mismatch (e.g., introducing volatility or regime shifts not present in the target data).
Curriculum Scheduling: Gradual annealing (synthetic $\to$ real) performed similarly to static mixing. However, hard-switch schedules (abruptly switching from synthetic to real data mid-training) were catastrophic, increasing MSE by 23.6% compared to static mixing.
Difficulty Conditioning: Difficulty levels had a minor impact; uniform difficulty performed marginally better than hard distributions.
Latent Correlation: Introducing cross-channel correlation ( $p_{latent}$ ) monotonically reduced degradation, particularly for high-dimensional datasets. A value of $p_{latent} = 0.5$ is suggested as a safe default.
Cache Efficiency: Pre-generating a fixed cache of 500 samples was found to be practically equivalent to on-the-fly generation in terms of performance, while reducing computational overhead.

Significance and Claims

The paper claims to provide concrete, actionable guidelines for integrating synthetic data into time series forecasting pipelines, moving beyond the assumption that "more data is always better."

Architecture Selection: The authors assert that architecture is the strongest predictor of outcome. Synthetic augmentation should be prioritized for channel-mixing architectures (CNNs, multi-variate attention) and avoided for channel-independent models.
Strategic Deployment: The study concludes that synthetic data is most effective in low-resource settings for receptive architectures, where it can match or even exceed full-data performance.
Operational Best Practices: The paper explicitly advises against hard-switch curriculum schedules and recommends the Seasonal-Trend bundle as a safe default for long-term forecasting tasks.
Contextualizing Foundation Models: The authors reconcile their findings with models like Chronos2, noting that the success of synthetic data in pre-training (where models see millions of diverse series) differs from fine-tuning (where models have few epochs to reconcile distribution mismatches). Their results suggest that synthetic data helps specifically when the model architecture is designed to learn from joint channel interactions, a condition met by modern foundational transformers but not by simpler linear or channel-independent models.

The study emphasizes that while synthetic data is not a universal panacea, it is a powerful tool when applied with architectural awareness and appropriate scheduling.