🤖 machine learning

Benchmarking Time Series Generation Methods for Privacy-Preserving Forecasting

This paper introduces a benchmark evaluating synthetic time series generation methods for privacy-preserving forecasting under a Train on Synthetic, Test on Real protocol, revealing that while no method fully replaces original data, simple transformation-based approaches and the proposed Grasynda-P model offer superior trade-offs between forecasting accuracy and privacy protection compared to deep generative models and noise-based anonymization.

Original authors: Luis Amorim, Vitor Cerqueira, Moises Santos, Paulo J. Azevedo, Carlos Soares

Published 2026-08-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Luis Amorim, Vitor Cerqueira, Moises Santos, Paulo J. Azevedo, Carlos Soares

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to predict the future, like guessing how much electricity a city will need next week or how many flu cases will appear next month. To do this, the robot needs to study past data. But here's the catch: that past data often contains private secrets about real people, like their health records or spending habits. You can't just hand the robot the real files; that would be a privacy disaster. So, scientists have come up with a clever workaround: instead of using the real data, they create "synthetic" data. Think of this as making a perfect, fake photocopy of a secret diary. The fake diary looks and feels just like the real one, with all the same patterns and rhythms, but it doesn't belong to any specific person. The goal is to train the robot on these fake diaries so it learns the rules of the world without ever seeing the real secrets.

But there's a big question hanging over this idea: If we only train the robot on the fake data, will it still be smart enough to make good predictions on the real world? And more importantly, is the fake data actually safe? Could a sneaky hacker look at the fake diary and figure out what was written in the real one? For a long time, researchers mostly used synthetic data just to add to the real data, like sprinkling extra sugar on a cake. They hadn't really tested what happens if you replace the entire cake with a synthetic version. This paper steps into that gap, acting like a giant taste-test competition. The researchers set up a "Train on Synthetic, Test on Real" challenge. They took seven different real-world datasets (like energy usage and tourism numbers), created fake versions of them using various methods, trained their prediction models on the fakes, and then tested how well those models could predict the real future. They also measured how "close" the fake data was to the real data to see if it was truly private.

The authors put this challenge to the test with a lineup of different "fake-data makers." Some were simple tricksters that just added a little noise or stretched the time axis, while others were complex deep-learning AI models trying to learn the soul of the data. They even tested a new, improved version of a graph-based generator they call Grasynda-P. This new method is like a master chef who doesn't just copy a recipe but studies the relationships between ingredients (the transitions between states) and then cooks up a new dish that tastes similar but uses different, smoothed-out ingredients so no one can trace it back to the original kitchen.

Here is what they found, and it's a bit of a mixed bag. First, the hard truth: no method can perfectly replace the original data. If you train a model on the fake stuff, it will almost always be slightly worse at predicting the future than if you trained it on the real stuff. It's like trying to learn to play soccer by watching a cartoon; you get the idea, but you won't be as good as someone who practiced on the real field.

Second, they discovered a tricky trade-off. The methods that were best at keeping secrets (the most private) were the worst at helping the robot predict the future. These were the "noise-based" methods, which are like scrambling the letters in a diary so much that the story becomes unreadable. They are very safe, but the robot can't learn anything from them. On the other hand, the methods that were best at helping the robot predict the future were the ones that stayed very close to the original data. But if you stay too close, you aren't really protecting privacy; it's like making a photocopy that is so clear you can still read the original handwriting.

However, there was a hero in the story: Grasynda-P. This new method managed to find a sweet spot. It didn't beat the original real data in prediction power (because nothing does), but it was much better at predicting the future than the super-secure noise methods. At the same time, it was much better at hiding the original data than the simple "stretch and wiggle" methods. The authors found that Grasynda-P sits on the "Pareto frontier," which is a fancy way of saying it's one of the best possible compromises you can make. You can't get better privacy without losing some prediction power, and you can't get better prediction power without losing some privacy, but Grasynda-P is right at the edge of that limit, doing the best job possible for both.

The paper also ruled out some popular ideas. They showed that the fancy, complex deep-learning models (like TimeVAE and TSDiff) didn't actually do a better job than the simpler methods in this specific "fake-only" training scenario. In fact, the simple tricks, like just adding a little jitter or scaling the numbers, often worked better for prediction than the heavy AI models, likely because the complex models got confused when they didn't have the real data to anchor them.

In the end, the researchers suggest that while we can't fully swap real data for fake data without a small price to pay in accuracy, we can get very close. By using graph-based methods like Grasynda-P, which focus on the structure of how data moves over time rather than just copying the numbers, we can create synthetic datasets that are safe enough to share and useful enough to train powerful AI. It's not a magic wand that solves privacy forever, but it's a very promising step toward a future where we can learn from data without exposing the people behind it.

Technical Summary: Benchmarking Synthetic Time Series Generation Methods for Privacy-Preserving Forecasting

Problem Statement

Time series forecasting is critical in privacy-sensitive domains such as healthcare, energy, and finance. However, access to original training data is often restricted due to privacy concerns, necessitating the use of released data (synthetic or anonymized) for model training. While synthetic time series generation has been extensively studied for data augmentation, its efficacy when used as the sole training source (replacing original data entirely) remains underexplored. Furthermore, the trade-off between forecasting utility and the privacy risk of released series is not well characterized. Existing privacy-oriented perturbation methods (e.g., LPA, FPA) often disrupt temporal correlations essential for forecasting, while deep generative models may fail to preserve necessary structural dynamics or offer insufficient privacy guarantees.

Methodology

The authors propose a systematic benchmark evaluating synthetic generation methods and noise-based anonymization baselines under a Train on Synthetic, Test on Real (TSTR) protocol.

Experimental Setup

Datasets: Seven publicly available datasets from the M1, M3, Tourism, and NN3 forecasting competitions, covering monthly and quarterly series across industry, demography, and economics.
Protocol: A forecasting model (NHITS) is trained exclusively on the released dataset $\tilde{D}$ (generated or perturbed) and evaluated on held-out real observations from the original dataset.
Metrics:
- Forecasting Utility: Measured by Mean Absolute Scaled Error (MASE).
- Empirical Privacy Risk: Measured by Distance to Closest Record (DCR) and Nearest Neighbour Distance Ratio (NNDR) between released series and original training series. Higher values indicate better privacy separation.
Methods Evaluated:
- Transformation-based: Jitter, Scaling, Time Warping, Magnitude Warping.
- Pattern Mixing: DBA, SeasonalMBB, TSMixup.
- Deep Generative Models: TimeVAE, TSDiff.
- Noise-based Baselines: Laplace Perturbation Algorithm (LPA) and Fourier Perturbation Algorithm (FPA) with $\epsilon=1.0$ .
- Proposed Method: Grasynda-P, an extension of the graph-based generator Grasynda.

The Grasynda-P Extension

The authors introduce Grasynda-P, which modifies the original Grasynda generator (which uses transition probability matrices over discrete state spaces) with two privacy-motivated changes:

Matrix Ensembling: Instead of deriving a transition matrix from a single series, the method averages the transition matrix of a target series with those of its $r$ most similar neighbors (based on Frobenius distance). This dilutes the influence of any single training record.
Continuous State-Value Sampling: Replaces the uniform sampling of original values from a state's set with Kernel Density Estimation (KDE). This generates continuous values from a smoothed distribution, preventing the exact reproduction of original observations.

Key Contributions

A Comprehensive Benchmark: The paper establishes a reference framework for evaluating synthetic time series generation under realistic privacy constraints (TSTR), jointly assessing forecasting performance and distance-based empirical privacy risk across seven datasets.
Grasynda-P: A novel privacy-motivated extension of the Grasynda generator that improves empirical privacy without compromising forecasting utility.
Characterization of Trade-offs: The study maps the Pareto frontier of the privacy–forecasting trade-off, identifying which methods offer the best balance between utility and privacy.

Results

No Perfect Substitute: No generation method fully substitutes for original training data; training on original data consistently yields the best forecasting performance.
Forecasting Performance: Simple transformation-based methods (e.g., Jitter, Scaling) generally outperform deep generative models (TimeVAE, TSDiff) in the TSTR setting. Grasynda-P achieves competitive forecasting performance, ranking ahead of several other generators.
Privacy Performance: Noise-based anonymization methods (LPA, FPA) yield the strongest empirical privacy scores (highest DCR/NNDR) but perform consistently worst for forecasting, confirming that direct value perturbation disrupts temporal structures.
The Trade-off: A strong negative correlation ( $\rho = -0.776$ ) exists between forecasting utility and empirical privacy separation. Methods preserving structural proximity to original data excel at forecasting but offer low privacy; methods with strong privacy separation often degrade forecasting utility.
Pareto Frontier: Grasynda-P lies on the Pareto frontier, meaning no other evaluated synthetic generator simultaneously achieves better forecasting utility and better empirical privacy separation. It offers stronger privacy separation than other generators while maintaining competitive forecasting performance.
Statistical Significance: Grasynda-P shows statistically significant improvements in empirical privacy (DCR) over the original Grasynda across all datasets ( $p < 0.05$ ) without a statistically significant loss in forecasting performance ( $p > 0.05$ ).

Significance and Claims

The paper claims to establish a reference point for evaluating and developing future privacy-aware synthetic time series generation methods. It highlights that:

The trade-off between utility and privacy is structural and persistent, even among synthetic generators.
Graph-based generation (as implemented in Grasynda-P) represents a favorable and underexplored approach for privacy-sensitive settings, as it operates on transition structures rather than direct value transformations, allowing for a better balance of the two objectives.
While noise-based methods offer the highest privacy, they are unsuitable as training data substitutes for forecasting tasks.
The proposed benchmark and the Grasynda-P method provide a practical pathway for organizations to generate synthetic data that preserves temporal dynamics necessary for forecasting while offering stronger empirical privacy separation than existing synthetic generators.

The authors modestly note limitations, including the use of empirical (not formal differential privacy) metrics, the focus on univariate series, and the use of a single forecasting architecture (NHITS). They suggest that connecting empirical and formal privacy perspectives and generalizing to multivariate or high-frequency data are directions for future work.