Adapt Data to Model: Adaptive Transformation Optimization for Domain-shared Time Series Foundation Models

This paper proposes TATO, a data-centric framework that optimizes a pipeline of adaptive transformations (context slicing, scale normalization, and outlier correction) to enable a single frozen large time series model to effectively adapt to diverse downstream domains, significantly improving forecasting accuracy and generalization while maintaining high computational efficiency.

The Gist

Imagine you have a super-smart, world-class chef (the Large Time Series Model, or LTM). This chef has trained on millions of recipes and can cook almost anything. However, there's a catch: this chef is frozen. You can't teach them new recipes, you can't change their taste buds, and you can't ask them to retrain. They are stuck with exactly what they know.

Now, imagine you hire this chef to cook for three very different groups of people:

1. The Italians: They want spicy, fresh pasta.
2. The Japanese: They want delicate, precise sushi.
3. The Americans: They want a hearty, greasy burger.

If you just hand the chef a bag of raw ingredients for the Italian dinner, they might try to make sushi out of it because that's what they are used to. The result? A disaster.

The Problem:
In the world of data, these "groups of people" are different real-world scenarios (like stock markets, weather patterns, or electricity usage). The "ingredients" are the raw data. The "frozen chef" is a powerful AI model that was trained on a mix of everything but doesn't know how to handle the specific quirks of your specific data. Usually, to fix this, you'd have to send the chef back to culinary school (retraining the model), which takes months, costs a fortune, and ruins their ability to cook for the other groups.

The Solution: TATO (The "Prep Station" Fix)
The paper introduces a brilliant new idea called TATO (Time-series Adaptive Transformation Optimization). Instead of trying to change the chef, TATO changes how the ingredients are prepared before they reach the chef.

Think of TATO as a super-smart kitchen prep station that sits right before the chef. Its job is to look at the raw ingredients and say, "Wait, the chef doesn't like raw onions for this dish. Let's chop them finely first. Oh, and this meat is too cold; let's thaw it. And this vegetable is rotten; let's cut that part off."

Here is how TATO works in simple steps:

1. The Three Magic Tools

TATO uses three main types of "prep tools" to fix the data before it hits the model:

• The Slicer (Context Slicing): Sometimes the chef gets too much history and gets confused. TATO cuts the data to show the chef just the right amount of recent history, like giving a chef a specific recipe card instead of a whole library of books.
• The Scale Adjuster (Normalization): Some data is huge (like a mountain) and some is tiny (like a pebble). TATO shrinks the mountain and grows the pebble so they look the same size to the chef. This helps the chef compare them fairly.
• The Outlier Fixer (Correction): Sometimes data has weird glitches (like a sensor breaking and saying the temperature is 500°F). TATO spots these "glitches," removes them, or fixes them so the chef doesn't get distracted by nonsense.

2. The "Trial and Error" Taste Test

How does TATO know which tools to use? It runs a rapid-fire taste test.

• It takes a small batch of your data.
• It tries thousands of different combinations of slicing, scaling, and fixing.
• It feeds these different versions to the frozen chef and sees which one results in the best prediction.
• It picks the perfect combination for your specific data.

3. Why It's a Game Changer

• It's Fast: Instead of months of retraining, TATO finds the perfect prep method in under 2 minutes.
• It's Cheap: You don't need a supercomputer to retrain the model; you just need a little bit of data to figure out the prep style.
• It's Universal: One frozen chef can now cook for the Italians, the Japanese, and the Americans perfectly, as long as the prep station adapts the ingredients for each group.

The Result

In their experiments, the researchers found that using this "prep station" (TATO) made the frozen chefs significantly better at predicting the future.

• In some cases, they reduced errors by 65% (that's like going from a terrible meal to a Michelin-star dish).
• On average, they reduced errors by 13.6%.

The Bottom Line:
The paper argues that instead of constantly trying to build bigger, smarter, and more expensive AI models, we should focus on adapting the data to fit the models we already have. It's like realizing that you don't need a new chef for every cuisine; you just need a really good prep station to get the ingredients ready.

Technical Summary

1. Problem Statement

Large Time Series Models (LTMs) have emerged as powerful tools for universal forecasting, capable of zero-shot and few-shot learning across diverse domains. However, they face significant challenges due to the inherent diversity and non-stationarity of real-world time series data.

• The Trade-off: A single frozen LTM often struggles to generalize across all domains without compromising accuracy.
• The Limitation of Fine-tuning: While fine-tuning specific LTM instances for each domain improves accuracy, it destroys the model's generalizability, incurs prohibitive computational costs, and requires massive data storage for multiple model instances.
• The Gap: Existing approaches focus heavily on model-centric development (pre-training or fine-tuning), overlooking the potential of data-centric adaptation. There is a lack of automated frameworks that can optimize data preprocessing pipelines to align diverse downstream data with a single frozen foundation model.

2. Methodology: TATO Framework

The authors propose TATO (Time-series Adaptive Transformation Optimization), a data-centric framework designed to enable FrozenForecasting. In this paradigm, the LTM remains completely frozen, while a lightweight, automated pipeline transforms the input data to match the model's learned distribution.

Core Components:

1. FrozenForecasting Paradigm:

   • The LTM parameters are fixed.
   • Optimization is performed solely on the data transformation pipeline ($h$) using a small subset of historical data ($D_{history}$).
   • The goal is to find $h^*$ that minimizes the loss $L(M, D_{history}, h)$, where $M$ is the frozen model.

2. Specialized Transformation Search Space:
   TATO constructs a search space comprising nine tunable operators categorized into three types:

   • Contextual Transformations: Adjust the inference context (e.g., Trimmer to adjust lookback length, Sampler for downsampling/upsampling, Aligner for patch padding).
   • Normalization Transformations: Adjust value ranges to ensure stability (e.g., Scaler for standard/robust normalization, Differencer to induce stationarity, Warper for log transformations).
   • Outlier Transformations: Mitigate noise and anomalies (e.g., Denoiser via moving averages, Imputator for interpolation, Clipper to bound extreme values).

3. Optimization Process:

   • Data Augmentation: To ensure robustness against distribution shifts, the optimization phase applies diverse augmentations (flipping, warping, noise injection, translation) to the training samples.
   • Hyperparameter Optimization: The framework uses Bayesian Optimization (Tree-structured Parzen Estimator - TPE) to search for the optimal combination of operators and their hyperparameters.
   • Heuristic Ordering: To reduce search complexity, operator ordering is constrained by rules (e.g., trimming first, outlier handling before normalization).

4. Two-Stage Pareto-Based Ranking:
   To select the final pipeline without overfitting to specific metrics:

   • Stage 1 (Filtering): Uses Pareto ranking on augmented data to filter out "risky" trials that perform poorly on any metric subset. This eliminates pipelines that only excel on one specific metric.
   • Stage 2 (Selection): Ranks the remaining candidates on the original (non-augmented) validation data using a weighted sum of error metrics (MSE and MSPE weighted highest).

3. Key Contributions

• New Paradigm (FrozenForecasting): Introduces a novel approach where a single frozen LTM serves all domains via adaptive data preprocessing, balancing specificity and generalization without retraining the model.
• TATO Framework: An automated, model-agnostic system that discovers optimal preprocessing pipelines through a curated search space of context, scale, and outlier operators.
• Robust Optimization Mechanism: Incorporates data augmentation during search and a two-stage Pareto ranking system to ensure the selected pipeline is robust across diverse metrics and data distributions.
• Efficiency: The method is computationally lightweight, typically completing optimization in under 2 minutes, making it practical for real-world deployment.

4. Experimental Results

The authors evaluated TATO on 8 datasets (ETT, Electricity, Exchange, Traffic, Weather) across 8 state-of-the-art LTMs (Timer, Moirai, Chronos).

• Performance Improvement:

  • Average MSE Reduction: 13.6% across all models and datasets.
  • Maximum MSE Reduction: 65.4% (observed on the Exchange dataset with Timer-LOTSA).
  • TATO improved or matched baseline performance in 84.3% of the 192 evaluated scenarios.
  • Significant gains were observed particularly in cases where vanilla baselines performed poorly (e.g., handling non-stationarity or outliers).

• Efficiency:

  • Optimization typically takes < 2 minutes (e.g., ~76s for 500 trials on Timer-UTSD).
  • Inference overhead is negligible (< 3ms per sample for batch size 1).

• Scalability & Ablation:

  • Performance scales positively with more trials and data samples, stabilizing around 500 samples.
  • Ablation studies confirmed that removing key components (like the Trimmer or TwoStageRank) significantly degrades performance or robustness.
  • TATO also provides additional gains (approx. 7% MSE reduction) even when applied to models that have already been universally fine-tuned, proving its complementary nature.

5. Significance

This paper shifts the focus from "model adaptation" to "data adaptation" for large time series models.

• Practical Deployment: It offers a viable solution for deploying large foundation models in production environments where retraining is too expensive or data privacy prevents fine-tuning.
• Generalization: By keeping the model frozen, it preserves the model's zero-shot capabilities while allowing it to adapt to specific domain characteristics through lightweight transformations.
• Sustainability: The approach reduces the carbon footprint and computational cost associated with maintaining and training multiple domain-specific model instances.

In summary, TATO demonstrates that optimizing how data is presented to a frozen model is a highly effective, efficient, and robust strategy for unlocking the full potential of Large Time Series Models in diverse, real-world applications.