Lightweight Time Series Data Valuation on Time Series Foundation Models via In-Context Finetuning

This paper proposes LTSV, a lightweight and efficient method for valuing time series data in foundation models by leveraging in-context finetuning and temporal block aggregation to overcome the computational bottlenecks and temporal dependency limitations of traditional approaches.

The Gist

Imagine you are a chef trying to create the world's best soup. You have a massive, high-tech kitchen (a Time Series Foundation Model) that can learn to cook almost anything. But here's the problem: you have a giant warehouse full of ingredients (your Time Series Data), and some of them are fresh and delicious, while others are rotten, stale, or just plain boring.

If you throw everything into the pot, the soup might be okay, but it won't be amazing. If you could somehow taste every single ingredient and know exactly which ones make the soup better and which ones ruin it, you could pick only the best ones. That is the goal of Data Valuation.

However, in the world of AI, checking the quality of these ingredients is usually like trying to count every grain of sand on a beach while the tide is coming in. The traditional methods are so slow and heavy that they break the computer before they finish.

This paper introduces LTSV, a clever, lightweight new way to taste-test your data ingredients without breaking a sweat.

The Old Way: The "Heavy Lifter"

Imagine the old method (called Influence Functions) as a giant, slow-moving crane. To check if one apple is good, the crane has to:

1. Lift the entire warehouse.
2. Remove the apple.
3. Weigh the whole warehouse again.
4. Put the apple back.
5. Repeat this for every single apple.

For a small kitchen, this is fine. But for a massive AI model with billions of parameters (like a skyscraper-sized warehouse), this crane is too heavy. It takes forever and costs a fortune in electricity. Plus, time series data is tricky because it's a story—what happened yesterday affects today. The old crane often misses the plot because it's too busy lifting heavy weights.

The New Way: The "Smart Taster" (LTSV)

The authors of this paper came up with LTSV (Lightweight Time Series Valuation). Instead of a giant crane, they use a smart, quick taster.

Here is how it works, using a simple analogy:

1. The "One-Step Taste Test" (In-Context Finetuning)

Instead of rebuilding the whole kitchen to test an ingredient, the chef takes a tiny pinch of the ingredient and adds it to a small, pre-made sample of soup (the Context).

• The Trick: They let the AI model take just one tiny step to learn from that ingredient.
• The Result: They check: "Did the soup taste better after adding this pinch?"
  • If the soup improved, that ingredient is High Value.
  • If the soup got worse or stayed the same, that ingredient is Low Value.

This is mathematically proven to be almost the same as the heavy crane method, but it's thousands of times faster because it only takes one tiny step instead of lifting the whole building.

2. The "Time-Traveling Blocks" (Temporal Block Aggregation)

Time series data is like a movie, not a photo. You can't just look at one frame and know the story.

• The Problem: If you only taste one second of a song, you might miss the chorus.
• The Solution: LTSV cuts the data into overlapping blocks (like chapters in a book). It tastes a whole chapter, then moves the window forward slightly and tastes the next overlapping chapter.
• By averaging these "chapter scores," it understands the flow and rhythm of the data, ensuring it doesn't miss the important "temporal dependencies" (the story connecting the data points).

Why This Matters

The researchers tested this on five different real-world datasets (like electricity usage, weather, and stock markets) and three different giant AI models.

• The Result: When they picked only the "Top 50%" of ingredients identified by LTSV and used them to train the AI, the AI performed better than if they had used all the data.
• The Bonus: Even if you take the "good ingredients" found by this method and give them to a different type of chef (a different AI model), those ingredients still make the new chef's soup taste great. This means the method is transferable.

The Bottom Line

LTSV is like having a magic spoon that can instantly tell you which data points are the "gold" and which are "dirt" in a massive dataset.

• It's fast (lightweight).
• It's smart (understands the story of time).
• It's effective (helps AI models learn better with less data).

This is a huge step forward because it allows us to build better, smarter AI models without needing to wait years for the computer to finish its calculations. It turns the impossible task of valuing massive data into a quick, manageable job.

Technical Summary

Here is a detailed technical summary of the paper "Lightweight Time Series Data Valuation on Time Series Foundation Models via In-Context Finetuning" (LTSV).

1. Problem Statement

Time Series Foundation Models (TSFMs) have achieved state-of-the-art performance by pretraining on massive, diverse datasets. However, their performance is heavily dependent on the quality of training data.

• The Challenge: Existing data valuation methods (e.g., Influence Functions, Shapley Values) are theoretically sound but computationally prohibitive for TSFMs. They require Hessian matrix inversions or exponential subset sampling, which scale poorly with models containing millions or billions of parameters.
• The Gap: There is a lack of scalable, efficient, and temporally aware methods to quantify the contribution of individual time series samples to TSFM performance. Traditional methods also often fail to preserve complex temporal dependencies inherent in time series data.

2. Methodology: LTSV

The authors propose LTSV (Lightweight Time Series Valuation), a framework that leverages in-context finetuning to approximate influence functions efficiently.

Core Theoretical Insight

The method is grounded in the theoretical evidence that one-step in-context finetuning approximates the classical influence function.

• Classical Influence Function: Measures how a perturbation in a target sample $z$ affects the loss on a context sample $z'$. It involves the inverse Hessian matrix ($H^{-1}$), which is computationally expensive ($O(P^3)$).
• LTSV Approximation: Instead of computing the Hessian, LTSV performs a single gradient update on the target sample $z$ to obtain updated parameters $\theta_{finetuned}$. The influence is estimated by the change in context loss before and after this update:
  $$\text{Infl}(z, z') \propto L(z'; \theta) - L(z'; \theta_{finetuned})$$
  This reduces complexity to $O(P)$ (linear scaling) as it only requires a single forward-backward pass.

Algorithmic Workflow

1. Temporal Block Aggregation: To handle temporal dependencies, the time series is not treated as isolated points but divided into overlapping blocks of fixed length $L$.
2. Block-Level Scoring: Each block is treated as a "target sample." The TSFM is fine-tuned on this block for one step, and the reduction in loss on a held-out context set is calculated as the block's value.
3. Hierarchical Aggregation:
   • Point-wise: Scores for individual time points are derived by averaging the scores of all overlapping blocks covering that point.
   • Sample-wise: The final value for a time series sample is the average of its constituent point-wise scores.
4. Multivariate Handling: The method naturally handles multivariate inputs by processing blocks with multiple channels simultaneously, avoiding the need for channel splitting or high-dimensional parameter matrices.

3. Key Contributions

1. Novel Framework: Introduction of LTSV, the first lightweight data valuation method specifically designed for Time Series Foundation Models using in-context finetuning.
2. Theoretical & Practical Efficiency: Theoretically proves that in-context finetuning approximates influence functions, achieving linear computational complexity ($O(nP)$) compared to the cubic complexity of traditional methods, making it feasible for billion-parameter models.
3. Temporal Awareness: Introduces temporal block aggregation to capture inter-temporal relationships, overcoming the limitations of point-wise valuation methods.
4. Transferability: Demonstrates that data valuations derived from a TSFM can effectively generalize to diverse downstream models (linear, Transformer-based, attention-based), acting as a universal quality metric.

4. Experimental Results

The authors evaluated LTSV on five datasets (Electricity, Exchange Rate, Weather, Illness, ETT) and three TSFM architectures (Time-MoE, Time-LLM, MOMENT).

• Data Selection Efficacy:
  • Fine-tuning TSFMs on the top 50% of samples identified by LTSV consistently outperformed fine-tuning on the bottom 50%.
  • In many cases, the top 50% achieved performance comparable to or better than using the full dataset, proving that LTSV can identify high-value data that drives model performance.
• Computational Efficiency:
  • Compared to classical Influence Functions, LTSV showed near-linear scaling with model size.
  • While Influence Functions became prohibitively slow for models with >40M parameters, LTSV maintained manageable runtimes even for 200M+ parameter models.
• Generalization:
  • Data valuations computed on a TSFM (e.g., Time-MoE) successfully improved downstream models (DLinear, PatchTST, PAttn) when used for data selection.
  • LTSV's performance in selecting high-quality data was comparable to, and sometimes better than, classical methods computed directly on the smaller downstream models.
• Ablation Study: The method proved robust to variations in block length ($L \in \{50, 75, 100, 125\}$), with moderate block sizes (75-100) providing optimal stability.

5. Significance

• Bridging Data and Model Generalization: LTSV provides a practical bridge between data attribution and model generalization, enabling efficient resource allocation in the era of large-scale time series models.
• Scalability: It solves the "computational bottleneck" of data valuation for foundation models, allowing researchers and practitioners to curate high-quality datasets without incurring massive computational costs.
• Quality Control: By identifying and filtering out corrupted or unrepresentative samples, LTSV enhances the training efficiency and robustness of TSFMs, which is critical for applications in finance, healthcare, and climate science.

In summary, LTSV offers a theoretically grounded, computationally efficient, and empirically robust solution for valuing time series data in the context of modern foundation models.