Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgment

The paper proposes TSRating, a novel meta-learning framework that leverages LLMs to generate quality comparisons across diverse domains and trains an efficient TSRater model to accurately and adaptively evaluate time series data quality without requiring extensive hypergradient computations.

The Gist

Imagine you are trying to teach a brilliant but inexperienced chef how to cook a perfect meal. You have a massive pantry filled with ingredients from all over the world: fresh vegetables from a farm, canned goods from a factory, and spices from a distant market.

The problem? Not all ingredients are created equal. Some vegetables are wilted, some cans are dented, and some spices have gone bad. If you feed the chef the bad stuff, the meal will taste terrible, no matter how good the chef is.

This is exactly the challenge researchers face with Time Series Data. This is data that changes over time, like stock market prices, heart rate monitors, weather patterns, or traffic flow. We have huge amounts of this data, but it's often messy, broken, or full of "noise."

The paper introduces a new system called TSRating to solve this problem. Here is how it works, explained simply:

1. The Problem: The "One-Size-Fits-All" Trap

Previously, scientists tried to grade data quality using complex math formulas (like "Influence Functions" or "Shapley Values"). Think of these like trying to use a microscope to inspect a whole forest.

• The Issue: These math tools are incredibly slow and expensive. Worse, they are designed for specific types of data. A tool perfect for grading stock market data might fail miserably when looking at weather data. It's like trying to use a fishing net to catch butterflies; it just doesn't work well across different "domains."

2. The Solution: The "Super-Expert" Judge (The LLM)

The authors realized that Large Language Models (LLMs)—the same AI behind chatbots—are actually very good at understanding patterns. They have "read" so much data during their training that they intuitively understand what a "good" pattern looks like versus a "bad" one.

Instead of using complex math, they asked the AI: "Look at these two time series graphs. Which one looks more reliable?"

To make the AI a fair judge, they gave it four specific criteria (like a rubric for a school project):

• Trend: Is there a clear direction (up or down), or is it just random noise?
• Frequency: Is there a steady rhythm (like a heartbeat), or is it chaotic?
• Amplitude: Are the changes big and meaningful, or tiny and insignificant?
• Pattern: Can you see a repeating shape or structure?

3. The Magic Trick: The "Intern" (TSRater)

Asking the AI to grade every single piece of data in a massive dataset would be too slow and expensive (like hiring a famous chef to taste every single grain of rice in a warehouse).

So, the researchers used a clever two-step process:

1. The Training Phase: They asked the AI to grade thousands of pairs of data samples.
2. The Intern Phase: They trained a smaller, faster, cheaper AI model (called TSRater) to mimic the famous AI's judgments.

Think of TSRater as a junior intern who has shadowed the Master Chef (the big LLM). The intern watched the Master grade thousands of samples and learned the rules. Now, the intern can grade new data almost instantly, for free, without needing the Master Chef's help every time.

4. The "Chameleon" Ability (Meta-Learning)

Here is the real genius part. Usually, an intern trained on stock market data would be terrible at grading weather data.

But the researchers used a technique called Meta-Learning. Imagine teaching the intern not just what to grade, but how to learn how to grade.

• They showed the intern examples from 9 different worlds: finance, healthcare, traffic, weather, etc.
• The intern learned to adapt quickly. Now, if you hand them a new dataset from a domain they've never seen before, they can adjust their "taste buds" and grade it accurately with very little extra practice.

5. The Results: Why It Matters

The researchers tested this system on 11 different real-world datasets.

• Accuracy: It found the "bad ingredients" better than the old math-heavy methods.
• Speed: It was much faster. While the old methods took hours or days to grade a dataset, the new system did it in minutes.
• Performance: When they used the "good" data selected by TSRating to train other AI models, those models became significantly smarter and more accurate.

The Big Picture Analogy

Imagine you are building a house.

• Old Method: You hire a team of engineers to measure every single brick with a laser scanner to see if it's perfect. It takes forever, costs a fortune, and they get confused if the bricks are from a different factory.
• TSRating Method: You hire a master builder (the LLM) to teach a skilled apprentice (TSRater) how to spot a bad brick just by looking at it. The apprentice can then quickly sort through a mountain of bricks from any factory, pick out the best ones, and hand them to the construction crew. The house gets built faster, cheaper, and stands much stronger.

In short: This paper teaches us how to use AI to act as a quality control inspector for time-based data, ensuring that the models we build are fed only the highest-quality "food" so they can perform at their best.

Technical Summary

Here is a detailed technical summary of the paper "Rating Quality of Diverse Time Series Data by Meta-Learning from LLM Judgment" (TSRating), published at ICLR 2026.

1. Problem Statement

Time series (TS) data is ubiquitous across domains like healthcare, finance, and industrial manufacturing, yet it suffers from pervasive quality issues such as missing values, sensor corruption, and irregular sampling. High-quality data is critical for the performance of both traditional TS models and emerging TS Foundation Models (TSFMs).

Existing data quality rating methods face two major limitations:

• Domain Specificity: Current approaches (e.g., TimeInf, TimeShap) are often tailored to specific domains or assume distributional homogeneity, failing to generalize across the vastly different characteristics of real-world TS data.
• Computational Inefficiency: Methods based on Influence Functions or Shapley values require expensive Hessian matrix calculations or exponential computational costs, making them impractical for large-scale or diverse datasets.

There is a pressing need for a unified, efficient, and domain-adaptive framework to rate TS data quality without prohibitive computational overhead.

2. Methodology: The TSRating Framework

The authors propose TSRating, a unified framework that leverages the inherent knowledge of Large Language Models (LLMs) to assess TS quality, followed by a meta-learned model for efficient inference.

A. LLM-Based Quality Judgment

Instead of calculating complex statistical metrics, the framework utilizes LLMs (e.g., GPT-4o-mini) to judge the quality of TS blocks based on four fundamental criteria widely recognized in TS analysis:

1. Trend: Directional movement (upward/downward/stable).
2. Frequency: Regularity of oscillations and periodicity.
3. Amplitude: Magnitude and consistency of fluctuations.
4. Pattern: Recurrent structures (seasonality, stationarity).

Process:

• Pairwise Comparison: Time series are segmented into overlapping blocks. LLMs are prompted to compare pairs of blocks ($B_i$ vs. $B_j$) based on the four criteria, outputting a binary preference ($B_i \succ B_j$).
• Confidence Scoring: To ensure reliability, multiple LLM responses are aggregated, and positional bias is mitigated by swapping block orders.
• Scalar Conversion: Binary preferences are converted into scalar quality scores using the Bradley-Terry model via Maximum Likelihood Estimation (MLE).

B. The TSRater Model

To avoid the high cost of querying LLMs for every new dataset, a lightweight rating model called TSRater is trained to mimic the LLM's judgments.

• Architecture: TSRater uses a frozen MOMENT (a TS Foundation Model) encoder to extract temporal features, followed by a Multi-Layer Perceptron (MLP) that maps these features to scalar quality scores.
• Training Objective: The MLP is trained to minimize the binary cross-entropy loss between its predicted scores and the LLM-derived preference labels.

C. Meta-Learning for Cross-Domain Adaptability

To ensure TSRater generalizes across diverse domains (e.g., energy, finance, weather), the authors employ Model-Agnostic Meta-Learning (MAML).

• Task Construction: The model is trained on 22 subsets from the Time-300B corpus, covering 9 distinct domains.
• Optimization: The framework uses SignSGD for inner-loop updates. This approximates the gradient direction using only the sign, effectively bypassing the need to compute second-order derivatives (Hessians) required by standard MAML. This significantly reduces computational complexity while maintaining fast adaptation capabilities.

3. Key Contributions

1. Unified Framework: Introduction of TSRating, the first framework to unify TS quality rating across diverse domains using LLM-guided meta-learning.
2. LLM as a Quality Oracle: Demonstration that LLMs, despite being text-trained, possess an inherent ability to comprehend and distinguish TS characteristics (trend, frequency, amplitude, pattern) when guided by specific prompts.
3. Efficient Meta-Learning: Development of a SignSGD-based meta-learning strategy that enables the rating model to adapt to new domains with minimal fine-tuning, avoiding the computational bottlenecks of traditional influence-based methods.
4. Scalability: The framework decouples the expensive LLM judgment phase (done once for training data) from the inference phase, where the lightweight TSRater can rate new data instantly.

4. Experimental Results

The framework was evaluated on 11 benchmark datasets across three tasks: Long-term Forecasting, Short-term Forecasting, and Classification, using various models (Linear, CNN, PatchTST, iTransformer, etc.).

• Estimation Accuracy: TSRating consistently outperformed state-of-the-art baselines (DataShapley, KNNShapley, DataOob, TimeInf). It achieved the best RMSE in 6/12 long-term forecasting cases and best accuracy in 10/12 classification cases.
• Data Pruning: When high-quality samples selected by TSRating were removed, downstream model performance degraded significantly faster than when samples selected by baselines were removed. This proves TSRating effectively identifies the most critical, high-quality data.
• Efficiency:
  • Runtime: TSRater's total runtime is comparable to DataOob and TimeInf but significantly faster than DataShapley (which took ~210,000s vs. TSRater's ~4,687s on Time-300B).
  • Reusability: Once meta-trained, applying TSRater to a new dataset requires only ~200 seconds (few-shot tuning + inference), whereas baselines require retraining from scratch.
• Foundation Model Finetuning: A case study showed that finetuning TSFMs (Time-MoE, Time-LLM, MOMENT) on high-quality subsets selected by TSRating significantly improved generalization (lower MSE/MAE) compared to training on low-quality or full datasets.
• Ablation Studies:
  • Meta-Learning: Meta-trained raters performed nearly as well as raters trained specifically on the target dataset, outperforming raters trained on unrelated domains.
  • Criteria Fusion: Combining all four criteria yielded more stable and superior results than using individual criteria.
  • Encoder Independence: TSRater's performance was robust across different TSFM encoders (MOMENT, Chronos, TimeGPT).

5. Significance

TSRating addresses a critical bottleneck in the era of data-centric AI for time series. By shifting the paradigm from computationally expensive, domain-specific statistical attribution to LLM-guided meta-learning, the paper offers a solution that is:

• Domain-Agnostic: Capable of handling heterogeneous data distributions without re-engineering.
• Computationally Feasible: Drastically reduces the cost of data valuation, making it viable for large-scale datasets.
• Practically Impactful: Directly enhances the performance of downstream tasks, including the training of expensive Foundation Models, by enabling the selection of "golden" data subsets.

This work bridges the gap between the interpretability of LLMs and the structural complexity of time series data, establishing a new standard for data quality assessment in temporal domains.