🤖 machine learning

MetaCaster: Meta-Harness-Optimized Agent for End-to-End Few-Shot Learning of Lightweight Time Series Forecasters

MetaCaster is a meta-harness-optimized multi-agent framework that enables the end-to-end few-shot learning of lightweight time series forecasters by employing agents to generate training data and textual contexts, thereby achieving high performance with minimal data and computational resources in resource-constrained scenarios.

Original authors: ChengAo Shen, Wenchao Yu, Fangyu Wu, Dongjin Song, Hanghang Tong, Dongsheng Luo, Wei Cheng, Haifeng Chen, Jingchao Ni

Published 2026-08-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: ChengAo Shen, Wenchao Yu, Fangyu Wu, Dongjin Song, Hanghang Tong, Dongsheng Luo, Wei Cheng, Haifeng Chen, Jingchao Ni

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Time series forecasting is the art of reading the past to understand the future. It is the mathematical engine behind predicting everything from the next hour's electricity demand to the spread of a flu outbreak. For decades, this field relied on specialized, compact computer programs designed to spot patterns in numbers. These programs were fast and efficient but needed vast amounts of historical data to learn, making them useless in situations where data is scarce, private, or slow to accumulate. Recently, a new wave of artificial intelligence has entered the scene, bringing with it massive language models capable of reasoning about text and numbers. While these giant models are powerful, they are often too heavy and expensive to run on the small devices or in the tight budgets where these predictions are actually needed. The central challenge has become finding a way to get the best of both worlds: the intelligence of a large system with the speed and efficiency of a small one, all while working with only a handful of examples.

Researchers have proposed a new approach called METACASTER, which shifts the role of artificial intelligence from being the predictor itself to becoming the architect of the predictor. Instead of asking a large language model to guess the next number in a sequence, the system asks the model to act as an engineer. Its job is to design and train a tiny, specialized forecasting program that can run on a simple device. This process begins with a very small set of real-world data, perhaps just ten or fifty examples of a specific pattern, along with a written description of the context, such as "electricity usage in a hospital." The system then uses a team of digital agents to generate a much larger, synthetic dataset that mimics the real one. This artificial data is not just random noise; it is carefully crafted to teach a lightweight model how to forecast accurately. Once this synthetic data is ready, another agent selects the best small forecasting model from a library of twenty-three different candidates and trains it on the new data. The result is a highly efficient, custom-built forecaster that performs as well as models trained on massive datasets, but without the need for them.

The researchers tested this method across eighteen different real-world datasets, ranging from traffic flow and weather patterns to energy consumption and sales figures. They compared their system against fourteen other methods, including traditional data augmentation techniques and the latest large-scale foundation models. The results showed that METACASTER could achieve high-quality predictions using only a few examples, often outperforming the other methods. In many cases, the system produced forecasts that were nearly as accurate as those generated by models trained on the full, original historical data. Perhaps most importantly, the final models it produced were incredibly lightweight. One selected model, for instance, required only 243 parameters to operate, a fraction of the size of the massive models that typically dominate the field. This allowed the system to run with significantly lower latency and energy costs, making it feasible to deploy on standard hardware rather than requiring expensive, power-hungry servers.

A key innovation in this work is the way the system improves itself. The researchers found that the performance of the agents depended heavily on the "harness" surrounding them—the set of instructions, tools, and rules that guide their behavior. To optimize this, they introduced a meta-agent that acts like a supervisor, constantly reviewing the work of the data generator and the trainer. If the generated data leads to poor forecasts, the supervisor analyzes the reasoning traces, identifies the flaw, and automatically rewrites the instructions for the generator to fix it. This process happens over several rounds of iteration, refining the system's ability to create useful synthetic data until it reaches a peak level of performance. Once this optimization is complete, the supervisor is discarded, and the final, streamlined system is ready for deployment.

The study also highlighted what this approach is not. The researchers demonstrated that simply trying to make the generated data look statistically similar to the real data, without focusing on the ultimate goal of forecasting, actually led to worse results. The system works because it is explicitly trained to produce data that improves the forecast, not just data that looks real. Furthermore, while the system is powerful, it does not work in a complete vacuum; it requires at least a few real examples to ground its understanding of the specific domain. Without any reference data, the system cannot reliably generate the necessary patterns. However, with just a small number of samples, it bridges the gap between the need for rapid deployment and the reality of limited data, offering a practical path forward for bringing advanced forecasting to resource-constrained environments.

Technical Summary: METACASTER

Problem Statement

Time series forecasting (TSF) is increasingly shifting toward multimodal and agentic settings, often leveraging Large Language Models (LLMs). However, using foundation models (LLMs or Time Series Foundation Models) as the computational core for forecasting faces significant barriers: the modality gap between discrete language tokens and continuous time series values, high computational costs, and unsustainability in resource-constrained environments (e.g., edge devices). While lightweight, specialized forecasters offer a more efficient alternative, they typically require substantial downstream training data, rendering them ineffective in domains with scarce, privacy-sensitive, or slowly accumulated time series (few-shot settings).

The core challenge addressed by this work is: Can lightweight forecasters be effectively trained with only a few samples? Existing few-shot approaches often rely on data augmentation or generative models that focus on data realism rather than downstream forecasting performance, leading to biased or suboptimal forecasters.

Methodology: METACASTER

The authors propose METACASTER (Meta-Harness-Optimized Agent for End-to-End Few-Shot Learning), a novel multi-agent framework that redefines the role of agents in TSF. Instead of acting as forecasters themselves, the agents function as intermediary engineers that automatically prepare specialized, lightweight forecasters for deployment.

The framework operates in two distinct phases: Harness Optimization and Deployment/Inference.

1. Framework Architecture

METACASTER employs three specialized agents:

Meta-Generator (MGAGENT): Given a few-shot support set ( $D_{sup}$ ) and textual context ( $C$ ), MGAGENT generates a sufficient synthetic dataset ( $\bar{D}$ ) that complies with domain constraints. Crucially, MGAGENT does not generate data directly; it uses its Harness (system prompts, skills, tools) to orchestrate a TS-Generator program that integrates domain-specific knowledge to synthesize data.
Forecaster Trainer (FTAGENT): This agent splits the generated dataset $\bar{D}$ into training and validation sets. It orchestrates the training of a library of lightweight forecasters (LT-LIB), performs hyperparameter tuning via grid search, and selects the best-performing model based on validation metrics.
Harness Proposer (HPAGENT): This is the meta-optimizer. It observes the performance gap between forecasters trained on authentic data versus those trained on the synthetic data generated by MGAGENT. HPAGENT iteratively updates the Harness (specifically the system prompts and skills) of MGAGENT to minimize this gap, effectively optimizing the data generation process for forecasting utility rather than just statistical realism.

2. The Meta-Harness Optimization

Unlike traditional fine-tuning of LLMs, METACASTER optimizes the Harness ( $\theta$ ) surrounding the LLM. The optimization objective (Eq. 1) minimizes the difference in forecasting errors between models trained on authentic data and those trained on generated data:
$\min_{\theta} \mathbb{E} \left[ \delta(\omega(f_l), \omega(\bar{f}_l)) \right]$
Where $\omega$ is the forecasting metric (e.g., MSE) and $\delta$ is a hinge-loss based measure penalizing cases where the synthetic-data-trained model underperforms the authentic-data-trained model.

HPAGENT operates in an outer loop, analyzing reasoning traces, training logs, and statistical distributions to diagnose failures (e.g., broken inter-variate correlations) and update MGAGENT's skills accordingly. Once optimization converges, HPAGENT is discarded, and the optimized Harness is transferred to the deployment agents.

3. LT-LIB (Lightweight TSF Library)

To support FTAGENT, the authors compiled LT-LIB, a unified library containing 23 state-of-the-art lightweight forecasters (2022–2026). These include linear models (e.g., MixLinear, DLinear), MLP-based models (e.g., TSMixer), and frequency-domain models (e.g., FITS). These models are significantly smaller (e.g., MixLinear has ~243 parameters) compared to Transformer-based foundation models.

4. Deployment and Inference

After the Harness optimization phase, the system deploys the optimized MGAGENT and FTAGENT (attached to user-selected LLM APIs) to generate data and train a specific lightweight forecaster for a target domain. Once the best forecaster is selected, the agents are dropped. Inference is performed solely by the lightweight forecaster, requiring no LLM calls and minimal computational resources.

Key Contributions

Problem Investigation: The paper investigates the challenging problem of few-shot learning for lightweight forecasters, addressing the trade-off between model efficiency and data scarcity.
METACASTER Framework: A novel meta-harness-optimized multi-agent system that treats agents as engineers. It aligns data generation directly with downstream forecasting performance, distinguishing it from standard generative models.
LT-LIB: The compilation of 23 SOTA lightweight forecasters into a unified library with a standardized API, facilitating automated training and selection.
Harness Optimization: The application of meta-harness optimization to time series agents, demonstrating that optimizing the agent's infrastructure (prompts/skills) is more effective and transferable than fine-tuning the LLM backbone itself.

Experimental Results

The authors evaluated METACASTER on 18 datasets across 9 domains (energy, traffic, healthcare, etc.), comparing it against 14 baselines, including:

Generation Models: TimeVAE, DiffTS, T2S, TimeDP, VerbalTS.
Augmentation Methods: Repeat, Bootstrap, Jitter, MagWarp.
Foundation Models: Chronos, Moirai, VisionTS, Time-LLM.
Agent Pipelines: TimeScientist.

Key Findings:

Performance: METACASTER outperformed generation and augmentation baselines in most cases. With $K \ge 30$ shots, it approached or surpassed the performance of models trained on full authentic datasets, suggesting optimized synthetic data can mitigate noise in raw data.
Generalization: The framework generalized well to Out-of-Domain (OOD) datasets, often outperforming baselines even in unseen domains.
Efficiency: Compared to Time Series Foundation Models (TSFMs), METACASTER achieved comparable performance with significantly lower computational costs. On the Solar dataset, it demonstrated up to $10^3\times$ lower latency and $10^5\times$ fewer parameters than TSFMs.
Ablation Studies:
- Replacing the forecasting-oriented loss with distribution-matching losses (MMD, Wasserstein) degraded performance, confirming that optimizing for data distribution alone is insufficient for forecasting.
- Contextual cues ( $C$ ) were found crucial for guiding domain-relevant generation.
- Different LLM backbones yielded comparable results when using the same optimized Harness, reinforcing that the Harness is the primary driver of performance.

Significance and Claims

The paper claims to establish a new paradigm in Time Series Forecasting: Agent-as-Engineer. By shifting the agent's role from direct prediction to the automated construction of efficient, task-specific forecasters, METACASTER bridges the gap between the data efficiency of few-shot learning and the computational efficiency of lightweight models.

The authors position this work as a solution for rapid deployment in resource-constrained and privacy-sensitive scenarios where collecting large historical datasets is infeasible. They emphasize that while the current work does not address the extreme zero-shot setting (where no examples exist), it provides a practical and computationally sustainable alternative to foundation models for scenarios where a small number of reference examples are available. The work sets a groundwork for further exploration into agentic time series forecasting and harness engineering.