🤖 machine learning

NeST: Neighborhood-aware semantic alignment and temporal modulation for LLM based time series forecasting

The paper proposes NeST, a novel framework that enhances LLM-based time series forecasting by generating neighborhood-aware text prototypes, aligning them with temporal representations via contrastive learning, and applying prototype-conditioned temporal modulation to achieve superior performance and generalization across diverse benchmarks and real-world tasks.

Original authors: Jayanie Bogahawatte, Sachith Seneviratne, Maneesha Perera, Saman Halgamuge

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jayanie Bogahawatte, Sachith Seneviratne, Maneesha Perera, Saman Halgamuge

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Predicting the future of numbers that change over time is a task that shapes how we power our cities, manage our finances, and prepare for the weather. For decades, scientists have relied on specialized mathematical tools to forecast these patterns, known as time series. These tools work well when the data behaves in predictable ways, but they often struggle when the patterns are complex, messy, or when there is very little data to learn from. In recent years, a new kind of artificial intelligence called a Large Language Model has emerged. Originally trained to understand human language, these models are incredibly good at spotting patterns in sequences of words. Because time series data is also a sequence, researchers have begun trying to teach these language models to predict numbers instead of sentences. The challenge lies in bridging the gap between the discrete, distinct words a language model knows and the smooth, continuous flow of real-world data like electricity usage or temperature.

A team of researchers at the University of Melbourne has developed a new method to solve this problem, calling their framework NeST. Their approach focuses on how to best translate the language model's understanding of words into a guide for understanding time. Instead of forcing the model to simply read a list of numbers as if they were words, or using heavy-handed techniques that require massive amounts of computing power, NeST creates a set of "text prototypes." Think of these prototypes as carefully chosen representative anchors that capture the local neighborhoods of meaning within the language model's vast vocabulary. The researchers trained the system to ensure these anchors stay true to the original structure of the language model's knowledge, rather than drifting into areas where the model has no real understanding.

Once these text anchors are established, the system aligns them with the actual time series data it needs to predict. It does this by finding the closest matches between the data and the text anchors, ensuring that the language model's semantic knowledge is directly relevant to the specific patterns in the numbers. Crucially, instead of mixing the text and numbers together in a way that might confuse the model's internal timing, NeST uses the text anchors to gently adjust the data. It scales and shifts the numerical features, allowing the language model to apply its learned wisdom without disrupting the natural rhythm of the time series. This process is efficient, requiring changes to only a small fraction of the model's settings while keeping the rest of its vast knowledge intact.

The results of this new approach are robust and consistent across a wide variety of tests. When tested on standard benchmarks for long-term forecasting, such as predicting electricity demand or traffic flow, NeST outperformed the current best methods, reducing prediction errors by an average of 1.2 percent. This might seem like a small number, but in the world of forecasting, even tiny improvements can lead to significant savings and better decision-making. The method also proved remarkably effective when data was scarce. In situations where the model had to learn from only a tiny fraction of the available data, or when it had to predict a new type of data it had never seen before, NeST maintained high accuracy, reducing errors by nearly 5 percent compared to other approaches.

Beyond the standard tests, the researchers applied NeST to a real-world challenge: forecasting the power output of distributed solar panels. Solar energy generation is notoriously difficult to predict because it depends on fluctuating weather conditions and varies across different locations. The system was tested on nine different datasets from solar stations across diverse geographical areas. In every case, NeST delivered the best overall performance, improving the accuracy of the predictions by 3.3 percent on average compared to existing methods designed specifically for solar energy. This suggests that the framework is not just a theoretical improvement but a practical tool capable of handling the messy, variable nature of real-world energy systems.

The success of NeST lies in its ability to respect the unique nature of both language and time. By preserving the geometric relationships within the language model's vocabulary and using them to guide the numerical data without forcing a rigid connection, the researchers created a system that is both flexible and precise. The visualizations of their work show that their method keeps the text anchors firmly rooted in the dense, meaningful regions of the language model's knowledge, unlike previous methods that allowed these anchors to drift into empty or irrelevant spaces. This careful alignment allows the model to leverage its deep understanding of patterns to make better predictions, even when the data is noisy or incomplete.

Ultimately, this work demonstrates that large language models can be effectively adapted for time series forecasting without losing their core strengths. The NeST framework offers a way to integrate textual knowledge into numerical prediction that is both computationally efficient and highly accurate. As the world generates more data and faces more complex forecasting challenges, methods like this provide a reliable path forward, proving that the tools we use to understand language can also help us understand the rhythms of the physical world. The findings suggest that with the right approach, the gap between text and time can be bridged, leading to more accurate predictions in fields ranging from energy management to economic planning.

Technical Summary: NeST – Neighborhood-aware Semantic Alignment and Temporal Modulation for LLM-based Time Series Forecasting

1. Problem Statement

Adapting Large Language Models (LLMs), which are pretrained on discrete text data, to forecast continuous time series signals presents significant challenges. While fine-tuning LLMs enables this adaptation, current methods face three primary limitations:

Ineffective Prompt Integration: Existing approaches often combine textual and time series modalities through simple concatenation or parameter-heavy cross-attention mechanisms. This creates a rigid coupling that may disrupt the intrinsic temporal structure of time series embeddings and offers limited control over how semantic priors interact with temporal features.
Suboptimal Prototype Learning: Current methods generate text prototypes using linear probing or fixed, pre-defined sets. These approaches often fail to preserve the geometric structure of the pretrained word token embedding space, leading to prototypes that may lie outside the valid manifold of the LLM. This is particularly detrimental in few-shot regimes where limited supervision can cause overfitting.
Inadequate Temporal Decomposition: Many methods rely on decomposition techniques (e.g., STL) to learn time series embeddings. However, these techniques may inadequately capture complex, non-stationary temporal dynamics common in real-world applications.

2. Methodology: The NeST Framework

The authors propose NeST (Neighborhood-aware Semantic Alignment and Temporal Modulation), a framework designed to formulate a new text-integrated time series prompt for fine-tuning LLMs. The architecture consists of five modules:

Module 1: Time Series Representation Learning

The input multivariate time series is partitioned into univariate channels. Each channel is normalized using Reversible Instance Normalization (RevIN) to mitigate distribution shifts. The data is then divided into patches (local time windows) using a patch embedding layer (1D convolution) followed by a linear layer to generate time series patch embeddings ( $P^{(i)}$ ) and a global input embedding ( $Z^{(i)}$ ).

Module 2: Neighborhood-aware Text Prototype Learning

Instead of learning prototypes solely via forecasting loss, NeST introduces a mechanism to preserve the semantic geometry of the LLM.

Initialization: A set of learnable text prototypes ( $E$ ) is initialized.
Optimization: Prototypes are optimized to act as centroids for local neighborhoods of the pretrained word token embeddings ( $W$ ). This is achieved by minimizing the Mean Squared Error (MSE) between each frozen word token and its nearest text prototype. This ensures prototypes remain within the dense regions of the pretrained embedding manifold, preserving semantic priors.

Module 3: Nearest-Neighbor Contrastive Learning (NNCL) based Alignment

To align the learned text prototypes with time series representations, NeST employs a Nearest-Neighbor Contrastive Learning objective.

Support Set: A First-In-First-Out (FIFO) queue ( $Q$ ) maintains a large set of text prototypes from previous batches to approximate the full distribution.
Contrastive Objective: For a given time series embedding $Z^{(i)}$ , the top- $k$ nearest-neighbor text prototypes are retrieved from the support set. A contrastive loss ( $L_{NNCL}$ ) is computed to pull the time series embedding closer to these relevant text prototypes, ensuring the prototypes are representative of the time series characteristics.

Module 4: Text Prototype Conditioned Temporal Modulation

This module integrates semantic information into the time series without altering the token sequence or introducing auxiliary tokens.

Aggregation: The top- $k$ retrieved text prototypes are aggregated into a global textual context vector ( $C$ ).
Modulation: A lightweight projection function maps $C$ to feature-wise scale ( $\gamma$ ) and shift ( $\beta$ ) parameters.
Application: These parameters are applied to the time series patch embeddings via residual interpolation: $P^{(i)*} = \gamma \odot P^{(i)} + \beta$ . This adaptively scales and shifts temporal features based on semantic context while preserving the intrinsic temporal structure.

Module 5: LLM Fine-tuning and Output

The modulated embeddings serve as the input prompt to the LLM. To maintain the pretrained prior and reduce computational overhead, NeST freezes the multi-head attention and feed-forward layers of the LLM. Only the positional embeddings and layer normalization parameters are fine-tuned. The LLM output is flattened and passed through a linear layer to generate the final forecast.

Total Loss: The optimization objective combines the forecasting loss ( $L_{forecast}$ ) with the prototype learning loss ( $L_{proto}$ ) and the contrastive alignment loss ( $L_{NNCL}$ ), weighted by a hyperparameter $\lambda$ .

3. Key Contributions

Neighborhood-aware Text Prototype Learning: A mechanism that learns text prototypes by explicitly preserving the local neighborhood structure within the pretrained LLM word token embedding space, acting as representative anchors for semantic regions.
NNCL-based Cross-modal Alignment: The introduction of a Nearest-Neighbor Contrastive Learning objective to align learned text prototypes with time series representations, utilizing a support set to enhance robustness in few-shot scenarios.
Prototype Conditioned Temporal Modulation: A feature-wise conditioning mechanism that injects textual semantics into time series representations via adaptive scaling and shifting, avoiding the rigid coupling and computational cost of token concatenation or cross-attention.
Comprehensive Empirical Evaluation: Demonstrated state-of-the-art performance across long-term, short-term, few-shot, and zero-shot forecasting benchmarks.
Real-world Validation: Successful application to distributed photovoltaic (PV) power forecasting across nine datasets, demonstrating robustness in renewable energy integration scenarios.

4. Experimental Results

The authors evaluated NeST on seven benchmark datasets for long-term forecasting (ETTh1, ETTh2, ETTm1, ETTm2, Weather, Traffic, and ECL), the M4 benchmark for short-term forecasting, and nine real-world PV datasets.

Long-term Forecasting: NeST achieved an average 1.2% reduction in MSE compared to the second-best state-of-the-art (SOTA) methods across the seven long-term forecasting benchmarks. It consistently outperformed baselines like S2TS-LLM, Time-LLM, and PatchTST.
Few-shot Forecasting: Using only 10% of training data, NeST showed strong performance, achieving the best results on three out of four ETT datasets and demonstrating robustness in data-scarce conditions.
Zero-shot Forecasting: In cross-domain transfer settings (training on one dataset, testing on another), NeST outperformed existing baselines in seven out of eight transfer scenarios, with MSE reductions ranging from 1.9% to 11.5% depending on the transfer pair.
Short-term Forecasting (M4 Benchmark): NeST achieved the best SMAPE (11.940) and MASE (1.607) among compared methods, showing competitive performance across various sampling intervals.
Real-world PV Forecasting: On distributed PV power datasets, NeST improved the average $R^2$ score by 4.9% on the first dataset (five stations) compared to the second-best method (an LLM-based approach specifically designed for PV forecasting) and by 1.2% on the second dataset (four stations).
Ablation Studies: Experiments confirmed that each component (Neighborhood-aware learning, NNCL, and Temporal Modulation) contributes positively to performance, with the most significant gains observed in few-shot settings.
Visualization: UMAP visualizations confirmed that NeST's prototypes remain embedded within the dense regions of the pretrained token manifold, unlike linear probing methods which produce isolated structures.

5. Significance and Claims

The paper claims that NeST addresses the critical gap in effectively integrating textual and time series modalities for LLM-based forecasting. By preserving the semantic geometry of the LLM and utilizing a parameter-efficient modulation strategy, NeST offers a generalizable framework that:

Enhances Generalization: Demonstrates strong zero-shot and few-shot capabilities, crucial for scenarios with limited labeled data.
Maintains Temporal Integrity: Avoids disrupting the intrinsic temporal structure of time series data through rigid token concatenation.
Offers Practical Utility: Proves effective not only on standard academic benchmarks but also in complex, real-world renewable energy forecasting tasks involving non-stationary and variable data.

The authors position NeST as a robust, practical framework for adapting LLMs to diverse time series forecasting tasks without requiring extensive architectural modifications or computationally expensive attention mechanisms.