From Text to Forecasts: Bridging Modality Gap with Temporal Evolution Semantic Space

The paper proposes TESS, a novel framework that bridges the modality gap in event-driven time-series forecasting by introducing a Temporal Evolution Semantic Space to translate qualitative textual descriptions into interpretable, quantitative temporal primitives, achieving significant error reductions over state-of-the-art baselines.

The Gist

The Big Problem: The "Language Barrier" Between Numbers and Words

Imagine you are trying to predict the weather for next week. You have two sources of information:

1. The Numbers: A spreadsheet showing yesterday's temperature, wind speed, and humidity. This is precise, but it only tells you what has happened.
2. The Words: A news report saying, "A massive storm front is moving in, causing chaos and sudden drops in temperature." This tells you why things might change, but it's vague. It uses words like "massive" and "chaos" instead of specific numbers.

The Problem:
Current AI models are great at reading numbers and great at reading words, but they are terrible at combining them.

• When the AI reads the news, it gets distracted by the fluff (e.g., "The sky looked gray and ominous").
• It struggles to translate "chaos" into a specific number like "wind speed will increase by 20 mph."
• Because of this confusion, the AI often ignores the helpful news or misinterprets it, leading to bad predictions.

The Solution: TESS (The "Translator" and "Filter")

The authors propose a new system called TESS. Think of TESS not as a direct translator, but as a specialized interpreter who sits between the news reporter and the weather forecaster.

Instead of letting the forecaster read the messy news article directly, TESS does two things:

1. The "Extraction" (The Translator)

TESS uses a powerful Large Language Model (like a super-smart human editor) to read the news article. But instead of just summarizing it, it forces the AI to extract only the four specific types of "time-travel clues" that actually matter for predicting the future:

• Mean Shift: "Is the average going up or down?" (e.g., "Prices will jump.")
• Volatility: "Is it going to be calm or chaotic?" (e.g., "Expect wild swings.")
• Shape: "What is the pattern?" (e.g., "It will go up, peak, and then crash.")
• Lag: "When will it happen and how long will it last?" (e.g., "The effect starts tomorrow and fades by Friday.")

The Analogy: Imagine the news article is a messy pile of LEGOs. The old AI tried to build a tower with the whole pile. TESS sorts the pile first, throwing away the broken pieces and the red bricks, and only handing the forecaster the specific blue and yellow bricks needed to build the next level of the tower.

2. The "Confidence Gate" (The Filter)

Sometimes, the news is vague, or the AI interpreter makes a mistake. Maybe the article says "prices might rise," but the AI guesses "prices will crash."

TESS has a Confidence Gate. It asks: "How sure are we about this clue?"

• If the AI is 90% sure the text means "prices will rise," the gate opens wide, and the forecaster uses that clue.
• If the AI is only 40% sure (because the text was confusing), the gate closes down, and the forecaster mostly ignores that clue to avoid making a mistake.

How It Works in Real Life

The paper tested this on real-world data, like Bitcoin prices and electricity usage.

• Without TESS: The AI reads a headline like "Market panic causes sell-off." It gets confused by the word "panic" and the surrounding fluff. It might predict a small drop when the market actually crashes.
• With TESS:
  1. TESS reads the headline.
  2. It extracts: Mean Shift = Strong Drop, Volatility = High, Lag = Immediate.
  3. It checks its confidence: "High confidence."
  4. It passes these clean, numerical-style clues to the forecasting model.
  5. The model predicts a sharp drop accurately.

The Results: Why It Matters

The researchers found that by using this "intermediate translator" (the Temporal Evolution Semantic Space):

• Accuracy jumped: They reduced prediction errors by up to 29%. That's a huge deal in finance or energy management.
• Less confusion: The model stopped getting distracted by irrelevant words in the news.
• Better handling of surprises: When unexpected events happen (like a storm or a stock market crash), TESS is much better at adjusting the forecast because it understands the story behind the numbers.

Summary Analogy

Imagine you are driving a car (the forecasting model).

• The Old Way: A passenger (the text) is shouting a long, rambling story about traffic ahead. You try to drive while listening, but you get distracted by the story's details and miss the actual turns.
• The TESS Way: A professional navigator (TESS) listens to the passenger, filters out the noise, and hands you a simple, clear map with only the essential instructions: "Turn left in 2 miles, speed up, then slow down." You drive smoothly and arrive exactly on time.

In short: TESS bridges the gap between the messy world of human language and the precise world of numbers, ensuring that stories actually help us predict the future rather than confusing us.

Technical Summary

1. Problem Statement

Time-series forecasting is critical for domains like finance and energy, but it faces significant challenges due to event-driven non-stationarity. While historical numerical data captures temporal dependencies, it often fails to predict sudden regime shifts caused by exogenous events (e.g., accidents, policy changes, sentiment shocks).

Recent approaches attempt to incorporate textual data (news, social media) to mitigate this. However, a fundamental modality gap exists:

• Numerical Data: Explicit, quantitative, and chronologically ordered.
• Textual Data: Implicit, qualitative, and unstructured. Text describes impacts diffusely (e.g., "significant rise") rather than quantitatively (e.g., "+15.3%").

Key Findings from Preliminary Analysis:
The authors conducted controlled semi-synthetic experiments revealing two critical bottlenecks in existing multimodal fusion methods:

1. Attention Distraction: Models fail to focus on prediction-relevant tokens, instead over-attending to redundant and noisy textual content.
2. Representational Mismatch: Even when redundant tokens are removed, models struggle to decode qualitative text semantics into usable quantitative signals, leading to optimization instability and poor forecasting performance.

2. Methodology: TESS Framework

To bridge this gap, the authors propose TESS (Temporal Evolution Semantic Space), a two-stage framework that acts as an intermediate bottleneck between text and numerical time series. Instead of fusing raw text embeddings directly with time series, TESS distills text into interpretable, numerically grounded Temporal Semantic Primitives (TSPs).

A. Temporal Evolution Semantic Space (The Intermediate Representation)

The core innovation is defining a space of four specific primitives that characterize temporal dynamics, inspired by expert forecasting practices:

1. Mean Shift: Quantifies changes in the average level (e.g., Strong Rise, Mild Drop).
2. Volatility Shift: Quantifies changes in variance/fluctuation intensity.
3. Shape: Encodes the morphological trend of the forecast horizon (e.g., Ascend, Peak, Oscillate).
4. Lag & Decay: Localizes the timing and persistence of an event's impact (e.g., Early Fade, Late Manifestation).

These primitives are defined with numerical verifiability, meaning ground-truth values can be calculated from the actual time series data, enabling supervised learning.

B. Stage 1: Text $\to$ Semantic Space (LLM Extraction with Gating)

• LLM Classification: A frozen Large Language Model (LLM) is prompted with structured definitions to classify the input text into the discrete categories of the four primitives.
• Confidence-Aware Gating: Since LLMs can hallucinate or misinterpret ambiguous text, a learnable gating network estimates the reliability of each extracted primitive.
  • It uses the log-probability margin between the top-1 and top-2 LLM predictions as an uncertainty signal.
  • A gating score $g_{t,k} \in [0,1]$ is generated. During inference, the semantic embedding is weighted by this score ($\tilde{h} = g \cdot h$), suppressing unreliable extractions.

C. Stage 2: Semantic Space $\to$ Numerical Space (Conditioned Forecasting)

• The gated primitive vectors are treated as semantic prefix tokens and concatenated with the patch embeddings of the historical time series.
• This combined sequence is fed into a Transformer-based forecaster (specifically PatchTST).
• The model learns to condition its numerical predictions on these explicit temporal evolution signals, effectively grounding the semantic information in the observed dynamics.

3. Key Contributions

1. Diagnosis of the Modality Gap: The paper provides empirical evidence that existing methods fail due to attention distraction by redundancy and the inability to translate qualitative text into quantitative signals.
2. TESS Framework: Introduction of an intermediate "Temporal Evolution Semantic Space" that decomposes text into four interpretable primitives (Mean, Volatility, Shape, Lag).
3. Confidence-Aware Gating: A novel mechanism to filter noisy LLM extractions, ensuring that only high-confidence semantic signals influence the forecast.
4. Theoretical Guarantee: Theorem 4.1 proves that compressing text into sufficient statistics (primitives) preserves predictive information while reducing dependence on token-level noise, theoretically improving generalization.

4. Experimental Results

The authors evaluated TESS on four real-world datasets: FNSPID and Bitcoin (Financial/High Non-stationarity) and Electricity and Environment (General/Low Non-stationarity).

• Performance Gains: TESS achieved up to a 29.1% reduction in MSE compared to state-of-the-art unimodal and multimodal baselines (e.g., PatchTST, TimeLLM, NewsForecasting).
  • On the Bitcoin dataset, it improved MAE by 18.2% and MSE by 29.1% over the best baseline.
  • On FNSPID, it showed consistent improvements across all metrics.
• Non-Stationary Scenarios: TESS demonstrated superior performance specifically in scenarios involving shape transitions, volatility changes, and mean shifts, outperforming baselines by 21–52% in MSE.
• Ablation Studies:
  • Removing the Semantic Space (TESS) caused a massive performance drop (MSE increased by ~46% on Bitcoin), confirming the necessity of the intermediate representation.
  • Removing the Gating mechanism led to smaller but significant drops, proving its role in filtering LLM errors.
• Convergence: TESS showed faster and smoother training convergence compared to direct text fusion methods, indicating the semantic space alleviates cross-modal optimization challenges.

5. Significance

This work addresses a critical limitation in multimodal time-series forecasting: the direct fusion of qualitative text and quantitative data is often suboptimal due to semantic misalignment. By introducing an interpretable, numerically grounded intermediate space, TESS effectively translates "human-like" event descriptions into "machine-readable" statistical conditions.

The approach is significant because:

• It moves beyond simple sentiment analysis or keyword extraction to structural temporal reasoning.
• It provides a robust solution to the "noise" problem in LLM-based text extraction via the confidence gating mechanism.
• It offers a generalizable framework for integrating exogenous textual information into any time-series forecasting model, particularly in volatile, event-driven environments like finance and energy markets.