Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative

This paper introduces Texts as Time Series (TaTS), a novel framework that leverages the periodic alignment between paired texts and time series data to enhance multimodal forecasting and imputation performance in existing numerical-only models without requiring architectural changes.

The Gist

Imagine you are trying to predict the weather. You have a thermometer that gives you a number every hour (the temperature). That's your time series. But you also have a weatherman's daily radio broadcast that describes the sky, the wind, and the pressure systems. That's your text.

For a long time, computer scientists building AI to predict the future have mostly ignored the weatherman's voice. They only looked at the numbers on the thermometer, thinking, "If I just crunch these numbers hard enough, I'll get the answer."

This paper, titled "Language in the Flow of Time," argues that we've been ignoring a huge clue. It introduces a new way to teach computers to listen to the weatherman while they look at the thermometer.

Here is the breakdown of their discovery and solution, using simple analogies:

1. The Big Discovery: "The Rhythm of Words"

The authors noticed something fascinating they call Chronological Textual Resonance (CTR).

Think of a time series (like stock prices or traffic jams) as a song with a specific beat. It has a rhythm: maybe it goes up every morning and down every night, or it has a yearly cycle like the seasons.

The authors found that the text associated with that data often sings the same song.

• The Analogy: Imagine a stock market crash. The numbers on the screen drop sharply. At the exact same time, the news headlines scream "CRISIS!" and "PANIC!"
• If you analyze the "rhythm" of those headlines, you'll find they pulse in sync with the stock market. When the market is calm, the news is calm. When the market is chaotic, the news is chaotic.

The text isn't just random noise; it has a hidden periodic pattern that mirrors the numbers. It's like the text is a shadow dancing to the same music as the numbers.

2. The Problem: The "Silent" Computer

Current AI models are like musicians who can only read sheet music (numbers) but are deaf to the lyrics (text).

• If you feed them only numbers, they miss the context.
• If you try to feed them text using old methods, the computer gets confused because it doesn't know how to line up the "words" with the "numbers." It's like trying to mix oil and water; they don't blend well.

3. The Solution: "Texts as Time Series" (TaTS)

The authors propose a clever trick called TaTS. Instead of trying to force the text to fit into a language model, they treat the text as if it were just another number.

• The Analogy: Imagine you are baking a cake. You have flour, sugar, and eggs (your original numbers). You also have a secret ingredient: a jar of "flavor notes" (the text).
• Instead of trying to eat the jar of notes separately, TaTS turns those notes into a liquid extract and pours them right into the batter.
• Now, your "batter" (the data) has both the original ingredients and the flavor notes mixed in perfectly.

How it works technically (in simple terms):

1. Translate: They take the text and turn it into a list of numbers (embeddings) using a smart language model (like GPT).
2. Shrink: They squeeze those long lists of numbers down into a smaller, manageable size (like compressing a file).
3. Mix: They stick these "text numbers" right next to the original "time numbers" to create a super-charged data stream.
4. Predict: They feed this super-charged stream into any existing time-series AI model. The model doesn't even know it's looking at text; it just thinks it's looking at a new, helpful variable.

4. The Results: Why It Matters

The authors tested this on real-world data:

• Economy: Mixing GDP numbers with economic news reports.
• Traffic: Mixing traffic flow numbers with traffic reports.
• Health: Mixing patient vitals with medical notes.

The Outcome:
By adding the "text flavor" to the "number batter," the AI models became significantly better at predicting the future.

• In some cases, the accuracy improved by over 30%.
• It worked with almost every type of existing AI model they tried, without needing to rebuild the models from scratch. It's a "plug-and-play" upgrade.

5. The "Magic Metric" (TT-Wasserstein)

The paper also invented a new ruler called TT-Wasserstein.

• The Analogy: Imagine you have a dance partner (the text) and a lead dancer (the numbers). This ruler measures how well they are dancing together.
• If the ruler shows a low score, it means the text and numbers are perfectly in sync (great resonance).
• If the score is high, they are dancing out of step (the text is just noise).
• This helps researchers know before they start: "Hey, this dataset has great text to use!" or "This text is garbage, don't bother."

Summary

This paper is like telling a detective: "You've been solving crimes by only looking at the footprints (numbers). But you're ignoring the witness statements (text) that are actually shouting the same story in a different voice. If you learn to listen to both voices together, you'll solve the case much faster and more accurately."

They didn't need to invent a new detective; they just taught the old ones how to listen to a second voice. And it worked beautifully.

Technical Summary

Here is a detailed technical summary of the paper "Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative" (TaTS), published at ICLR 2026.

1. Problem Statement

While time series modeling has advanced significantly for numerical data, the integration of multimodal time series—specifically those paired with contextual textual information at each timestamp—remains underexplored.

• Current Limitations: Existing approaches often treat time series and text as separate modalities or ignore the unique positional and periodic characteristics inherent to time-series-paired texts.
• The Gap: Numerical-only models miss valuable contextual explanations (e.g., government announcements during pandemic infection rates), while current multimodal methods often fail to leverage the specific temporal dynamics shared between the text and the numbers.
• Core Question: What unique attributes characterize time-series-paired texts, and how can they be systematically integrated to improve forecasting and imputation without redesigning existing models?

2. Key Insight: Chronological Textual Resonance (CTR)

The authors identify a novel phenomenon termed Chronological Textual Resonance (CTR).

• Observation: Time-series-paired texts often exhibit periodic patterns that closely mirror the periodicity of their corresponding numerical time series. For example, in monthly economic data, the accompanying text embeddings show a dominant frequency of 12 (annual cycle), matching the time series.
• Theoretical Basis: This is grounded in the Platonic Representation Hypothesis (PRH), which posits that different modalities describing the same underlying event converge to a shared latent space. Since texts often evolve in response to numerical trends (or share external drivers like seasons), their hidden representations inherit similar temporal dynamics.
• Quantification: The authors propose TT-Wasserstein, a metric based on the Wasserstein distance between the normalized frequency spectra of the time series and the text embeddings. A lower TT-Wasserstein score indicates higher alignment (stronger CTR) and predicts better model performance.

3. Methodology: Texts as Time Series (TaTS)

The authors propose TaTS, a plug-and-play framework that transforms paired texts into auxiliary variables, allowing them to be seamlessly integrated into any existing numerical-only time series model.

Workflow:

1. Text Encoding: Paired texts $S = \{s_1, ..., s_T\}$ are encoded into embeddings $E = \{e_1, ..., e_T\}$ using a pre-trained Large Language Model (e.g., GPT-2, BERT, LLaMA).
2. Dimensionality Reduction: Since text embeddings ($d_{text}$) are high-dimensional, a Multi-Layer Perceptron (MLP) projects them into a lower-dimensional space ($d_{mapped}$) to match the scale of time series variables.
   $$z_t = \text{MLP}(e_t)$$
3. Augmentation: The projected text embeddings $Z$ are treated as auxiliary variables and concatenated with the original numerical time series $X$ to form a unified multimodal sequence $U$.
   $$U = [X; Z^\top]$$
4. Unified Modeling: The augmented sequence $U$ is fed into an existing time series forecasting or imputation model (e.g., iTransformer, PatchTST, DLinear). The model learns to capture both numerical and textual dynamics jointly.
5. Training: The framework is trained end-to-end using standard loss functions (e.g., MSE) for forecasting or imputation tasks.

Key Design Features:

• Modularity: It does not require modifying the architecture of the backbone time series model.
• Compatibility: Works with Transformer-based, linear, and frequency-based models.
• Robustness: The MLP projection allows the model to learn the relevance of text, effectively down-weighting noisy or irrelevant textual inputs.

4. Key Contributions

1. Discovery of CTR: Uncovered that time-series-paired texts exhibit periodic patterns aligned with numerical data, validating the Platonic Representation Hypothesis in a temporal context.
2. TT-Wasserstein Metric: Introduced a new metric to quantify the alignment quality between text and time series, serving as a diagnostic tool for dataset quality and potential model improvement.
3. TaTS Framework: Proposed a simple, effective, and architecture-agnostic method to integrate text as auxiliary variables, bridging the gap between unimodal and multimodal time series modeling.
4. State-of-the-Art Performance: Demonstrated that TaTS significantly outperforms numerical-only baselines and existing multimodal libraries (MM-TSFLib) across diverse datasets.

5. Experimental Results

The framework was evaluated on 18 real-world datasets (including Time-MMD, FNSPID, and FNF) covering domains like Economy, Climate, Traffic, Health, and Finance.

• Forecasting Performance:
  • TaTS consistently achieved the best performance across 9 different time series models (e.g., iTransformer, PatchTST, Autoformer).
  • Improvements: Achieved an average performance improvement of >5% on 6 out of 9 datasets. On the "Environment" dataset, it delivered a remarkable >30% reduction in MSE.
  • Comparison: Outperformed specialized multimodal baselines like MM-TSFLib, ChatTime (zero-shot), and GPT4MTS.
• Imputation Performance:
  • TaTS significantly enhanced imputation capabilities, reducing errors by up to 30% compared to baselines on datasets like Climate and Economy.
• Ablation Studies:
  • CTR Correlation: Lower TT-Wasserstein scores (stronger alignment) correlated with higher performance gains.
  • Robustness: The model remained effective even when 25% of texts were randomly dropped.
  • Noise Handling: When texts were shuffled (destroying alignment), performance dropped to baseline levels, confirming that gains come from genuine temporal resonance, not just noise.
  • Efficiency: TaTS introduces only a ~1% increase in parameters and ~8% increase in training time, yet yields a ~14% improvement in forecasting accuracy.

6. Significance and Impact

• Paradigm Shift: Moves beyond treating text as a static covariate or a separate stream, instead recognizing it as a dynamic, periodic variable intrinsic to the time series flow.
• Practical Utility: The "plug-and-play" nature of TaTS allows researchers and practitioners to upgrade existing numerical models to handle multimodal data with minimal engineering effort.
• Scalability: The method is computationally efficient and compatible with the latest large language models, making it suitable for real-world applications in finance, healthcare, and climate science where textual context is abundant.
• Future Direction: Opens avenues for exploring latent communication between time series models and language models to build general time-series intelligence.

In summary, TaTS successfully leverages the Chronological Textual Resonance to unify text and time series, proving that treating text as an auxiliary time-series variable is a powerful, efficient, and generalizable strategy for multimodal forecasting.