💬 NLP

GALA: Generation-Aware Cross-Modal Alignment for Text-to-Time-Series Synthesis

GALA introduces a novel two-stage framework that achieves state-of-the-art text-to-time-series synthesis by employing generation-aware cross-modal alignment to create caption embeddings specifically optimized for guiding flow-matching generators, thereby simultaneously improving both signal fidelity and text adherence.

Original authors: Haochen Zhang, Gengwei Zhang, Laura Yao, Nicholas Knoz, Tianlong Chen

Published 2026-08-17

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Haochen Zhang, Gengwei Zhang, Laura Yao, Nicholas Knoz, Tianlong Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to paint. You could show it a thousand pictures of cats and say, "Draw a cat," but that's like giving it a rigid checklist. What if you wanted it to draw a "grumpy cat sitting in a sunbeam" or a "cat chasing a laser pointer"? That's the power of using natural language—words that flow like a story—to guide creation. In the world of data science, there's a similar challenge with "time series." These are just lists of numbers that change over time, like your heart rate during a run, the temperature outside every hour, or the stock market's daily swings. Scientists have been trying to teach computers to generate these number-lists based on text descriptions, like "a smooth, rising trend with a sudden spike." But here's the catch: the computers are often terrible at listening. They either ignore the specific details of the story or they make up numbers that look real but don't actually match the story. It's like a musician who can play a perfect note but can't follow the sheet music to play the right song.

This is where a new method called GALA comes in. Think of GALA as a super-smart translator and a strict coach rolled into one. The researchers behind this paper realized that the problem wasn't just the music (the time series) or the sheet music (the text), but how the two were being connected. They built a system that forces the computer to truly understand the relationship between the words and the numbers before it even tries to create anything. By doing this, GALA manages to create time series that not only look realistic but also follow the story you told it perfectly. It's a big step forward because it stops the computer from just guessing and starts it from a place of genuine understanding.

The Problem: The "Ghost" Connection

In the past, when scientists tried to make computers generate time series from text, they used a shortcut. They would take a text description (like "a heartbeat that speeds up") and run it through a standard text-reading program to get a "fingerprint" of the words. Then, they would feed that fingerprint into a generator to create the numbers. The problem was that this fingerprint was made for reading, not for creating. It was like handing a painter a description of a sunset written for a poet; the painter might understand the words, but they wouldn't know how to mix the paints to match the specific colors the poet meant.

The researchers found that existing methods had two main flaws. First, if they just used the text fingerprint as-is, the computer would make up numbers that looked okay but didn't match the story. Second, if they tried to train the text-reading program while it was making the numbers, the computer would get confused. It would either make the numbers look perfect but ignore the story, or follow the story so strictly that the numbers looked fake and broken. It was a lose-lose situation, like trying to walk and chew gum at the same time, but failing at both.

The Solution: The Two-Stage Dance

The authors of this paper, working at the University of North Carolina at Chapel Hill, proposed a new way to handle this called GALA (Generation-Aware cross-modaL Alignment). Instead of trying to do everything at once, they broke the process into two distinct stages, like a dance with a rehearsal and a performance.

Stage 1: The Rehearsal (Alignment)
In this first stage, the computer learns to speak the same language as the time series. They take a pre-trained text encoder (the "reader") and a time-series model (the "number expert") and force them to hang out together. But they don't just let them chat; they make them play a matching game.

The Contrastive Game: The computer is shown a text description and a time series. It has to learn that these two go together, and that they don't go with other random pairs. This is like a teacher showing a student a picture of a dog and saying, "This is a dog," and then showing a picture of a cat and saying, "This is not a dog."
The Secret Sauce (The Auxiliary Loss): Here is where GALA gets clever. The researchers realized that just matching the text and numbers isn't enough. The text fingerprint needs to contain enough information to actually build the numbers. So, they added a second task: they asked the text fingerprint to try and "rebuild" the time series on its own. If the text fingerprint was too vague, the rebuild would fail. This forced the computer to make the text fingerprint super-detailed and specific. It's like telling the painter, "Not only do you need to know what a sunset is, but you also need to be able to mix the paints to recreate it perfectly."

Stage 2: The Performance (Generation)
Once the text encoder has been "rehearsed" and aligned with the time series, it gets frozen (locked in place). Now, the computer uses this super-tuned text fingerprint to drive a generator. Because the fingerprint was trained to be "generation-aware," the generator knows exactly what to do. It doesn't have to guess or trade off between looking real and following the story; it just follows the instructions perfectly.

What They Found: A New Record

The team tested GALA on a massive dataset called TSFragment-600K, which contains over 600,000 pairs of text descriptions and time series data. They looked at four different types of data (energy, traffic, electricity, and general time series) and tested three different lengths of time (24, 48, and 96 steps).

The results were impressive. GALA didn't just do okay; it set a new state-of-the-art record.

The Scoreboard: Out of 36 different measurement columns (combining the four domains and three lengths), GALA came in first place in 30 of them.
The Rankings: When they averaged the rankings, GALA was nearly perfect, with an average rank of 1.08 for the shorter lengths and 1.42 for the longest. The next best method was around 1.92 to 2.00.
The Trade-off Broken: The most exciting finding was that GALA broke the old rule that you had to choose between "real-looking" data and "story-accurate" data. Previous methods had to sacrifice one for the other. GALA improved both at the same time. The data looked more realistic (better FID scores) and followed the text descriptions much more closely (better CTTP and JFTSD scores).

Why It Matters

The paper explicitly rules out the idea that you can just train the text encoder and the generator together in one big lump. Their experiments showed that doing that leads to a trade-off where you lose quality in one area to gain it in another. They also showed that simply using a frozen text encoder (one that never changes) isn't good enough because it doesn't understand the specific needs of generating numbers.

The key takeaway is that you need a dedicated "alignment" phase where the text and numbers are forced to understand each other before the generation happens. And crucially, that alignment phase needs to include a "generation test" to make sure the text fingerprint is detailed enough to actually build the numbers.

By separating the "learning to understand" phase from the "learning to create" phase, and by making sure the understanding is deep enough to build, GALA manages to do what previous methods couldn't: it creates time series that are both mathematically sound and perfectly obedient to the story you tell them. It's a reminder that sometimes, to get the best performance, you have to slow down, rehearse, and make sure your tools are truly ready before you start the show.

Technical Summary: GALA (Generation-Aware Cross-Modal Alignment for Text-to-Time-Series Synthesis)

Problem Statement

The paper addresses the challenge of text-conditional time series generation, where the goal is to synthesize realistic time series data ( $X$ ) based on free-form natural language descriptions ( $C$ ). While unconditional generation and generation conditioned on structured signals (e.g., class labels) are established, natural language offers superior flexibility and semantic expressiveness for describing time series morphology (e.g., clinical reports for ECGs).

The authors identify a critical gap in existing text-to-time-series methods (e.g., T2S, DiffuSETS, VerbalTS, BRIDGE): the conditioning representation is either frozen from a text-only encoder or adapted solely through the generation objective (e.g., denoising loss). In neither case is the text embedding explicitly aligned with the time-series modality. Consequently, the conditioning signal is sub-optimized for generation, often forcing a trade-off between fidelity (realism of the generated series) and caption adherence (how well the series matches the text description).

Methodology: GALA

GALA proposes a two-stage approach to decouple cross-modal alignment from the generation process, creating a "generation-aware" shared embedding space.

Stage 1: Cross-Modal Alignment

The objective is to learn a shared embedding space where text and time series are aligned, but specifically optimized for synthesis rather than just retrieval.

Encoders:
- Text Encoder: Uses Embedding-Gemma-300M. The backbone is frozen, adapted via LoRA on attention projections.
- Time-Series Encoder: Uses Chronos-2. The backbone is frozen, adapted via LoRA.
- Magnitude Preservation: A critical modification replaces Chronos-2's instance normalization (which discards absolute scale) with a dataset-level global z-score normalization. This ensures the embedding retains absolute level and scale information often referenced in captions.
Loss Functions:
1. Contrastive Loss ( $L_{align}$ ): A symmetric InfoNCE loss aligns the text and series embeddings in a shared hypersphere.
2. Auxiliary Generative Loss ( $L_{gen}$ ): A lightweight conditional denoiser ( $g_\xi$ ), conditioned only on the text embedding, is trained to synthesize the paired time series. This loss forces the text embedding to contain sufficient information to reconstruct the series, ensuring the alignment is "generation-aware."
Optimization: The total loss is $L_{stage-1} = L_{align} + \lambda_{dw} E[L_{gen}]$ . The encoders are updated via LoRA, while the auxiliary denoiser is discarded after this stage.

Stage 2: Conditional Generation

Generator: A flow-matching generator (DiT-style diffusion transformer) operates in the observation space.
Conditioning: The text encoder is frozen after Stage 1. Its output embedding serves as the sole conditioning signal for the generator.
Process: Time series are mapped to square images via delay embedding, generated by the flow-matching model, and mapped back to time series.

Key Contributions

Identification of a Blind Spot: The authors demonstrate that current methods fail to deliberately match conditioning representations to the signal modality, leading to under-optimized signals.
GALA Framework: A two-stage method that decouples alignment from generation. It introduces a generation-aware auxiliary loss to the alignment stage, ensuring embeddings are not just discriminative (good for retrieval) but generative (sufficient for synthesis).
Theoretical Justification: Using an information-theoretic decomposition of the KL divergence, the paper argues that minimizing representation error ( $R(E)$ ) requires maximizing mutual information $I(X; U)$ . Contrastive loss alone saturates once negatives are ranked; the auxiliary generative loss provides the necessary pressure to retain fine-grained reconstruction details.
Empirical Validation: Extensive experiments on TSFragment-600K (4 domains: ETTh1s, ETTm1, Electricity, Traffic; 3 lengths: 24, 48, 96).

Experimental Results

Evaluated on TSFragment-600K against five baselines (VerbalTS, T2S, DiffuSETS, BRIDGE, Text2Motion) using FID (fidelity), CTTP (caption adherence), and JFTSD (joint fidelity/adherence).

State-of-the-Art Performance: GALA ranks first in 30 out of 36 metric columns.
- Average Rank: 1.08 (L=24), 1.08 (L=48), 1.42 (L=96), compared to the strongest baseline (VerbalTS) at 1.92/2.00/1.75.
- Caption Adherence: GALA achieves the highest CTTP in all 12 dataset-length settings and the lowest JFTSD in 11 of 12.
- Fidelity: GALA achieves the best FID in 7 of 12 columns and remains competitive elsewhere.
Ablation Insights:
- Two-Stage vs. One-Stage: A dedicated alignment stage significantly outperforms training alignment and generation jointly.
- Necessity of Auxiliary Loss: Removing the auxiliary generative loss ( $\lambda_{dw}=0$ ) degrades both fidelity and adherence, confirming the loss is a necessary component, not an add-on.
- Backbone Adaptation: The auxiliary loss only works if the encoder backbones are adaptable (via LoRA or full fine-tuning); a frozen backbone with a trainable head fails to improve performance.
- Magnitude Preservation: Global z-score normalization significantly improves caption adherence (CTTP/JFTSD) with negligible impact on FID compared to instance normalization.
- Text Encoder Tuning: Adapting the text encoder is the primary driver of caption adherence gains; adapting the time-series encoder provides marginal additional benefits.

Significance and Claims

The paper claims that GALA breaks the traditional trade-off between fidelity and caption adherence in text-to-time-series generation. By explicitly aligning the text embedding to the time-series modality using a generation-aware objective, the method produces conditioning signals that are both semantically faithful and sufficient for high-quality synthesis.

The authors emphasize that their approach is not merely a plug-in of existing cross-modal alignment techniques (which are often optimized for retrieval/ranking) but a tailored solution where the alignment is explicitly shaped by the requirements of the generative task. The results suggest that "generation-aware" alignment is a necessary component for high-performance text-conditional time series synthesis, particularly when dealing with free-form natural language descriptions.