🤖 machine learning

FADTI: Fourier and Attention Driven Diffusion for Multivariate Time Series Imputation

The paper introduces FADTI, a diffusion-based framework for multivariate time series imputation that enhances generalization under high missing rates and distribution shifts by integrating a learnable Fourier Bias Projection module with self-attention and gated convolution to inject explicit frequency-domain inductive biases.

Original authors: Runze Li, Hanchen Wang, Wenjie Zhang, Binghao Li, Yu Zhang, Xuemin Lin, Ying Zhang

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Runze Li, Hanchen Wang, Wenjie Zhang, Binghao Li, Yu Zhang, Xuemin Lin, Ying Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to finish a jigsaw puzzle, but someone has ripped out huge chunks of the picture. Some pieces are missing randomly (like a few scattered stars), while other times, entire sections of the sky are gone. Your goal is to guess what those missing pieces looked like based on the clues you have left.

This is exactly what FADTI does, but instead of a puzzle, it's fixing broken data streams (like weather reports, heart rates, or traffic sensors) where information is missing.

Here is how the paper explains this solution in simple terms:

The Problem: The "Broken Record" Effect

Most current computer programs try to fill in missing data by looking at the immediate neighbors. It's like trying to guess the next word in a sentence by only looking at the word right before it.

The Issue: If a sensor breaks for a long time (a "long gap"), these programs get confused. They often try to fill the gap with a flat line or a random guess, which ruins the natural rhythm of the data.
The Frequency Problem: Some programs try to look at the "music" of the data (its patterns and cycles). But if you try to analyze the music of a song that has huge chunks of silence, the computer gets confused. It thinks the silence is part of the song, creating a distorted, noisy sound.

The Solution: FADTI (The "Musical Architect")

The authors built a new system called FADTI (Fourier and Attention Driven Diffusion). Think of it as a smart architect who doesn't just look at the bricks next to the hole but listens to the entire song the building is supposed to be singing.

Here are the three main tricks FADTI uses:

1. The "Ghost Ear" (Fourier Bias Projection)

This is the paper's biggest innovation.

How it works: Instead of trying to analyze the broken, silent parts of the data directly (which causes distortion), FADTI looks at the clean parts of the data to learn the "rhythm" or "frequency" of the whole story.
The Analogy: Imagine you are trying to finish a song, but the microphone broke in the middle. A normal computer tries to guess the missing notes by looking at the silence. FADTI, however, listens to the rest of the song to understand the melody's "shape." It then projects that "shape" (the frequency bias) onto the missing part as a guide. It tells the computer: "Don't just guess; remember that this song usually goes up and down in a smooth wave, so fill the gap with that wave."
The Tools: They use three different "ears" to listen to this rhythm:
- DFT: Listens to the whole song at once (good for steady rhythms).
- STFT: Listens to short clips of the song (good for rhythms that change over time).
- FrSST: A super-sharp ear that can hear complex, wobbly rhythms that other ears miss.

2. The "Attention Team" (Self-Attention)

While the "Ghost Ear" handles the rhythm, FADTI also uses an Attention Mechanism.

The Analogy: Imagine a team of detectives. Some detectives look at the timeline (what happened yesterday vs. today), while others look at the relationships between different variables (e.g., if the temperature goes up, does the humidity go down?).
FADTI combines these detectives. It looks at the long-term history and the relationships between different sensors simultaneously to make a smart guess.

3. The "Denoising Sculptor" (Diffusion Model)

Finally, FADTI uses a Diffusion Model.

The Analogy: Imagine a sculptor starting with a block of noisy, static-filled clay. The sculptor doesn't just carve the final shape instantly. Instead, they slowly chip away the noise, step by step, refining the statue until the missing pieces emerge clearly.
FADTI starts with a random guess for the missing data and slowly "denoises" it, using the rhythm guide (from step 1) and the detective clues (from step 2) to carve out the most accurate shape possible.

Why It's Better (The Results)

The paper tested FADTI on several real-world datasets, including:

Energy data (power usage).
Weather data (temperature, humidity).
Traffic data (car speeds).
Yeast data (a new biological dataset tracking cell growth).

The findings:

Accuracy: FADTI filled in the missing pieces more accurately than any other method, especially when large chunks of data were missing (up to 50% of the data gone!).
Handling Chaos: It was particularly good at handling "non-stationary" data—data that changes its rhythm over time (like a heartbeat speeding up or a storm changing direction).
Efficiency: Even though it does a lot of complex math, it can generate high-quality results with fewer "guesses" (samples) than previous methods, making it faster in practice.

Summary

In short, FADTI is a new way to fix broken data. Instead of just looking at the immediate neighbors or getting confused by silence, it listens to the "music" of the entire dataset to understand the underlying rhythm. It then uses that rhythm to guide a step-by-step sculpting process, filling in the missing gaps with a shape that fits perfectly with the rest of the story.

Technical Summary: FADTI – Fourier and Attention Driven Diffusion for Multivariate Time Series Imputation

1. Problem Definition

Multivariate Time Series Imputation (MTSI) addresses the pervasive issue of missing values in domains such as healthcare, traffic forecasting, and biological modeling. These gaps arise from sensor failures, communication losses, and irregular sampling. While existing methods, particularly Transformer-based models (e.g., SAITS) and diffusion-based models (e.g., CSDI), have improved temporal dependency modeling and uncertainty estimation, they face a critical limitation: frequency distortion.

Current approaches often rely heavily on time-domain modeling. When applying frequency-domain techniques directly to masked or zero-filled inputs, the missing positions are treated as real signal values. This naive projection distorts the estimated spectrum, causing frequency features to be dominated by missingness artifacts rather than underlying temporal dynamics. Consequently, recovering long-range periodic patterns or structured temporal gaps under severe sparsity remains challenging.

2. Methodology: The FADTI Framework

The authors propose FADTI (Fourier and Attention Driven Diffusion), a conditional diffusion framework designed to integrate frequency-domain inductive bias into the denoising process without directly estimating spectra from corrupted inputs.

Core Architecture

FADTI follows a conditional diffusion pipeline where the reverse process is conditioned on observed values ( $x^{co}_0$ ) and the observation mask ( $m$ ). The denoising network consists of stacked residual blocks that combine three key components:

Temporal-Frequency Modeling: The core innovation, the Fourier Bias Projection (FBP) module.
Feature Attention: Captures cross-variable dependencies.
Gated Filtering: Updates residual representations efficiently.

The Fourier Bias Projection (FBP) Module

Unlike direct Fourier projection on masked inputs, FBP injects learnable frequency-aware bias into the intermediate hidden states of the denoising network. The process operates as follows:

Decomposition: Intermediate hidden states ( $H$ ) are separated into a smooth trend ( $H_{trend}$ ) and a high-frequency residual ( $H_{res}$ ) using 1D average pooling.
Spectral Transformation: A spectral operator $F_\nu(\cdot)$ $F_{ν} (\cdot)$ is applied channel-wise to these components. The paper instantiates this with three transforms:
- DFT (Discrete Fourier Transform): Captures global periodicity and stable trends.
- STFT (Short-Time Fourier Transform): Captures localized, time-varying frequency patterns.
- FrSST (Synchrosqueezing-based Short-Time Fractional Fourier Transform): Sharpens non-stationary time-frequency structures.
Bias Injection: The complex spectral coefficients are projected onto fixed cosine and sine bases to retain magnitude-phase information. These are mapped back to the temporal feature space to form a learnable bias ( $H_{freq}$ ).
Integration: This frequency-aware bias is fused with the original features and fed into temporal self-attention or gated dilated convolutions, guiding the network to preserve smooth trends and periodic structures.

Training and Inference

Training: The model is trained using a masked denoising score matching objective. A conditional mask ( $M^{co}$ ) is sampled from the observation mask to reveal values to the model, while a target mask ( $M^{ta}$ ) hides originally observed values to provide supervision. The loss is computed only on the supervised target entries.
Inference: Missing entries are initialized with Gaussian noise. The model iteratively applies the reverse diffusion process, conditioning on the fixed observed values, to generate probabilistic imputations.

3. Key Contributions

The paper outlines four primary contributions:

FADTI Framework: A frequency-aware conditional diffusion framework that integrates Fourier-domain inductive bias directly into the denoising process, addressing the challenge of structured missing patterns.
Flexible FBP Module: A novel module supporting DFT, STFT, and FrSST instantiations. This allows adaptive modeling of both stationary and non-stationary temporal structures without relying on corrupted spectra from masked inputs.
Efficient Temporal Modeling: The incorporation of a lightweight local modeling module (gated convolution) to complement global attention dependencies, reducing computational overhead.
Comprehensive Evaluation: Extensive experiments on multiple benchmarks, including a newly introduced biological time series dataset (Yeast), demonstrating consistent improvements in accuracy, uncertainty estimation, and sampling efficiency.

4. Experimental Results

The authors evaluated FADTI on three real-world datasets: ETT (Energy), Weather (Meteorology), and Yeast (Biology), under both point-wise and time-wise missing patterns at 10% and 50% missing rates.

Accuracy (MAE): FADTI achieved the lowest Mean Absolute Error (MAE) in most settings. Notably, it showed significant gains over baselines like CSDI, MTSCI, and SAITS, particularly under high missing rates (50%) and on the Yeast dataset.
Uncertainty (CRPS): FADTI demonstrated superior calibration and sharpness, achieving the lowest Continuous Ranked Probability Score (CRPS) in most cases, indicating better probabilistic quality.
Robustness: The model maintained strong performance under time-wise missing patterns (contiguous gaps), where many baselines degraded significantly.
Sampling Efficiency: FADTI achieved better accuracy and CRPS with fewer generated samples ( $K$ ) compared to CSDI and MTSCI. For instance, FADTI with $K=2$ outperformed CSDI and MTSCI with $K=28$ on the ETT dataset.
Ablation Studies:
- Removing FBP (using simple back-projection) resulted in performance drops, confirming that coupling bias with frequency projection is critical.
- DFT-based variants performed best on regular patterns (ETT, Weather), while STFT and FrSST variants were more competitive for time-wise missing gaps requiring localized spectral analysis.
Zero-Shot Forecasting: In a zero-shot imputation-to-forecasting experiment on Yeast, FADTI outperformed deterministic baselines and the diffusion-based MTSCI, showing strong extrapolation capabilities.

5. Significance and Claims

The paper claims that FADTI represents a significant advancement by bridging the gap between frequency-domain signal processing and generative diffusion models for imputation. Its significance lies in:

Solving Frequency Distortion: By projecting intermediate hidden states rather than raw masked inputs, FADTI avoids the spectral artifacts that plague naive frequency-based approaches.
Handling Non-Stationarity: The inclusion of FrSST allows the model to capture non-stationary oscillatory patterns, a capability rarely seen in MTSI literature.
Robustness in Sparse Regimes: The framework is particularly effective under high missing rates and irregular missing patterns, common in real-world sensor data.
Generalizability: The introduction of the Yeast dataset and the successful zero-shot forecasting results suggest the model's applicability extends beyond standard imputation tasks to complex, non-stationary biological environments.

The authors conclude that while FADTI achieves state-of-the-art performance, future work will focus on extending the framework to irregular time grids and adaptive spectral transforms.