🤖 AI

PMDformer: Patch-Mean Decoupling Information Transformer for Long-term Forecasting

PMDformer is a novel long-term time series forecasting model that introduces patch-mean decoupling to separate trend and shape information, combined with Trend Restoration and Proximal Variable attention mechanisms, to effectively capture shape similarities and cross-variable dependencies while outperforming existing state-of-the-art methods.

Original authors: Ao Hu, Liangjian Wen, Jiang Duan, Yong Dai, He Yan, Dongkai Wang, Jun Wang, Yukun Zhang, Ruoxi Jiang, Zenglin Xu

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ao Hu, Liangjian Wen, Jiang Duan, Yong Dai, He Yan, Dongkai Wang, Jun Wang, Yukun Zhang, Ruoxi Jiang, Zenglin Xu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the future weather, stock market trends, or traffic flow based on a long history of data. This is called Long-Term Time Series Forecasting. The challenge is that this data is messy; it has huge spikes, deep drops, and changes in scale that make it hard to see the actual "shape" or pattern of what's happening.

The paper introduces a new AI model called PMDformer. Think of it as a super-smart detective that has learned a special trick to ignore the "noise" of size and focus entirely on the "shape" of the story.

Here is how it works, broken down into simple analogies:

1. The Problem: The "Volume" Distortion

Imagine you are looking at three different mountains on a map.

Mountain A is a tiny hill (0 to 10 meters high).
Mountain B is a medium hill (0 to 100 meters high).
Mountain C is a giant peak (0 to 1,000 meters high).

If you ask a standard AI to find which two mountains look the same, it might get confused. It sees that Mountain C is huge and Mountain A is tiny, so it thinks they are totally different. But if you look at the slope (the shape), Mountain A and Mountain B might actually have the exact same steepness, just at different scales.

Old AI models get distracted by the size (the scale) of the data. They think a big spike is more important than a small one, even if the small one has the exact same pattern. This leads to bad predictions.

2. The Solution: "Patching" and "Decoupling"

The authors' model, PMDformer, uses three main tricks to solve this:

Trick #1: The "Mean-Subtraction" Glasses (Patch-Mean Decoupling)

Instead of looking at the raw numbers, the model breaks the long history of data into small chunks called patches (like cutting a long movie into short scenes).

For each scene, it does something clever: It subtracts the average height.

Imagine you have a graph that goes from 100 to 110. The model says, "Okay, let's ignore the fact that it's at 100. Let's just look at how it moves relative to 100."
Now, a graph going from 100 to 110 looks exactly the same as a graph going from 1,000 to 1,010.
The Result: The model can finally see that two different data streams have the exact same shape (up and down pattern), even if one is huge and the other is tiny. It separates the "trend" (the average height) from the "shape" (the wiggles).

Trick #2: The "Recent Neighbor" Rule (Proximal Variable Attention)

In the real world, relationships between things change. For example, in the stock market, two companies might move together during a crisis, but drift apart during calm times.

Old models look at the entire history (maybe 5 years of data) to guess how variables relate. This is like trying to predict your best friend's mood today by looking at how they acted 5 years ago. It's too much noise!

PMDformer says: "Let's only look at the most recent chunk of data."

It focuses only on the very last scene of the movie to see how the variables are interacting right now.
This prevents the model from getting confused by old, outdated relationships that don't apply anymore.

Trick #3: The "Trend Restorer" (Trend Restoration Attention)

Remember Trick #1, where we subtracted the average height to see the shape? Well, if we forget that average height entirely, our prediction will be flat and useless. We need to know where the data is, not just how it moves.

So, after the model figures out the shapes and relationships, it has a special step called Trend Restoration.

It takes the "average height" (the trend) it set aside earlier and gently puts it back into the final prediction.
The Result: The model gets the best of both worlds: it predicts the pattern accurately (because it ignored the size noise) and the scale accurately (because it put the trend back in).

3. The Results: Why It Wins

The paper tested this model against the best existing AI models on real-world data like electricity usage, traffic flow, and weather.

The Analogy: Imagine a race where everyone is trying to predict the future. The other runners are tripping over their own shoelaces (the scale differences) and looking at old maps (outdated history).
The Winner: PMDformer runs smoothly because it sees the true shape of the path and only looks at the map for the next mile.
The Outcome: The paper claims PMDformer is more accurate and stable than all the other top models tested. It makes fewer mistakes in predicting the future.

Summary

PMDformer is a new forecasting tool that:

Ignores the volume to see the true shape of the data.
Focuses on the recent past to avoid outdated patterns.
Re-adds the big picture so the final prediction is realistic.

By doing this, it solves the problem of "scale bias" and gives us a clearer, more accurate crystal ball for the future.

Technical Summary: PMDformer

Problem Statement

Long-term time series forecasting (LTSF) is critical for domains such as energy management, finance, and traffic prediction. While Transformer-based models have adopted patch-based strategies to capture long-range dependencies, they face a fundamental challenge: accurately modeling shape similarities across patches and variables is often obscured by scale differences.

Time series data is inherently non-stationary, causing patch scales to fluctuate wildly. As demonstrated in the paper, these scale variations can dominate attention mechanisms, causing models to prioritize patches with larger magnitudes over those with genuinely similar shapes. For instance, a patch with a high mean value may be incorrectly deemed more similar to a query patch than a patch with a lower mean but a more identical shape trajectory. Furthermore, existing variable-dependent (VD) methods often model interactions over the entire historical window, introducing noise from outdated correlations and risking overfitting, while standard normalization techniques (like Z-score) that remove standard deviation inadvertently distort the intrinsic shape of the data.

Methodology

The authors propose PMDformer, a model designed to decouple trend and shape information to facilitate accurate shape-focused attention. The architecture consists of four synergistic modules:

1. Patch-Mean Decoupling (PMD)

Instead of full normalization, PMD partitions the input sequence into non-overlapping patches and subtracts the mean of each patch ( $\mu$ ).

Mechanism: For a patch $P$ , the residual shape $r$ is calculated as $r = P - \mu$ .
Effect: This centers each patch to zero mean, explicitly separating the long-range trend (encoded in the means) from the residual shape information. Crucially, unlike Z-score normalization, PMD preserves the original amplitude variations and intrinsic shape structure. This allows the attention mechanism to focus on true shape similarities rather than being biased by scale.

2. Proximal Variable Attention (PVA)

To address the non-stationary nature of cross-variable relationships, PVA restricts variable interaction modeling to the most recent time segment.

Mechanism: PVA applies Multi-Head Self-Attention (MHSA) exclusively on the tokens of the last (most proximal) patch across all variables.
Effect: By focusing on the most recent interactions, the model captures the most predictive shape similarities for future dynamics, avoiding noise from historical drifts and reducing computational complexity from $O(C^2N)$ to $O(C^2)$ .

3. Trend Restoration Attention (TRA)

To ensure global dynamics are not lost during shape-focused processing, TRA reintegrates the trend information.

Mechanism: A parameter-shared Transformer encoder processes the shape embeddings. The Query ( $Q$ ) and Key ( $K$ ) projections operate solely on the shape residuals to model inter-patch shape similarity. However, the Value ( $V$ ) pathway explicitly incorporates the decoupled patch means ( $\mu$ ) via an additive connection: $V = P W_V + \mu$ .
Effect: This architectural separation allows the model to jointly encode fine-grained local shape dependencies (via $Q/K$ ) and global trend dynamics (via $V$ ), ensuring stable forecasts that respect long-range trends.

4. Projection Layer

The final layer re-incorporates the per-patch trend means into the refined shape embeddings before generating multi-step forecasts, ensuring the output is well-calibrated with the original scale and long-term dynamics.

Key Contributions

Patch-Mean Decoupling (PMD): A novel mechanism that separates trend and residual shape within the attention module via mean subtraction. This enables the effective capture of shape similarity among temporal patches and variables without distorting amplitudes.
Proximal Variable Attention (PVA): A module that focuses cross-variable attention on the most recent patch, capturing the most relevant shape similarities and mitigating overfitting to outdated correlations.
Trend Restoration Attention (TRA): A method to restore decoupled trend information into the value pathway of the attention mechanism, allowing for the joint encoding of local shapes and global trends.
Empirical Performance: Extensive experiments demonstrate that PMDformer achieves superior stability and accuracy compared to existing state-of-the-art methods across multiple LTSF benchmarks.

Experimental Results

The authors evaluated PMDformer on eight real-world datasets (ECL, Traffic, Weather, Solar, ETTh1, ETTh2, ETTm1, ETTm2) against nine baselines, including TQNet, TimeBase, SOFTS, SparseTSF, ModernTCN, iTransformer, TimeMixer, and PatchTST.

Overall Performance: PMDformer achieved the lowest Mean Squared Error (MSE) and Mean Absolute Error (MAE) on 7 out of 8 datasets.
Comparative Gains:
- Compared to TimeBase (a leading patch-based model), PMDformer showed an average MSE reduction of 5.68% and MAE reduction of 6.61%.
- Compared to TQNet, it achieved an average MSE reduction of 8.62% and MAE reduction of 9.96%.
- Compared to iTransformer, it delivered an average MSE reduction of 11.44% and MAE reduction of 12.38%.
Ablation Studies:
- Removing PMD or replacing it with standard normalization (e.g., SAN or Z-score) resulted in performance degradation, confirming the necessity of preserving shape while decoupling trends.
- Replacing PVA with attention over all historical tokens or removing it entirely led to significant performance drops, validating the importance of focusing on proximal variable interactions.
- Replacing TRA with standard self-attention or removing it caused substantial errors, highlighting the critical role of trend restoration.

Significance

The paper claims that PMDformer addresses a core limitation in current LTSF architectures: the inability to distinguish true shape similarity from scale bias in non-stationary data. By explicitly decoupling trends from shapes and focusing variable interactions on the most relevant temporal windows, the model provides a more robust framework for long-term forecasting. The results suggest that shape-centric Transformer designs, which prioritize structural similarity over raw magnitude, offer a significant advantage in stability and accuracy for complex time series tasks. The authors position this work as a step toward more reliable forecasting in critical infrastructure domains like energy and finance.