💻 computer science

PatchMamS2S: A Mamba-based Multivariate Time Series Forecasting Model in Power Domain with Patch Mining via Prior-based Weight Fusion

This paper proposes PatchMamS2S, a lightweight Mamba-2-based multivariate time series forecasting model tailored for the power domain that combines patch mining, physical prior-guided weight fusion, and an efficient Seq2Seq architecture to overcome the accuracy-resource trade-offs of existing universal models.

Original authors: Yuhan Lin, Haoyuan Ouyang, Hongyu Chen, Liping Xiong, Zhiyong Hong, Zhishuang Wang

Published 2026-08-12

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Yuhan Lin, Haoyuan Ouyang, Hongyu Chen, Liping Xiong, Zhiyong Hong, Zhishuang Wang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the future, but instead of guessing the weather or the stock market, you are trying to guess how much electricity a whole city will need next week. This is the world of Multivariate Time Series Forecasting. Think of it as listening to a chaotic orchestra where every instrument (temperature, solar power, factory schedules, and human habits) plays a different tune, and you need to predict the next note of the entire symphony. For a long time, scientists tried to build one "super-brain" model that could listen to any orchestra, anywhere, no matter how complex. But here's the catch: these super-brains are often so heavy and hungry for computer power that they can't fit on the small, cheap computers sitting inside real-world power grids. It's like trying to run a Hollywood blockbuster movie on a toy calculator.

Enter Mamba, a new type of AI architecture that is like a super-efficient, lightweight runner. Unlike the old heavy models that get slower and slower as the song gets longer, Mamba stays fast and lean, no matter how much data it has to process. However, even this lightweight runner has some baggage: it sometimes stacks too many layers (making it heavy again) and struggles to understand how different instruments in the orchestra influence each other. The big question is: Can we build a model that is light enough to run on a tiny power grid computer, smart enough to understand the specific rhythm of electricity, and accurate enough to keep the lights on?

This paper introduces PatchMamS2S, a clever new solution designed specifically for the power industry. The researchers argue that chasing a "universal" model for everything is a bit of a trap; instead, we need a tool tailored to the unique physics of electricity. They built a system that treats electricity data like a patchwork quilt. Instead of looking at the whole history of power usage at once, they chop it into small, manageable "patches" of different sizes. Some patches are tiny, catching quick, jittery fluctuations (like a sudden spike when a factory turns on), while others are large, capturing the slow, smooth trends (like the daily cycle of people waking up and going to sleep).

The magic happens when they stitch these patches back together. Instead of letting the computer guess how to mix them, the researchers use "physical priors"—basically, they tell the model, "Hey, we know that small patches are great for details, and big patches are great for the big picture." They assign specific weights to these patches based on this real-world logic, fusing them into a clear picture of the future. This is then fed into a streamlined engine called MamS2S, a custom-built version of Mamba that doesn't waste energy by stacking unnecessary layers.

The results are impressive. When tested on five different power datasets (including real-world electricity prices and transformer temperatures) and even some non-power data like weather and finance, PatchMamS2S didn't just keep up; it often beat the best existing models. It managed to be significantly more accurate than its competitors while using a fraction of the memory. In fact, compared to another heavy Mamba-based model, this new design used 8 times less memory and trained 20 times faster. The authors suggest that this proves you don't need a massive, expensive supercomputer to predict power loads; with the right "patchwork" strategy and a little bit of physics-based guidance, you can build a model that is both a genius and a lightweight champion, ready to run on the small devices actually controlling our power grids.

Technical Summary of PatchMamS2S

Problem Statement
Multivariate Time Series Forecasting (MTSF) in the power sector faces a critical dilemma: existing "universal" architectures struggle to balance prediction accuracy with the strict computational constraints of terminal power grid devices. Transformer-based models, while powerful, incur quadratic computational complexity and excessive memory usage, rendering them unsuitable for edge deployment. Conversely, simple linear models lack the capacity to capture the complex, nonlinear physical characteristics of power loads. Furthermore, existing State Space Models (SSMs), particularly Mamba-based frameworks, often suffer from architectural redundancy due to blind deep stacking and fail to sufficiently mine multivariate correlation relationships with low parameter counts. The paper argues that pursuing a single "universal architecture" for all domains is a misconception, necessitating a tailored solution for the power industry that respects physical operational logic and real-time responsiveness.

Methodology
The authors propose PatchMamS2S, a lightweight, Mamba-based forecasting model designed specifically for power load data. The architecture integrates three core components:

Multi-Scale Patch Embedding (PatchEmb):
Instead of single-scale patching, the model employs a multi-scale patch fusion mechanism. It segments the input time series into patches of varying lengths ( $P = [p_1, p_2, \dots, p_M]$ ) using sliding windows. This allows the model to simultaneously capture microscopic short-term fluctuations and macroscopic long-term trends, effectively covering multi-scale features without the high computational burden of global attention mechanisms.
Prior-Based Weight Fusion:
The model adopts a seasonal-trend decomposition strategy followed by a physics-informed fusion mechanism. It decomposes multi-scale patch embeddings into seasonal and trend components. Crucially, it utilizes fixed, prior-based weights to fuse these components rather than learning them blindly.
- Seasonal Fusion: Assigns higher weights to small-scale patches (narrow field of view) which are better at capturing high-frequency, localized fluctuations.
- Trend Fusion: Assigns higher weights to large-scale patches (broad field of view) which better characterize smooth, global trends.
  This approach injects physical interpretability into the model, ensuring that the reconstruction of features aligns with the natural "division of labor" of different time scales.
MamS2S (Lightweight Seq2Seq Architecture):
Built upon Mamba-2, the authors construct a custom Seq2Seq architecture named MamS2S. To address the incompatibility between Mamba's continuous state-space evolution and traditional Seq2Seq requirements (which rely on explicit hidden states), the authors modify the Mamba module to explicitly extract the final hidden state of the encoder. Additionally, they remove the one-dimensional convolution operation preceding the SSM module. This structural simplification ensures strict sequence causality (preventing future information leakage) and reduces complexity, enabling efficient dependency modeling at a linear complexity level $O(L)$ .

Key Contributions

Tailored Patch Representation: A multi-scale patching strategy that accurately captures the unique periodic patterns of power loads while balancing efficiency and representational capacity.
Physics-Informed Fusion: A departure from "black-box" automated fusion, utilizing fixed prior weights to reconstruct seasonal and trend features, thereby enhancing both predictive performance and physical interpretability.
Lightweight MamS2S Architecture: A novel Seq2Seq design based on Mamba-2 that avoids the blind deep stacking common in current SSM research. It achieves efficient dependency modeling with linear complexity, significantly lowering hardware deployment barriers and enabling training on low-compute GPUs (e.g., NVIDIA 3050).

Experimental Results
The model was evaluated on five power-domain datasets (including ETTh1/2, ETTm1/2, and SXEP) and three non-power datasets (Weather, Exchange, Solar).

Power Domain Performance: PatchMamS2S achieved superior overall performance compared to state-of-the-art baselines (including iTransformer, PatchTST, Crossmamba, and S-Mamba). On the ETTh1 dataset, it reduced MSE and MAE by 4.86% and 3.35% respectively compared to the second-best model. It demonstrated robust generalization even in small-sample scenarios (SXEP dataset).
Efficiency: The model significantly outperformed other Mamba-based models in efficiency. Compared to Crossmamba, PatchMamS2S reduced GPU memory usage by 8 times and shortened training time per epoch by over 200 seconds. It achieved a parameter count of 4.5M, which is competitive with Transformer models but with superior efficiency.
Generalization: Despite being designed for power systems, the model achieved SOTA performance in 4 out of 24 cases across non-electricity domains, demonstrating strong cross-domain generalization capabilities.

Significance and Claims
The paper claims that PatchMamS2S successfully breaks the application bottleneck of high-complexity models on terminal power grid devices. By avoiding the redundant stacking of Mamba modules and integrating physical priors, the model offers a low-cost, high-performance solution that balances accuracy and computational efficiency. The authors posit that this work provides a theoretical and practical foundation for deploying complex deep learning models in industrial scenarios where resources are limited. They conclude that while the field is moving toward large-scale foundational models, integrating such models with domain-specific physical constraints remains a key challenge, and PatchMamS2S represents a step toward efficient, domain-tailored forecasting.

Technical Summary of PatchMamS2S

More like this