🤖 machine learning

NPMixer: Hierarchical Neighboring Patch Mixing for Time Series Forecasting

NPMixer is a hierarchical time series forecasting model that combines a learnable stationary wavelet transform for adaptive trend-detail decomposition with a neighboring patch mixing mechanism to effectively capture both local temporal dynamics and global channel correlations, achieving state-of-the-art performance across multiple benchmarks.

Original authors: Jung Min Choi, Vijaya Krishna Yalavarthi, Lars Schmidt-Thieme

Published 2026-05-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jung Min Choi, Vijaya Krishna Yalavarthi, Lars Schmidt-Thieme

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather for the next week. You have a massive notebook filled with thousands of data points: temperature, humidity, wind speed, and pressure, recorded every minute for the past few months.

The challenge is that this data is messy. It has tiny, chaotic fluctuations (like a sudden gust of wind) and big, slow-moving trends (like the gradual warming of spring). Traditional methods often get confused by the noise, while complex AI models sometimes get bogged down trying to read every single minute of data, wasting time on redundant information.

The paper introduces a new AI model called NPMixer (Neighboring Patching Mixer) that solves this by acting like a smart, multi-level editor for your data notebook. Here is how it works, broken down into simple concepts:

1. The "Learnable Wavelet" Filter: Tuning Your Radio

First, NPMixer needs to separate the "signal" from the "noise." Imagine you are listening to a radio station, but the signal is mixed with static and different frequencies.

What it does: Instead of using a fixed radio dial (like standard math tools), NPMixer has a Learnable Stationary Wavelet Transform. Think of this as a radio tuner that learns exactly which frequencies are important for your specific data.
The Magic: It splits the data into two parts:
- The Trend (The Slow Song): The smooth, long-term patterns (like the general rise in temperature over a month).
- The Details (The Fast Beats): The quick, jagged fluctuations (like sudden spikes in electricity usage).
Why it matters: By separating these, the model doesn't get confused when it tries to predict the future. It can focus on the "song" without getting distracted by the "static."

2. The "Non-Overlapping Patches": Reading in Chunks

Most AI models look at data using a sliding window that overlaps heavily (like reading a book where you read the first 10 words, then the second 10 words, then the third 10, and so on). This is slow and repetitive.

NPMixer's Approach: It cuts the data into neat, non-overlapping chunks (patches), like cutting a long loaf of bread into distinct slices.
The Benefit: It stops the model from wasting energy reading the same words twice. It treats each slice as a unique token, making the process much faster and cleaner.

3. The "Hierarchical Mixer": The Team of Editors

This is the core innovation. Once the data is sliced into chunks, NPMixer uses a "Hierarchical Mixer" to understand how these chunks relate to each other.

The Analogy: Imagine a newsroom with a hierarchy of editors.
- Level 1 (The Interns): They look at two adjacent slices of bread (patches) side-by-side and figure out how they connect locally.
- Level 2 (The Editors): They take the work of the interns and pair up those pairs. Now they are looking at a bigger picture, understanding how a group of four slices relates.
- Level 3 (The Chief Editor): They look at even larger groups, connecting the local details to the global story.
The "Group-Mix-Dissolve" Strategy: At each level, the model pairs up neighbors, mixes their information to find patterns, and then "dissolves" them back into a larger group to mix with the next neighbor. This allows the model to see both the tiny details and the big picture without getting lost in the middle.

4. The "Channel-Mixing": Connecting the Dots

In multivariate forecasting, you aren't just predicting one thing (like temperature); you are predicting many things at once (temperature, wind, humidity). These variables often influence each other.

The Solution: NPMixer uses a special "Channel-Mixing Encoder" specifically for the "fast beat" (high-frequency) details. It acts like a team meeting where the "Wind" variable talks to the "Temperature" variable to understand how they affect each other.
Crucial Detail: It does not do this for the "Slow Song" (the trend). It leaves the long-term trend alone so it doesn't get messed up by the noise of the other variables. This keeps the big picture stable.

The Results: Why It Wins

The authors tested NPMixer on seven real-world datasets (like electricity usage, traffic, and weather).

The Scorecard: In 20 out of 28 different test scenarios, NPMixer made fewer mistakes (lower error rates) than the current best models.
The Takeaway: By combining a smart filter (Wavelet), efficient slicing (Non-overlapping patches), and a smart hierarchy of mixing (Neighboring Mixer), the model can predict the future more accurately than its competitors, especially for long-term forecasts.

In short: NPMixer is like a super-efficient editor that knows how to tune out the noise, cut the story into perfect pieces, and then have a team of editors work together from the small details up to the big picture, ensuring the final prediction is both accurate and stable.

Technical Summary: NPMixer

Problem Statement

Multivariate time series forecasting (TSF) faces significant challenges due to the dual complexity of local temporal dynamics and global dependencies across multiple variables. While recent deep learning approaches have moved beyond traditional statistical methods (e.g., ARIMA) to utilize Transformers and MLPs, existing solutions often struggle with efficiency and feature redundancy. Specifically, many patch-based methods (e.g., PatchTST) rely on overlapping windows to capture local semantics, which increases computational overhead and risks introducing redundant information that leads to overfitting. Furthermore, while frequency-domain approaches (e.g., FEDformer, WaveForM) help isolate periodicities, standard Discrete Wavelet Transforms (DWT) suffer from downsampling, which breaks translation invariance and can lead to feature loss.

Methodology: NPMixer

The authors propose NPMixer (Neighboring Patching Mixer), a hierarchical architecture designed to capture multi-scale temporal dynamics and cross-channel correlations without the redundancy of dense sliding windows. The framework consists of three core components:

1. Learnable Stationary Wavelet Transform (LSWT)

Instead of using fixed basis functions (like Haar or Daubechies) or standard DWT, NPMixer employs a Learnable Stationary Wavelet Transform.

Mechanism: The model optimizes filter coefficients ( $h_0, h_1$ ) during training via backpropagation, allowing the decomposition to adapt dynamically to the specific spectral characteristics of the data.
Stationarity: Unlike DWT, the Stationary Wavelet Transform (SWT) omits downsampling. This preserves the original sequence length and maintains translation invariance, ensuring that signal shifts do not alter the feature representation.
Decomposition: The input is decomposed into a global approximation coefficient ( $A_M$ , representing the trend) and multi-scale detail coefficients ( $\{D_m\}$ , representing high-frequency fluctuations).

2. Channel-Mixing Encoder

This component is applied exclusively to the detail coefficients (high-frequency components).

Architecture: Adopting an inverted architecture similar to iTransformer, the model treats the entire look-back series of each variable as a distinct token.
Function: A Multi-Head Self-Attention (MSA) mechanism models pairwise correlations between variables, learning a dynamic graph of multivariate interactions. This allows the model to refine high-frequency features based on the state of correlated variables.
Exclusion: The approximation branch (trend) bypasses this encoder to preserve the smooth, univariate trajectory of the global trend without introducing noise from cross-channel mixing.

3. Neighboring Mixer Block

This is the core innovation for capturing temporal dependencies, utilizing a non-overlapping patching strategy.

Non-Overlapping Patching: The sequence is divided into non-overlapping patches of length $P$ .
Hierarchical Pairing: The model employs an iterative "Group-Mix-Dissolve" strategy. At each hierarchy level, adjacent temporal blocks are paired and processed through a deep Multi-Patch MLP to learn relational features.
Asymmetric Update: To align with the temporal structure of forecasting, the model uses a learnable gate ( $\alpha_k$ ) to enforce a directional bias. The initial block updates based on future-oriented signals (interaction with the subsequent block), while subsequent blocks prioritize historical continuity (integration from preceding context).
Progression: Through recursive pairing, the receptive field expands efficiently, transitioning from learning local granular features to capturing global sequence-wide patterns without the computational cost of overlapping strides.

4. Reconstruction and Projection

The processed features are reconstructed via a Learnable Inverse SWT (LISWT) and passed through a Final Projection layer (a global bottleneck transformation) to generate the forecast. The output undergoes RevIN de-normalization to restore the original scale.

Key Contributions

Recursive Patch Aggregation: The paper demonstrates that aggregating non-overlapping patches hierarchically effectively captures the transition from local details to global dependencies, enriching the representation of multi-scale dynamics.
Frequency-Domain Multi-Scale Learning: The authors show that performing multi-scale learning within the frequency domain (via LSWT) is crucial for capturing precise temporal dynamics across different resolutions.
State-of-the-Art Architecture: NPMixer is proposed as a simple yet effective architecture that achieves state-of-the-art results across various multivariate time series datasets for long-term forecasting tasks.

Experimental Results

The model was evaluated on seven benchmark datasets (ETTh1, ETTh2, ETTm1, ETTm2, Weather, Electricity, Traffic) against six state-of-the-art baselines (SimpleTM, TimeXer, CrossGNN, iTransformer, PatchTST, TimeMixer).

Performance: NPMixer consistently outperformed baselines, achieving the best Mean Squared Error (MSE) in 20 out of 28 evaluated experimental setups (71.4%) and the best Mean Absolute Error (MAE) in 18 setups.
Specific Gains: Notable improvements were observed on the ETTm1 and ETTm2 datasets, where NPMixer improved average MSE by 4.98% over the second-best variant (SimpleTM).
Ablation Studies:
- Removing the Neighboring Mixer (replacing it with a single-patch mixer) caused significant performance degradation (e.g., an 18.01% MSE increase on the Traffic dataset), highlighting the necessity of hierarchical pairing.
- Removing the Channel-Mixing Encoder led to a 17.19% MSE increase on the high-dimensional Traffic dataset, confirming the importance of explicit cross-channel correlation modeling in the frequency domain.
- The Learnable aspect of the wavelet transform provided consistent gains, particularly on ETTm1 and Weather datasets.
Look-Back Window Analysis: NPMixer demonstrated robustness with longer look-back windows, achieving the best results in 5 out of 7 datasets when window sizes were optimized, suggesting the model effectively exploits richer historical information.

Significance and Claims

The paper claims that NPMixer establishes a new state-of-the-art for multivariate time series forecasting by synergizing learnable spectral decomposition with hierarchical temporal mixing.

Efficiency vs. Redundancy: By utilizing non-overlapping patches and hierarchical pairing, the model captures multi-scale representations without the computational redundancy inherent in overlapping sliding windows.
Stability and Precision: The integration of LSWT allows the model to isolate and process specific periodicities and noise patterns independently, while the Channel-Mixing Encoder ensures that cross-variable correlations are captured without destabilizing the global trend.
Future Outlook: The authors acknowledge limitations regarding strict input length constraints (due to exponential block sizing) and sensitivity to initial patch size. They propose future work on flexible, length-adaptive patching and dynamic filter adaptation to handle non-stationary concept drift.

The work is positioned as a step forward in modeling complex temporal dynamics, with potential positive societal impacts in energy grid optimization, weather forecasting, and logistics planning through improved forecasting accuracy.