🤖 machine learning

CAMP: A Cycle-Aware Multi-Scale Patch Mixer for Time Series Forecasting

CAMP is a novel time series forecasting framework that integrates adaptive cycle learning, horizon-guided patch mixing, and multi-scale residual modeling to effectively handle variable periodic patterns, position-dependent contextual needs, and multi-resolution dynamics, achieving state-of-the-art performance across multiple long-term forecasting benchmarks.

Original authors: Jung Min Choi, Vijaya Krishna yalavarthi, Lars Schmidt-Thieme

Published 2026-08-06

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Jung Min Choi, Vijaya Krishna yalavarthi, Lars Schmidt-Thieme

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the future, but instead of a crystal ball, you have a stream of data that looks like a chaotic scribble. This is the world of time series forecasting, a branch of data science dedicated to guessing what happens next based on what happened before. Think of it as trying to predict the next note in a song, the next wave in the ocean, or the next rush of traffic on a highway. The data isn't just random noise; it often has a rhythm. Sometimes it's a steady beat, like a heart rate; other times, it's a complex melody with repeating verses and choruses. Scientists call these repeating patterns "cycles."

For a long time, computers trying to predict these patterns had a major blind spot. They often assumed that the "beat" of the data was the same for everyone, everywhere, and at every moment. It was like trying to teach a robot to dance to a song by telling it, "The beat is always 120 steps per minute," even when the music suddenly slowed down or sped up. Furthermore, when looking at a long line of data, these computers treated every single piece of information the same way, whether it was from yesterday or from five minutes ago. This paper tackles these rigid assumptions, suggesting that to predict the future accurately, a computer needs to be more like a flexible musician who listens to the specific rhythm of this song, right now, and knows which notes are the most important to remember.

Meet CAMP: The Time-Traveling DJ

Enter CAMP (Cycle-Aware Multi-Scale Patch Mixer), a new forecasting model that acts like a super-smart DJ for time series data. The authors, Jung Min Choi and colleagues from the University of Hildesheim, realized that real-world data is messy. A dataset might have a daily cycle, a weekly cycle, or even a cycle that changes speed depending on how far back you look. Old methods tried to force all this data into a single, pre-set rhythm, which often led to missed beats and bad predictions. CAMP, however, decides to listen to the music first before it starts dancing.

The Magic of the Adaptive Rhythm
Imagine you are looking at a long video of a busy street. Some cars move in a steady stream (a cycle), while others stop and start randomly (residual noise). Old models would try to guess the speed of the traffic by looking at the whole city and picking one average speed. If the traffic suddenly slowed down for a parade, the model would get confused.

CAMP does something different. It uses a tool called the Fast Fourier Transform (FFT)—think of it as a super-fast frequency analyzer—to look at each specific window of data. It asks, "What is the dominant beat in this specific chunk of time?" It then builds a custom rhythm for that exact moment. It doesn't need a pre-written schedule; it figures out the beat on the fly. This allows it to generate a "cyclic component" (the predictable, repeating part) and a "residual" (the messy, unpredictable part) separately. It's like separating the drum track from the guitar solo so you can fix them individually.

The "Patch" Puzzle and the Horizon Guide
Once CAMP has stripped away the predictable rhythm, it's left with the messy residual data. To handle this, it chops the data into little chunks called "patches." Imagine cutting a long strip of film into individual frames. Most models treat every frame the same way, shuffling them around to find patterns. But CAMP noticed something clever: the frames closest to the "forecast boundary" (the very edge where the prediction starts) are super important and shouldn't be messed with too much. The older frames, however, need a broader context to make sense.

To solve this, CAMP uses a Horizon-Guided Patch Mixer. Think of this as a teacher grading a student's essay. The teacher reads the most recent sentences (the ones closest to the end) very carefully and leaves them mostly alone because they are the conclusion. But for the earlier sentences, the teacher mixes them up with other ideas to find the deeper themes. CAMP does the same: it aggressively mixes the older patches to find long-range connections but keeps the newest patches pristine so the model doesn't lose the most immediate information.

The Multi-Scale Magic
Finally, the messy residual data is often complex, containing both big, slow waves and tiny, fast jitters. CAMP uses a Stationary Wavelet Transform (SWT) to split this mess into different "scales" or resolutions. It's like using a set of different-sized sieves to sort sand: one sieve catches the big rocks, another catches the pebbles, and a fine one catches the dust. CAMP processes each of these layers with its own specialized mixer, then combines them all back together. This ensures that no matter how big or small the fluctuation is, it gets the attention it deserves.

What the Numbers Say

The authors tested CAMP on seven different long-term forecasting benchmarks, including weather data, electricity usage, and traffic patterns. The results were impressive. Across these datasets, CAMP achieved the best average Mean Squared Error (MSE) on six out of seven datasets and the best or tied-best Mean Absolute Error (MAE) on six. In the world of traffic forecasting (using the PEMS benchmarks), it won the most comparisons in 16 different settings.

For example, on the ETTh2 dataset (electricity usage), CAMP reduced the error to an MSE of 0.309, beating the second-best model (PatchTST) which had an error of 0.331. On the Weather dataset, it achieved an MSE of 0.219. These aren't just tiny improvements; they represent a significant leap in accuracy, suggesting that the model's ability to adapt to changing cycles and handle different scales of data is working exactly as intended.

Why This Matters (and What It's Not)

The paper suggests that the key to better predictions isn't just having a bigger computer, but having a smarter way to listen. By admitting that cycles change and that recent data needs special treatment, CAMP avoids the pitfalls of rigid, one-size-fits-all models. The authors ran "ablation studies" (tests where they removed parts of the model) to prove that every piece of their puzzle is necessary. When they removed the adaptive cycle learning or the special patch mixing, the performance dropped, confirming that these specific tricks are what make the model work.

However, the authors are careful not to call this a magic bullet. They note that CAMP is computationally heavier than some simpler models and that its frequency-based cycle estimation might struggle if the data is too short or has no clear rhythm at all. They also suggest that for datasets with hundreds of variables that are all tightly connected, the model's current way of mixing channels might need to be more selective to avoid confusion.

In short, CAMP suggests that if you want to predict the future, you shouldn't just memorize the past; you should understand the rhythm of the moment, respect the importance of the present, and look at the big picture and the small details all at once. It's a flexible, multi-layered approach that suggests the future is less about rigid formulas and more about adaptive listening.

Technical Summary: CAMP – A Cycle-Aware Multi-Scale Patch Mixer for Time Series Forecasting

Problem Statement

Real-world time series are frequently governed by recurring patterns, yet the dominant periods of these cycles often vary across different datasets, forecasting settings, and even individual input windows. Existing cycle-aware forecasting models typically rely on a single, predefined cycle length selected at the dataset level (e.g., via autocorrelation analysis). This approach is restrictive when periodic behavior changes over time or when multiple cycles coexist within a single sequence.

Furthermore, standard patch-based models often process all patch positions uniformly. However, patches farther from the forecast boundary may require broader contextual refinement, whereas recent patches (closer to the boundary) contain information that should be preserved more directly. Finally, after removing cyclic behavior, the remaining residual dynamics often span multiple temporal resolutions, which cannot be adequately described at a single scale.

Methodology

The authors propose CAMP (Cycle-Aware Multi-Scale Patch Mixer), a framework designed to separately model cyclic and residual dynamics while addressing the limitations of fixed cycle lengths and uniform patch processing. The architecture consists of three primary components:

1. Adaptive Cycle Learning (ACL)

Unlike methods that require predefined cycle lengths, ACL identifies dominant frequencies independently for each input window.

Frequency Estimation: For each sample, the model applies a Fast Fourier Transform (FFT) to the look-back window. It computes a channel-averaged amplitude spectrum, suppresses the DC component, and selects the top- $K$ frequency bins.
Cycle Synthesis: These selected frequency bins define sinusoidal features (sine and cosine pairs) over both the historical look-back and future prediction intervals. A shared two-layer MLP maps these Fourier features to channel-wise cycle values.
Gating: Separate sigmoid gates are applied to the historical and future cyclic components. This allows the model to control how much cyclic structure is removed from the input and how much is reinjected into the forecast. The network is initialized to zero, allowing the model to start as a residual-only predictor and gradually introduce cyclic information during training.

2. Multi-Resolution Residual Decomposition

After subtracting the gated historical cycle from the normalized input, the remaining residual is processed using a Stationary Wavelet Transform (SWT).

Undecimated Decomposition: The SWT decomposes the residual into an approximation stream ( $A_J$ ) and multiple detail streams ( $D_J, \dots, D_1$ ) at different levels. Because SWT is undecimated, all streams retain the original sequence length $L$ , ensuring temporal alignment across resolutions.
Stream-Specific Encoding: Each stream is independently patched and processed by specific MLP mixers:
- The coarsest detail stream ( $D_J$ ) is processed by the Horizon-Guided Patch Mixer (HGPM).
- The approximation stream ( $A_J$ ) and finer detail streams ( $D_{J-1} \dots D_1$ ) are processed by Simple Patch Mixers.

3. Horizon-Guided Patch Mixer (HGPM)

This module addresses the varying needs of patches based on their distance from the forecast boundary.

Progressive Refinement: HGPM processes shrinking prefixes of the patch sequence in reverse order (from $N-1$ down to $1$).
Update Mechanism: At each step $i$ , only the prefix $U_{1:i}$ is updated via a residual correction involving feature, temporal, and channel mixing.
Boundary Preservation: The most recent patch ( $U_N$ ), which is closest to the forecast horizon, receives no cross-patch updates from the HGPM. It retains its representation after only patch projection and intra-patch refinement. This design ensures that information near the forecast boundary is preserved directly, while earlier patches benefit from broader temporal context.

4. Aggregation and Prediction

The encoded representations from all wavelet streams are averaged (without learnable weights) to form a unified representation. A linear prediction head maps this to the residual forecast. The predicted future cyclic component is added to the residual forecast, and the result is denormalized via RevIN to produce the final output.

Key Contributions

Sample-Adaptive Cycle Learning: CAMP identifies dominant frequencies for each input sample and generates historical and future cyclic components without requiring predefined cycle lengths or dataset-level cycle selection.
Multi-Resolution Residual Modeling: The framework combines SWT-based decomposition with patch-level MLP encoding to capture complementary dynamics at different temporal scales.
Horizon-Guided Patch Mixing: The introduction of HGPM allows for position-dependent refinement, preserving recent information while progressively refining earlier context.
Performance: The model achieves state-of-the-art or competitive results on both long-term and short-term forecasting benchmarks.

Experimental Results

The authors evaluated CAMP on seven long-term forecasting datasets (ETTm1, ETTm2, ETTh1, ETTh2, Electricity, Weather, Traffic) and four short-term traffic datasets (PEMS03, PEMS04, PEMS07, PEMS08).

Long-Term Forecasting: CAMP achieved the best average MSE on six of the seven datasets and the best or tied-best MAE on six. Notably, it showed significant improvements on ETTh2 and Weather.
Short-Term Forecasting (PEMS): CAMP obtained the highest win count across 16 settings on the PEMS benchmarks, achieving the best MSE in 10 settings and the best MAE in 7. It demonstrated particular strength in short horizons ( $H=12$ ).
Ablation Studies: Removing any core component (ACL, Intra-patch Mixer, SWT, or HGPM) consistently degraded performance. Replacing ACL with a recurrent cycle module (from CycleNet) also resulted in lower accuracy, confirming the benefit of adaptive frequency selection.
Generalizability: Integrating the ACL module into the existing CycleNet model improved its performance across all tested datasets, suggesting that adaptive cycle learning is a beneficial component for cycle-based forecasting models in general.

Significance and Claims

The paper claims that CAMP effectively addresses the limitations of fixed-cycle assumptions and uniform patch processing in time series forecasting. By separating cyclic and residual dynamics and modeling the latter at multiple resolutions with horizon-aware refinement, CAMP captures complementary temporal structures.

The authors note that while CAMP achieves strong performance, it comes with higher computational costs. Additionally, FFT-based cycle estimation may be less reliable for very short or weakly periodic inputs, and dense channel interaction might be less suitable for highly multivariate datasets where selective channel interaction could be preferable. Future work is proposed to explore more efficient patch mixing, robust cycle estimation, and selective channel-aware modeling.