Technical Summary: SPDM – Geometry-Modulated State Space Modeling with Manifold Constraints

1. Problem Statement

Multivariate time series forecasting faces a fundamental challenge: capturing the continuously evolving correlation structures among interacting variables. Existing neural sequence architectures, including Transformers and standard State-Space Models (SSMs) like Mamba, largely treat inter-channel relationships as static or discard their geometric continuity.

• Transformers rely on self-attention mechanisms that incur quadratic computational complexity ($O(L^2)$) and struggle to propagate correlation structures continuously along the temporal dimension.
• Standard SSMs (e.g., Mamba) achieve linear complexity but typically model multivariate interactions as flat sequences or static graphs, failing to account for the intrinsic geometric properties of evolving covariance matrices.
• Existing Geometric Methods often rely on static geometric priors or apply geometric transformations as auxiliary mechanisms, rather than modeling the correlation structure as a continuous trajectory on a principled geometric manifold.

The core problem is the lack of an architecture that can simultaneously maintain the computational efficiency of SSMs while explicitly modeling the evolving cross-variable correlation structure as a continuous geometric trajectory.

2. Methodology: SPDM Architecture

The authors propose SPDM (Geometry-Modulated State Space Modeling with Manifold Constraints), an architecture that treats the evolving cross-variable correlation structure as a continuous trajectory on the Symmetric Positive Definite (SPD) manifold ($\mathcal{P}_N$). The model integrates two cooperating paths to realize manifold-constrained state-space dynamics.

Path A: SPD Manifold Trajectory Path

This path constructs and processes the geometric evolution of correlations:

1. Dynamic Covariance Construction: The input sequence is unfolded into overlapping sliding windows. For each window, a sample covariance matrix $\Sigma_t$ is computed, tracing a discrete trajectory on $\mathcal{P}_N$.
2. Tangent Space Projection: To enable efficient processing by standard neural blocks, the trajectory is projected to a Euclidean tangent space.
   • A reference point is established using the Log-Euclidean Fréchet Mean ($\bar{\Sigma}$), which respects the curved geometry of the manifold.
   • Each covariance matrix is projected to the tangent space $T_{\bar{\Sigma}}\mathcal{P}_N$ via the logarithmic map: $V_t = \log_{\bar{\Sigma}}(\Sigma_t)$.
   • The resulting symmetric matrices are flattened into upper-triangular vectors.
3. Geometry Mamba: A dedicated Mamba module processes the sequence of tangent vectors to capture the temporal dynamics of the geometric trajectory, producing modulation parameters ($\mathbf{h}_{geo}$).
4. Sparse Window Alignment: To bridge the granularity gap between window-level geometric features and variable-level tokens, a sparse alignment strategy is employed. Geometric signals are injected only at specific time steps corresponding to the window centers, avoiding the information smearing caused by linear interpolation.

Path B: Modulated Mamba Encoder

This path processes the raw time series while being guided by the geometric signals from Path A:

1. Inverted Embedding: Variables are treated as tokens, and the sequence is transposed so the Mamba scans across variables rather than time steps.
2. Geometry-Modulated Mamba: The core innovation lies in directly injecting geometric signals into the selective state-space parameters ($\mathbf{B}$ and $\mathbf{C}$) of the Mamba VC (Variable Channel) encoder.
   • Instead of treating geometry as an external conditioning signal, the model intercepts the internal SSM parameterization.
   • The geometric signals are projected and additively injected into $\mathbf{B}$ and $\mathbf{C}$: $\mathbf{B} = \mathbf{B}_{base} + \mathbf{W}_{geo}^B(\Delta_{geo})$ and $\mathbf{C} = \mathbf{C}_{base} + \mathbf{W}_{geo}^C(\Delta_{geo})$.
   • This mechanism allows the SSM's input-to-state and state-to-output pathways to adaptively respond to changes in the underlying correlation topology.

Complexity and Efficiency

The architecture preserves the linear time complexity ($O(L)$) of the Mamba parallel scan. While Path A involves eigendecompositions ($O(N^3)$), the authors employ a low-rank projection for high-dimensional datasets to reduce this to $O(R^3)$, ensuring scalability. The overall complexity remains linear with respect to the sequence length and variable count.

3. Key Contributions

The paper outlines three primary contributions:

1. Principle of Manifold-Constrained State-Space Dynamics: The authors demonstrate that treating evolving cross-variable correlations as continuous Riemannian trajectories on the SPD manifold provides effective geometric regularization for SSM-based forecasting.
2. SPDM Architecture: A geometry-aware SSM that realizes this principle by scanning SPD manifold trajectories in the tangent space and directly modulating Mamba's selective parameters via geometric gating. This achieves intrinsic geometric awareness without external conditioning or sacrificing linear complexity.
3. Empirical Validation: Comprehensive experiments on eleven real-world benchmark datasets confirm that manifold-constrained dynamics are the dominant factor behind the model's performance, rather than isolated architectural components.

4. Experimental Results

The authors evaluated SPDM on eleven datasets (including ETTh1/2, ETTm1/2, ECL, Exchange, Weather, PEMS03/07/08, and Illness) covering diverse domains, variable counts (7 to 883), and temporal granularities.

• Forecasting Accuracy: SPDM achieved state-of-the-art (SOTA) or near-SOTA performance across the board. It secured the best average MSE/MAE on ETTh1, ETTh2, Weather, and Illness. On high-dimensional datasets like ECL (321 variables) and PEMS benchmarks, SPDM consistently outperformed strong baselines like S-Mamba, iTransformer, and PatchTST, particularly in capturing complex inter-variable dynamics.
• Ablation Studies:
  • Geometric Injection: Removing the injection of geometric signals into $\mathbf{B}$ and $\mathbf{C}$ caused significant performance degradation, especially on high-dimensional datasets (e.g., a 12.6% MSE increase on ECL), validating the necessity of direct parameter modulation.
  • Alignment Strategy: The proposed sparse window alignment outperformed linear interpolation, confirming that preserving the native temporal resolution of covariance windows is crucial.
  • Regularization: Explicit geodesic smoothness regularization was found to be redundant, as the log-Euclidean Fréchet mean and tangent projection inherently enforce sufficient geometric smoothness.
• Efficiency: SPDM demonstrated superior training speed and memory efficiency compared to Transformer-based and patching-based models, maintaining low GPU memory usage while achieving competitive accuracy.
• Robustness: The model showed high resilience to Gaussian noise, maintaining functional accuracy even under strong perturbations ($\sigma=0.5$), attributed to the geometric regularizing effect of the manifold projection.
• Long-Range Modeling: Unlike baselines that suffer from performance saturation or degradation with increasing lookback windows, SPDM exhibited a smooth, monotonic decrease in error as the lookback length increased.

5. Significance and Claims

The paper claims that SPDM establishes a foundational paradigm for manifold-constrained neural dynamics within state-space architectures. Its significance lies in:

• Bridging Geometry and Efficiency: It successfully integrates Riemannian geometry (specifically SPD manifold constraints) with the linear-time efficiency of Mamba, overcoming the trade-off between geometric fidelity and computational scalability.
• Principled Regularization: The work posits that geometric constraints act as a principled regularizer that guides and stabilizes the selective scanning dynamics of SSMs, allowing the model to distinguish between genuine structural regime shifts and stochastic noise.
• Architectural Intervention: By intervening at the SSM parameterization level (modulating $\mathbf{B}$ and $\mathbf{C}$), SPDM makes geometry a formative component of the sequence modeling process, rather than an auxiliary add-on.

The authors conclude that this approach opens avenues for geometric deep learning on structured non-Euclidean representations, with potential future extensions including handling irregularly sampled signals and leveraging higher-order product manifolds.