⚡ electrical engineering

Data-Driven Regularized Time-Limited h2 Model Reduction from Noisy Impulse Responses

This paper proposes a data-driven, regularized time-limited $H_2$ model reduction method for discrete-time linear systems that utilizes only noisy impulse response data to achieve superior accuracy and robustness compared to unregularized alternatives.

Original authors: Hiroki Sakamoto, Kazuhiro Sato

Published 2026-05-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hiroki Sakamoto, Kazuhiro Sato

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Shrinking a Giant Machine

Imagine you have a massive, incredibly complex machine (like a giant industrial robot or a sophisticated audio system). This machine has millions of moving parts (mathematically, these are "states"). While it works perfectly, it is too heavy and slow to simulate on a laptop. You need a simplified version (a "Reduced-Order Model") that acts just like the big machine but is light enough to run quickly.

Usually, to build this simplified version, you need the blueprints (the exact mathematical equations) of the big machine. But in the real world, you often don't have the blueprints. You only have a black box. All you can do is poke it with a stick (send an impulse) and listen to the sound it makes (the "impulse response").

The Problem: The "Noisy" Listening Room

The paper addresses a specific problem:

The Black Box: We only have data from poking the machine, not the blueprints.
The Time Limit: We only care about how the machine behaves for a short, specific amount of time (like the first 500 seconds), not forever.
The Noise: The data we collect is messy. Imagine trying to hear a whisper in a room where a fan is humming. The "whisper" is the true machine behavior, and the "hum" is random noise.

If you try to build a simplified model using this messy data without care, the model will overfit. This means it will memorize the "hum" (the noise) instead of the "whisper" (the real behavior). It's like a student who memorizes the random scribbles on a test paper instead of learning the actual math, resulting in a model that fails when tested on new situations.

The Solution: A "Smart" Simplification

The authors propose a new method to build this simplified model directly from the noisy data, without ever needing the blueprints. They call it a "Data-Driven Regularized Time-Limited h2 Model Reduction."

Here is how it works, broken down into three simple concepts:

1. The "Time-Limited" Focus

Think of a movie. You don't need to know how the movie ends to understand the first 10 minutes. This method focuses only on the first $L$ seconds of the machine's reaction. It ignores everything that happens after that, ensuring the simplified model is perfect for that specific short window.

2. The "Regularization" (The Safety Net)

This is the paper's main innovation. When the data is noisy, the math tries to fit the noise, which makes the model worse.

The Analogy: Imagine you are trying to draw a smooth curve through a set of dots on a piece of paper. Some dots are where they should be, but others are scattered randomly because your hand was shaking (noise).
Without Regularization: You try to connect every dot perfectly. Your line becomes a jagged, crazy mess.
With Regularization: You add a rule: "The line must be smooth." You are allowed to miss a few noisy dots if it means the line stays smooth and follows the general trend.
In the Paper: They use a "kernel" (a mathematical tool) to encode the idea that real machines usually behave smoothly and settle down over time. This acts as a filter that tells the algorithm, "Ignore the crazy spikes; assume the machine is smooth."

3. The "Gradient" (The Compass)

To find the best simplified model, the computer needs to know which direction to move to improve the result.

The Old Way: Usually, you need the blueprints to calculate this direction.
The New Way: The authors proved a mathematical trick. They showed that you can calculate this "compass direction" (the gradient) using only the noisy sound data you collected. You don't need to know the internal gears of the machine; you just need to listen to the noise and the signal together.

How They Tested It

They tested this method on a standard benchmark called the "CD Player model" (a famous test case in engineering).

Scenario A (Quiet Room): When the data was clean, their method worked well, finding a very accurate simplified model.
Scenario B (Loud Room): When they added a lot of noise (simulating a very messy real-world environment), the standard methods failed. They memorized the noise and produced bad models.
The Winner: The authors' "Regularized" method acted like a noise-canceling headphone. Even with the loud fan humming, it ignored the noise and produced a simplified model that was much closer to the truth than the other methods.

Summary

This paper presents a new way to shrink complex machines into simple models when you only have messy, noisy data and no blueprints. By adding a "smoothness rule" (regularization) to the math, they prevent the model from getting confused by the noise, ensuring the simplified version remains accurate for a specific time window.

1. Problem Statement

The paper addresses the challenge of Model Order Reduction (MOR) for discrete-time Linear Time-Invariant (LTI) systems when the system matrices ( $A, B, C$ ) are unknown and only noisy impulse response data is available.

Context: In many engineering applications, system performance is governed by behavior over a finite time horizon ( $L$ ). Standard MOR techniques often assume access to system matrices or infinite-horizon data.
The Gap: Existing data-driven methods (like the Loewner framework or Eigensystem Realization Algorithm - ERA) often fail to guarantee optimality in the time-limited $h_2$ norm or suffer from overfitting when the available impulse response data is corrupted by noise.
Objective: To develop a method that directly constructs a Reduced-Order Model (ROM) minimizing the time-limited $h_2$ error using only noisy impulse response data, without performing a preliminary system identification step that might introduce bias or error.

2. Methodology

The proposed approach formulates a regularized optimization problem that can be solved entirely from data.

A. Problem Formulation

The goal is to find a ROM with matrices $(\hat{A}, \hat{B}, \hat{C})$ that minimizes the time-limited $h_2$ error between the true system and the ROM.

Objective Function: The standard time-limited $h_2$ objective is reformulated using impulse response data.
Data-Driven Objective: Since the true impulse response $h[k]$ is unknown, the method uses noisy data $\tilde{h}[k] = h[k] + \eta[k]$ . The data-fitting term is constructed directly from $\tilde{h}[k]$ .
Regularization: To prevent overfitting to noise, a regularization term $R(\hat{\theta})$ $R (\hat{θ})$ is added. This term is inspired by kernel-based system identification. It penalizes ROMs whose impulse responses are inconsistent with prior knowledge (e.g., smoothness and stability).
- The regularization uses a TC (Tuned Correlation) kernel, which encodes the prior that the impulse response decays smoothly.
- The total objective function is:
  $J_{L,\lambda}(\hat{\theta}) = \tilde{f}_{L,data}(\hat{\theta}) + \lambda R(\hat{\theta})$
  where $\lambda$ is the regularization parameter.

B. Gradient Derivation (Key Technical Contribution)

A major hurdle in data-driven optimization is computing gradients without system matrices. The authors prove that the gradient of the regularized objective function can be expressed solely in terms of the noisy impulse response data and the current ROM parameters.

They derive explicit formulas for $\nabla_{\hat{A}} J$ , $\nabla_{\hat{B}} J$ , and $\nabla_{\hat{C}} J$ .
Crucially, these formulas do not depend on the dimension $n$ of the original high-order system, making the method scalable for large-scale systems.
This avoids the need to solve Lyapunov or Sylvester equations involving the unknown original matrices, which would be computationally expensive ( $O(n^3)$ ) and impossible without identification.

C. Optimization Algorithm

The paper proposes Algorithm 1, a gradient-based method (specifically a backtracking line search with Armijo condition) to solve the non-convex optimization problem.

Initialization: The algorithm can be initialized with a ROM obtained from standard methods like ERA or random initialization.
Convergence: Theoretical analysis guarantees that the sequence of iterates converges to a stationary point of the objective function.

3. Key Contributions

Formulation of Regularized Time-Limited $h_2$ MOR: The paper establishes a framework for minimizing the time-limited $h_2$ error directly from noisy data, incorporating a kernel-based regularization term to handle noise robustly.
Data-Only Gradient Representation: It provides a novel mathematical derivation showing that both the objective function and its gradient can be computed using only the noisy impulse response sequence, eliminating the need for system identification or access to original system matrices.
Scalability: The gradient computation is independent of the original system dimension $n$ , making it suitable for large-scale systems where traditional model-based approaches are infeasible.
Robustness to Noise: The introduction of the regularization term specifically addresses the overfitting issues common in unregularized data-driven MOR under noisy conditions.

4. Experimental Results

The method was validated using the CD player benchmark from the SLICOT collection (discretized to $n=120$ ).

Low Noise Scenario ( $\sigma = 10^{-3}$ ):
- The proposed method (with or without regularization) successfully reduced the relative time-limited $h_2$ error compared to the initial ERA model.
- Using ERA as an initialization point yielded better final results than random initialization.
High Noise Scenario ( $\sigma = 0.5$ ):
- Unregularized Method ( $\lambda=0$ ): The algorithm overfit the noise, resulting in a final ROM with a higher error than the initial ERA model.
- Regularized Method ( $\lambda > 0$ ): The proposed method successfully suppressed overfitting. By tuning $\lambda$ , the algorithm converged to a ROM with significantly lower error than the initial point and the unregularized version.
Comparison with SysID+ERA:
- The authors compared their direct optimization approach against a two-step pipeline: Kernel-based System Identification followed by ERA (SysID+ERA).
- Result: The proposed direct regularized method outperformed SysID+ERA across various noise levels. This suggests that avoiding the intermediate step of estimating a full impulse response (which introduces estimation error) and optimizing the ROM directly is more effective.
Extreme Noise: At very high noise levels ( $\sigma=50$ ), performance degraded for all methods, indicating a limit to the signal-to-noise ratio the method can handle, though the regularized approach still performed best among the tested alternatives.

5. Significance

This paper is significant because it bridges the gap between optimal model reduction theory and practical data-driven constraints.

Direct Optimization: It moves beyond "identify then reduce" paradigms, offering a "reduce directly from data" approach that is mathematically rigorous regarding the $h_2$ norm.
Noise Robustness: By integrating regularization directly into the MOR objective, it provides a robust solution for real-world scenarios where sensor data is inherently noisy.
Computational Efficiency: The ability to compute gradients without knowing the system dimension $n$ makes this method applicable to very large-scale systems where traditional MOR techniques fail due to memory or computational constraints.

In summary, the authors present a robust, scalable, and theoretically grounded algorithm for constructing optimal reduced-order models from noisy data, demonstrating superior performance over existing identification-based pipelines in noisy environments.