Randomized Atomic Feature Models for Physics-Informed Identification of Dynamic Systems

Technical Summary: Randomized Atomic Feature Models for Physics-Informed Identification of Dynamic Systems

1. Problem Statement

System identification sits at the intersection of data-driven learning and physical modeling. Existing methods generally fall into two categories, each with limitations:

• Sparse Regression (Finite Dictionary): Methods like Sparse Identification of Nonlinear Dynamics (SINDy) use preselected basis functions. While estimation reduces to linear least squares with sparsity penalties, incorporating physical priors (stability, passivity, DC-gain bounds, settling times) requires custom basis functions or complex constraints, often destroying the simplicity of the original formulation.
• Kernel Regularization (Infinite Dictionary): Methods using Reproducing Kernel Hilbert Spaces (RKHS) with kernels like Tuned-Correlated (TC) or Stable-Spline implicitly handle infinite dictionaries and encode stability. However, the atoms are hidden, the Gram matrix scales with data size, and imposing multiple hard engineering constraints simultaneously often requires redesigning the kernel or solving hybrid problems.

The paper addresses the need for a framework that bridges these approaches: one that retains the interpretability of explicit modal dictionaries, the scalability of randomized features, and the ability to enforce diverse physical constraints within a single convex optimization problem.

2. Methodology

The authors propose Randomized Atomic Features (RAF) based on stable atomic features sampled from a prescribed disk in the complex plane.

Core Representation:
The impulse response $h(t)$ is approximated as a random superposition of stable atoms (damped complex exponentials):
$$h(t) \approx \sum_{m=1}^M c_m p_m^t, \quad |p_m| \le \rho < 1$$
where poles $p_m$ are sampled from a distribution over an admissible disk, and residues $c_m$ are learned from data.

Mathematical Framework:

• Atomic Gauge: The method defines a finite atomic gauge $\gamma_{M,T}$ induced by the sampled poles. The $\ell_1$ penalty in the regression is shown to be exactly this finite atomic-gauge penalty.
• Kernel Connection: The empirical kernel formed by sampled poles approximates a continuous disk-supported kernel $K_\mu(s,t) = \int_{|p|\le\rho} p^s \bar{p}^t d\mu(p)$. This connects the approach to Amplitude-Modulated Locally Stationary (AMLS) kernels, providing a random-feature interpretation for a stable modal RKHS.
• Convex Optimization: The identification is cast as a regularized least-squares problem:
  $$\min_{c,D} \frac{1}{2}\|y - Xc - Du\|_2^2 + \lambda \Omega(c)$$
  where $X$ is the design matrix of convolved atoms. Crucially, the pole locations are fixed during this convex stage, making the problem tractable.

Physics-Informed Constraints:
The framework allows physical priors to be imposed either via the sampling law (e.g., restricting poles to specific sectors or annular regions) or via convex constraints on the residues and feedthrough terms. Supported constraints include:

• Stability margins (via $|p_m| \le \rho$).
• Settling-time and tail budgets (via $\ell_1$ tail bounds).
• DC-gain bounds and Markov parameters.
• Passivity (via KYP LMI).
• Monotonicity and no-overshoot (via sign constraints on real poles).
• Frequency-response masks (via SOCP or LMI constraints).

3. Key Contributions

The paper makes five primary theoretical and practical contributions:

1. Unification: It unifies finite modal dictionaries, kernel regularization, and randomized atomic features through the concept of disk-supported pole measures.
2. Disk–Bochner Theorem: The authors prove a new operator-theoretic result. They show that positive measures over stable poles generate positive-definite kernels with a radius-dependent shift defect. Conversely, they prove that a scalar disk moment representation for an arbitrary kernel exists if and only if the canonical shift associated with the kernel is subnormal. This distinguishes simple contractivity from the stronger spectral structure required for a scalar measure representation.
3. Nevanlinna–Pick (NP) Connection: The paper establishes a consistent connection between the RAF framework and NP interpolation. It demonstrates that normalized transfer functions satisfying gain constraints can be parameterized via Linear Fractional Transformations (LFT), allowing the RAF model to serve as an inner approximation of the Schur set-membership problem.
4. Sparse Recovery and Conditioning: The authors analyze the conditioning of the disk-Vandermonde matrix. They highlight that for disk-supported atoms, both angular and radial separation are critical for sparse recovery guarantees, as poles with similar radii and angles can be nearly collinear over finite horizons.
5. Engineering Prior Integration: The framework demonstrates how diverse engineering priors (stability, passivity, monotonicity, etc.) can be encoded simultaneously in a single master convex problem (QP/SOCP/LMI) without redesigning the kernel.

4. Results and Numerical Validation

The paper presents a numerical example on a SISO system with two underdamped modes, excited by low-bandwidth input with additive noise (SNR $\approx$ 30 dB).

• Scenario: The data is "under-informative," meaning many stable models fit the output, but they extrapolate differently.
• Comparison: The RAF method is compared against Prediction-Error Modeling (PEM), Kernel Regularized Methods (KRM), and Atomic Norm Minimization (ANM).
• Findings:
  • With only stability priors, randomized methods (ANM, RAF) fit the data but fail to uniquely recover the true impulse response due to poor excitation.
  • Adding pole-sector constraints and sparsity improves modal recovery for both ANM and RAF.
  • RAF Superiority: When additional convex constraints (DC gain, BIBO budget, frequency masks) are added, RAF significantly improves the recovered impulse response and yields more informative candidate pole regions.
  • The example illustrates that when data is poor, the ability to impose hard engineering priors is more critical than the specific choice of loss function.

5. Significance and Claims

The paper positions RAF not as a replacement for kernel methods or atomic norm minimization, but as a practical bridge between them.

• Interpretability: Unlike pure kernel methods, RAF features are explicit physical modes (damped exponentials), allowing direct interpretation and constraint enforcement.
• Scalability: Unlike gridless atomic norm methods that solve infinite-dimensional problems, RAF reduces the problem to a scalable finite-dimensional convex regression.
• Physics-Informed: The framework explicitly encodes stability margins, modal localization, and time/frequency-domain error bounds directly into the optimization.
• Modest Claims on Recovery: The authors are explicit that the method does not guarantee exact pole recovery from a finite dictionary alone. Active atoms are interpreted as candidate pole regions rather than certified continuous estimates. Exact recovery requires additional assumptions (informative data, sufficient separation, low noise) and often a two-stage strategy where the convex RAF solution serves as a warm start for local nonlinear refinement.

In summary, the paper provides a mathematically rigorous and computationally tractable framework for system identification that leverages randomized features to combine the flexibility of kernel methods with the physical interpretability and constraint-handling capabilities of explicit modal dictionaries.