Gradient-based filtering under misspecification: Stability and error bounds

1. Problem Statement

The paper addresses the challenge of tracking multidimensional, time-varying parameters in the presence of noisy observations and potential model misspecification.

• Context: In fields like econometrics and finance, variables often evolve over time. Standard filtering techniques (e.g., Kalman filters) assume specific dynamic structures. However, researchers often use Score-Driven (SD) filters (also known as Dynamic Conditional Score or Generalized Autoregressive Score models) where parameters are updated based on the gradient of a postulated log-likelihood.
• The Gap: Existing literature largely focuses on explicit gradient methods (evaluating the gradient at the predicted parameter) and often assumes the model is correctly specified or provides only single-step guarantees. There is a lack of theoretical understanding regarding the stability and error bounds of these filters when the model is misspecified, particularly in multivariate settings and when using implicit gradient methods.
• Core Question: Can stochastic gradient methods (specifically SD filters) reliably track a moving target when the underlying data-generating process (DGP) is unknown or misspecified, and how do explicit versus implicit formulations compare?

2. Methodology

The authors analyze two variants of score-driven filters:

1. Explicit Score-Driven (ESD) Filters: Update the parameter $\theta_{t|t}$ using the gradient evaluated at the predicted parameter $\theta_{t|t-1}$.
   $$\theta_{t|t}^{ex} = \theta_{t|t-1}^{ex} + H_t \nabla \ell(y_t | \theta_{t|t-1}^{ex})$$
2. Implicit Score-Driven (ISD) Filters: Update the parameter using the gradient evaluated at the updated parameter $\theta_{t|t}$. This is equivalent to solving an optimization problem with a quadratic penalty.
   $$\theta_{t|t}^{im} = \theta_{t|t-1}^{im} + H_t \nabla \ell(y_t | \theta_{t|t}^{im})$$
   $$\theta_{t|t}^{im} = \arg\max_{\theta} \left\{ \ell(y_t | \theta) - \frac{1}{2} \|\theta - \theta_{t|t-1}^{im}\|_{P_t}^2 \right\}$$

Theoretical Framework:

• Stability Analysis: The authors derive sufficient conditions for exponential stability. This means that two filter paths starting from different initial values but driven by the same data converge exponentially fast to each other. Crucially, these conditions depend only on the postulated density and filter parameters, not on the true DGP.
• Error Bounds: Under mild moment conditions on the DGP (existence of a pseudo-true parameter with finite variance increments), the authors derive finite-sample and asymptotic Mean Squared Error (MSE) bounds. These bounds measure the distance between the filtered path and the "pseudo-true" parameter path (the best possible approximation given the misspecified model).
• Key Assumptions:
  • Assumption 1 (Regularity): Bounds on the Hessian of the log-likelihood (defining strong concavity $\alpha$ and Lipschitz continuity $\beta$).
  • Assumption 2 (Moment Conditions): Constraints on the pseudo-true parameter's increments and the score's variance.

3. Key Contributions

A. Novel Stability Conditions (Theorem 1)

The paper establishes new, verifiable sufficient conditions for the exponential stability of multivariate SD filters under arbitrary misspecification.

• Implicit Filters (ISD): Stability is guaranteed if the penalty matrix eigenvalues are sufficiently large relative to the non-concavity of the log-likelihood. These conditions hold even if the log-likelihood is non-concave, provided the penalty dominates the curvature.
• Explicit Filters (ESD): Stability requires Lipschitz continuity of the score (bounded Hessian, $\beta < \infty$) and a learning rate that satisfies $\lambda_{\max}(H) \leq 2/\beta$.
• Insight: The paper highlights that the Lipschitz condition is often overlooked in SD literature but is critical for explicit filters. If violated (e.g., in Poisson models with time-varying intensity), explicit filters can diverge.

B. Finite-Sample and Asymptotic MSE Bounds (Theorem 2)

The authors provide formal guarantees for tracking accuracy under misspecification.

• They derive recursive bounds on the weighted MSE relative to the pseudo-true parameter.
• The bounds are composed of terms related to initialization, score noise, and state drift.
• Optimization: The bounds can be minimized analytically with respect to tuning parameters (like the learning rate), offering a practical guide for hyperparameter selection.

C. Superiority of Implicit Methods

A central theoretical finding is that Implicit (ISD) filters require substantially weaker assumptions than Explicit (ESD) filters.

• ISD filters do not require the score to be Lipschitz continuous.
• ISD filters remain stable even when the Hessian is unbounded (e.g., $\beta = \infty$), whereas ESD filters become unstable or diverge in such scenarios.

4. Results from Simulation Studies

The authors validate their theory through three Monte Carlo experiments:

1. High-Dimensional Linear Model (Least Squares):

   • Compared against recent alternatives (Nesterov's ONM, Madden et al., Cutler et al.).
   • Result: The ISD filter achieved the lowest empirical and theoretical MSEs across all dimensions and time steps. Notably, the ISD filter's performance improved as the Lipschitz constant increased, whereas ESD filters were constrained by the learning rate cap.

2. Nine Nonlinear DGPs (Koopman et al., 2016):

   • Tested various distributions (Poisson, Negative Binomial, Gamma, etc.) with low, medium, and high state volatility.
   • Result:
     • Low Volatility: Both filters performed similarly.
     • Medium/High Volatility: ESD filters frequently became unstable or diverged in models where the score was not Lipschitz continuous (e.g., Poisson, Exponential, Gamma).
     • ISD Performance: The ISD filter remained stable and accurate across all settings, including those where ESD filters diverged.

3. Dynamic Poisson Count Model:

   • A classic case where the score is not Lipschitz continuous ($\beta = \infty$).
   • Result: All variants of the ESD filter diverged as state volatility increased. The ISD filter remained stable and accurately tracked the pseudo-true state, even with a misspecified link function (quadratic vs. exponential).

5. Significance and Implications

• Robustness to Misspecification: The paper provides the first uniform theoretical guarantees for SD filters that hold regardless of the true data-generating process. This is crucial for real-world applications where models are rarely perfectly specified.
• The "Implicit" Advantage: The study rigorously demonstrates that implicit gradient methods (proximal methods) offer superior stability and performance guarantees compared to explicit methods, particularly in non-linear and high-volatility environments.
• Practical Guidance: The derived error bounds offer a method to analytically tune learning rates and penalty matrices to minimize tracking error, moving beyond heuristic selection.
• Resolution of Divergence Issues: The paper explains why explicit SD filters sometimes fail in practice (violation of Lipschitz conditions) and offers the implicit formulation as a theoretically grounded solution that prevents divergence.

In summary, this paper bridges the gap between stochastic optimization theory and time-series filtering, proving that implicit score-driven filters are a more robust and theoretically sound choice for tracking time-varying parameters under model misspecification.