Predictively-Oriented Kalman Filtering

Technical Summary: Predictively-Oriented Kalman Filtering

Problem Statement
The paper addresses the challenge of online filtering in nonlinear state-space models where the underlying dynamical model ($x_k = f(x_{k-1}, \epsilon_k)$) or the measurement model ($y_k = h(x_k, \xi_k)$) may be misspecified. Standard Bayesian approaches, including the Kalman Filter (KF) and its nonlinear extensions like the Extended Kalman Filter (EKF), rely on the assumption that the model is well-specified. When this assumption fails—due to outliers, unmodeled dynamics, or incorrect parameterization—standard methods often produce over-confident inferences. They tend to collapse uncertainty, leading to poor predictive performance because they cannot adapt to the discrepancy between the model and the true data-generating process. While existing robust methods (e.g., generalised Bayesian approaches like WoLF) attempt to mitigate this by down-weighting outliers or using heavy-tailed noise distributions, they often require user-specified hyper-parameters or fail to address systematic model misspecification effectively.

Methodology
The authors propose EKF-PrO (Predictively-Oriented Extended Kalman Filter), a post-Bayesian filtering framework that departs from strict adherence to Bayes' theorem to prioritize predictive performance.

1. Optimization Perspective: The filtering task is reformulated as an optimization problem over the space of probability distributions. The filtering distribution $q_{k|k}$ is defined as the minimizer of an objective function comprising a loss term $\ell(y_k; q)$ and a Kullback-Leibler divergence (KLD) regularizer $KL(q \| q_{k|k-1})$.
2. Predictively-Oriented (PrO) Loss: Unlike standard Bayesian filtering which uses negative log-likelihood, the PrO filter employs a loss function defined as:
   $$\ell^{(PrO)}(y_k; q) := -\log \left( \int p(y_k|x_k)q(x_k)dx_k \right)$$
   This formulation treats the filtering distribution $q$ as the parameter of a mixture model. Minimizing this loss is equivalent to minimizing the KLD between the model's predictive distribution and the true data-generating distribution, thereby automatically calibrating uncertainty even under misspecification.
3. Approximate Linear-Gaussian Update: Since the PrO objective does not yield a closed-form solution for general state-space models, the authors develop a fast approximate update analogous to the EKF.
   • Prediction Step: Follows standard KF/EKF procedures using the Chapman-Kolmogorov equation (or linearization for nonlinear $f$).
   • Update Step: The authors restrict the search space to Gaussian distributions (variational family). They derive an explicit form for the objective function, revealing a Difference-of-Convex (DoC) structure. This allows the use of efficient numerical optimization techniques to solve for the posterior mean and covariance.
   • Nonlinear Extension (EKF-PrO): For nonlinear models, the method linearizes the measurement function $h$ around the current iterate of the optimization path (similar to the Iterated EKF) rather than just the predictive mean, leveraging the DoC structure to refine the update.

Key Contributions

• Hyper-parameter-Free Robustness: The EKF-PrO method requires no tunable hyper-parameters, contrasting with many robust filtering techniques that rely on user-specified weights or noise covariance inflation factors.
• Theoretical Calibration: The approach ensures that learning (posterior concentration) occurs if and only if the model is well-specified. In misspecified settings, it avoids the "collapse of uncertainty" typical of Bayesian methods, maintaining higher posterior variance to reflect model inadequacy.
• Computational Efficiency: The authors provide a fast approximate update procedure with computational complexity comparable to existing EKF variants ($O(I d^3)$, where $I$ is the number of inner-loop iterations), making it suitable for real-time applications.
• DoC Optimization: The derivation of the PrO objective as a Difference-of-Convex function enables efficient numerical minimization for the covariance update, a key technical hurdle in applying PrO posteriors to state-space models.

Empirical Results
The paper evaluates EKF-PrO against standard EKF, Iterated EKF, and state-of-the-art robust filters (EKF-B, EKF-IW, WoLF-IMQ, WoLF-TMD) across three scenarios:

1. 2D Object Tracking: In scenarios with systematic misspecification (e.g., Student-t process noise) or maneuvering dynamics, EKF-PrO significantly outperforms standard EKF and outlier-robust methods like WoLF. While WoLF performs well when only the measurement model is misspecified (outliers), it struggles with systematic dynamical errors.
2. Lorenz96 Chaotic System: In a high-dimensional ($d=60$) nonlinear setting with systematic misspecification, EKF-PrO demonstrates superior performance. The study notes that outlier-robust filters can sometimes worsen performance in systematic misspecification by rejecting valid but unexpected observations, whereas EKF-PrO adapts its uncertainty without discarding data.
3. Conservativity: Theoretical analysis (Proposition 4) and empirical results confirm that EKF-PrO is more conservative than KF; it updates the mean less aggressively in the presence of outliers and inflates the covariance more significantly, leading to better long-term predictions.

Significance and Claims
The paper positions EKF-PrO as a necessary alternative to (generalised) Bayesian inference in settings where model misspecification is generic and unavoidable. The authors argue that rather than attempting to fit a more complex model to force well-calibrated posteriors, one should directly target predictive performance.

The significance of the work lies in bridging the gap between the theoretical benefits of predictively-oriented posteriors (which offer robustness without over-confidence) and the practical need for fast, online filtering algorithms. The authors modestly acknowledge that while the method is effective, the theoretical properties (such as statistical efficiency and dimension dependence) are not fully understood, and the computational cost is currently 10–20 times higher than standard KF, though still comparable to other robust baselines. They view the development of this efficient update procedure as the primary technical contribution, enabling the application of PrO posteriors to dynamic state-space problems.