📊 statistics

Function-Space Priors for Bayesian Neural ODEs with Application to Vessel Trajectory Prediction

This paper proposes a novel framework for Bayesian Neural ODEs that improves vessel trajectory prediction by augmenting standard weight-space priors with a function-space Gaussian process regularizer on the vector field, combined with probabilistic multiple shooting to effectively handle irregular and long-term AIS data while providing well-calibrated uncertainty estimates.

Original authors: Jaeyeong Lee, Wonmo Koo, Heeyoung Kim

Published 2026-06-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jaeyeong Lee, Wonmo Koo, Heeyoung Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict where a ship will go next. You have a logbook of its past positions (from a system called AIS), but the entries are messy: sometimes they come every minute, sometimes every hour, and sometimes the logbook has blank pages where the signal was lost.

The goal is to draw a smooth line through these messy dots to guess the future path, but with a crucial twist: you also need to know how sure you are about that guess. If the ship is in a stormy, unpredictable area, your prediction should say, "I'm not very sure," rather than confidently pointing to a spot that might be wrong.

This paper introduces a new way to teach a computer (specifically, a "Bayesian Neural ODE") to do this job better. Here is how they did it, explained simply:

1. The Problem with the Old Way

Think of the computer's "brain" (the neural network) as a recipe for how the ship moves.

The Old Method: The computer was taught to follow a recipe where every ingredient (the numbers inside the brain) was just randomly chosen from a standard bag of flour. This is like baking a cake without caring if the texture is smooth or lumpy. It works, but it doesn't understand that ships usually move smoothly and don't teleport.
The Limitation: Because the "recipe" wasn't forced to be smooth, the computer sometimes got confused by the messy data and made wild guesses about where the ship would go.

2. The New Idea: "The Smoothness Rule"

The authors realized that ships follow rules: they don't usually jerk around instantly; they turn gradually. They wanted to teach the computer this rule directly.

Instead of just checking the ingredients (the weights), they decided to check the output of the recipe at specific "checkpoints."

The Analogy: Imagine you are teaching a student to draw a curve.
- Old way: You tell them, "Here are the rules for your pencil grip."
- New way: You tell them, "Here are 20 specific spots on the paper. If you draw a line through these spots, it must look smooth and follow the shape of a river, not a jagged lightning bolt."

They call this a "Function-Space Prior." They placed a "smoothness rule" (based on a mathematical concept called a Gaussian Process) directly on the ship's movement rules at these specific checkpoints.

3. Handling the Messy Data: "The Relay Race"

Real ship journeys are long and the data is broken up. If you try to calculate the whole trip in one go, small errors pile up like a snowball rolling down a hill, making the prediction useless.

The Solution: They used a technique called "Probabilistic Multiple Shooting."
The Analogy: Instead of one runner trying to run the whole marathon, they broke the race into short legs.
- They divided the ship's journey into short segments.
- They let the computer solve each short segment independently.
- Then, they used a "soft handshake" to make sure the end of one segment matched the start of the next.
- This keeps the math stable and prevents the "snowball of errors" from getting too big.

4. Where to Place the Checkpoints?

The authors tested two ways to choose those 20 "checkpoints" where they enforce the smoothness rule:

Random Clustering: Picking spots where the ship is usually found (like picking random towns on a map).
Maneuver-Weighted: Picking spots where the ship is turning or slowing down.
- Why? This is like putting extra guardrails on a sharp curve on a highway rather than on a straight road. The paper found that focusing on these "turning points" gave the best results because that's where the ship's behavior is most complex and needs the most guidance.

5. The Results

When they tested this new method on real ship data from New York Harbor:

Accuracy: It predicted the ship's future path better than the old methods.
Confidence: It gave much better "uncertainty estimates."
- The Visual: When they looked at the "confidence zone" (the shaded area showing where the ship might be), the old method's zone exploded outward wildly, covering miles of ocean in minutes. The new method's zone grew slowly and smoothly, which is much more realistic. It knew when it was guessing and when it was sure.

Summary

The paper didn't invent a new type of ship or a new way to sail. Instead, they improved the mathematical compass used to predict where ships go. By forcing the computer to respect the natural "smoothness" of ship movements at key turning points, and by breaking long trips into manageable chunks, they created a system that is both more accurate and more honest about how sure it is.

Technical Summary: Function-Space Priors for Bayesian Neural ODEs with Application to Vessel Trajectory Prediction

Problem Statement
Vessel trajectory prediction using Automatic Identification System (AIS) data is critical for maritime situational awareness but faces significant challenges. AIS data is characterized by irregular sampling, missing reports, and complex, nonlinear dynamics driven by navigation rules and environmental factors. While accurate point forecasts are necessary, maritime decision-making also requires well-calibrated uncertainty estimates to handle safety-critical tasks like collision risk assessment.

Standard Bayesian Neural Ordinary Differential Equations (Neural ODEs) offer a framework for continuous-time modeling with uncertainty quantification by placing priors over neural vector field parameters. However, the standard approach relies on weight-space priors (e.g., isotropic Gaussian distributions over weights), which fail to encode interpretable structural properties of vessel dynamics, such as smoothness and spatial locality. Conversely, function-space priors (e.g., Gaussian Processes) can encode these properties but face a fundamental challenge in the Neural ODE setting: the observable quantity is the trajectory (the ODE solution), not the vector field itself. Placing a GP prior directly on trajectories requires propagating distributions through a nonlinear ODE solver, which is analytically intractable. Existing function-space Bayesian neural network methods for static mappings do not transfer directly to Neural ODEs without numerical instability.

Methodology
The authors propose a practical approach that imposes a Gaussian Process (GP) kernel-based prior directly on the neural vector field evaluated at a finite set of measurement points, rather than on the trajectories themselves.

Latent ODE Formulation: Vessel dynamics are modeled as a continuous-time system $dx(t)/dt = f_\theta(x(t), t)$ , where $f_\theta$ is a neural vector field. Uncertainty is quantified via approximate Bayesian inference on the parameters $\theta$ .
Probabilistic Multiple Shooting: To handle long, irregular trajectories and ensure training stability, the authors employ probabilistic multiple shooting. This partitions the trajectory into segments with learnable boundary states, decoupling inference across segments while maintaining global temporal consistency through soft continuity constraints.
Function-Space Regularization:
- A fixed set of measurement points $Z = \{z^{(m)}\}_{m=1}^M$ is defined in the state space.
- A GP prior is imposed on the vector field evaluations $F_Z(\theta)$ at these points: $\text{vec}(F_Z(\theta)) \sim \mathcal{N}(0, K_Z \otimes I_d)$ .
- This induces a regularizer $R_{FS}(\theta)$ added to the variational objective. This term penalizes deviations of the vector field from the structure implied by the GP prior (e.g., smoothness) without requiring a tractable trajectory-level prior.
- The authors note that while this is a weaker condition than a full trajectory-level GP prior, it provides a tractable and interpretable inductive bias.
Measurement Point Selection: Two strategies are investigated for selecting $Z$ : (i) $k$ -means clustering on training trajectory states, and (ii) maneuver-weighted sampling that oversamples states near turning or deceleration events.
Optimization: The objective combines the standard weight-space Evidence Lower Bound (ELBO) with the function-space regularizer. The authors interpret this as Generalized Variational Inference, avoiding the infinite KL divergence issues associated with direct function-space priors on parametric families.

Key Contributions

Function-Space Regularization for Bayesian Neural ODEs: The paper introduces a method to incorporate GP-kernel-based priors into the variational objective of Neural ODEs. This enables interpretable structural bias (e.g., smoothness) on the learned dynamics while retaining the flexibility of neural network parameterizations.
Empirical Comparison: The authors systematically compare their proposed method against standard weight-space Bayesian Neural ODEs and Gaussian Process ODE (GPODE) baselines using real-world AIS data.
Analysis of Measurement Point Design: The study investigates practical strategies for selecting measurement points, demonstrating that concentrating points near dynamic events (maneuvers) improves prediction quality.

Experimental Results
The method was evaluated on AIS data from the New York/New Jersey Harbor region (January 2024), involving 1,589 vessel trajectories. The protocol involved per-trajectory inference with short history windows (10–20 minutes) and a 10-minute prediction horizon.

Predictive Accuracy: Both variants of the proposed method (FS-BayesNODE) significantly outperformed GPODE in point prediction accuracy. The maneuver-weighted variant achieved the lowest Average Displacement Error (ADE: 2.66 km) and Final Displacement Error (FDE: 3.82 km), reducing error by approximately 40% compared to GPODE.
Uncertainty Calibration: All Bayesian Neural ODE variants demonstrated substantially better probabilistic forecasting metrics (NLL and CRPS) compared to GPODE. Specifically, the function-space regularizer improved the Continuous Ranked Probability Score (CRPS) by roughly 69% compared to GPODE.
Qualitative Analysis: Visual inspection of predictive uncertainty showed that GPODE produced uncertainty bands that expanded abruptly and excessively over the prediction horizon. In contrast, the proposed method produced uncertainty that grew smoothly and gradually, reflecting a more realistic increase in epistemic uncertainty.
Measurement Points: The maneuver-weighted selection strategy outperformed the $k$ -means strategy, confirming that placing measurement points near dynamic events provides more informative regularization.

Significance and Claims
The paper claims that by augmenting the standard weight-space variational objective with a kernel-based function-space regularizer, it is possible to inject interpretable structural bias directly into the inference procedure of Bayesian Neural ODEs. The authors assert that this approach successfully addresses the limitations of weight-space priors (lack of structural encoding) and the intractability of direct trajectory-level GP priors in ODE settings.

The work demonstrates that this practical approach improves both predictive accuracy and uncertainty calibration for vessel trajectory prediction under realistic AIS conditions (irregular sampling, missing data). The authors position this as the first empirical exploration of function-space priors in the Neural ODE setting, offering a scalable alternative to GPODEs that avoids cubic computational scaling with inducing points while providing better-calibrated uncertainty estimates.