Neural Diffusion Intensity Models for Point Process Data

This paper introduces Neural Diffusion Intensity Models, a variational framework that leverages neural SDEs and a theoretical drift correction to enable efficient, amortized inference of latent intensity paths in Cox processes, achieving accurate posterior estimation with orders-of-magnitude speedups over traditional MCMC methods.

The Gist

The Big Picture: Predicting the Unpredictable

Imagine you are a manager at a busy bank call center. You have a log of every single phone call that comes in. Some minutes are quiet; some are chaotic. You want to understand why the calls happen the way they do.

Is it just random noise? Or is there an underlying "mood" or "pressure" in the system that makes calls cluster together?

This paper introduces a new tool to figure out that hidden "mood." It calls this tool Neural Diffusion Intensity Models.

---

The Problem: The "Ghost" in the Machine

To understand the paper, we first need to understand the problem it solves.

1. The "Overdispersion" Mystery
If you look at the call data, you'll notice something weird. Sometimes, 10 calls come in one minute, and then zero for the next 10. If calls were purely random (like raindrops hitting a roof), the numbers would be more consistent. But in reality, the numbers swing wildly. This is called overdispersion.

2. The Hidden "Intensity"
The authors say this happens because there is a hidden force, which they call the Intensity. Think of the Intensity as a hidden weather system.

• When the "weather" is stormy (high intensity), calls come in a flood.
• When the "weather" is calm (low intensity), calls are sparse.

The problem is: We can't see the weather. We only see the rain (the calls). We have to guess what the weather was like just by looking at the puddles on the ground.

3. The Old Way: The Slow Detective
Previously, to guess the weather, statisticians used a method called MCMC (Markov Chain Monte Carlo).

• The Analogy: Imagine trying to guess the weather by throwing a dart at a map of the sky, checking if it matches the rain, and if not, throwing another dart. You have to throw millions of darts to get a good guess.
• The Downside: It's incredibly slow. If you want to know the weather for a new day, you have to start throwing darts all over again. It's like trying to solve a Sudoku puzzle by guessing every number from scratch every time you want to play.

---

The Solution: The "Smart GPS"

The authors propose a new method called Neural Diffusion Intensity Models. Instead of throwing darts, they build a Smart GPS that learns the rules of the weather.

Here is how it works, broken down into three simple steps:

1. The "Neural SDE" (The Weather Simulator)

They use a Neural Network (a type of AI) to learn the rules of how the "weather" (Intensity) changes over time.

• Analogy: Imagine a video game character (the AI) learning how wind and rain move. It learns that "if the pressure is high, the wind usually blows left."
• The Math Bit: They call this a "Stochastic Differential Equation" (SDE). In plain English, it's just a formula that describes how the hidden intensity drifts and jiggles over time.

2. The "Drift Correction" (The Magic Trick)

This is the paper's biggest breakthrough. They discovered a mathematical rule (using something called Enlargement of Filtrations) that says:

• If you know the rain fell at specific times, you can mathematically "correct" your weather forecast to match exactly what happened.
• Analogy: Imagine you are driving a car (the weather) and you see a pothole (a phone call). The old way was to stop the car, look at the map, and guess where the pothole came from. The new way is to have a GPS that instantly adjusts your steering wheel the moment you see a pothole, keeping you on the perfect path without stopping.
• The Result: This "correction" turns the messy, hard-to-solve problem into a clean, smooth path that the AI can follow instantly.

3. Amortized Inference (The One-Time Learning)

This is the "killer feature."

• The Old Way: Every time you get new data (a new day of calls), you have to run the slow "dart-throwing" simulation again.
• The New Way: The AI learns a single map (the "encoder") that works for any day of calls.
• Analogy: Instead of learning to drive a new car every time you go to the grocery store, you learn the rules of driving once. Now, whenever you get in a car (new data), you just drive. You don't need to re-learn how to steer.
• The Speedup: This makes the process 10 to 100 times faster than the old methods.

---

Why Does This Matter?

The paper tested this on fake data and real data from a US Bank call center.

1. It's Accurate: It figured out the hidden "weather" (intensity) almost perfectly, matching the slow, expensive methods.
2. It's Fast: It did the job in seconds that used to take hours.
3. It's Flexible: It can handle complex patterns, like the fact that call centers get busy at 9 AM and quiet at 2 PM, without needing a human to write specific rules for those times.

The Takeaway

Think of this paper as upgrading from a slow, manual map to a real-time, self-driving GPS for understanding random events.

• Old Method: "Let's guess the weather by throwing darts for 5 hours."
• New Method: "Let's teach an AI the laws of physics so it can drive us to the answer in 5 seconds."

This allows businesses and scientists to understand complex, messy data (like stock market crashes, disease outbreaks, or call center spikes) in real-time, rather than waiting days for a computer to finish its calculations.

Technical Summary

1. Problem Statement

The paper addresses the challenge of modeling and inferring overdispersed point process data (e.g., call center arrivals, neural spikes, financial transactions). Standard inhomogeneous Poisson processes often fail to capture the high variability (overdispersion) observed in real-world data.

• The Model: The authors propose using Cox processes (doubly stochastic Poisson processes), where event arrivals follow a Poisson process with a latent stochastic intensity $Z_t$.
• The Dynamics: The intensity $Z_t$ is modeled as a solution to a Stochastic Differential Equation (SDE) (a diffusion process), allowing for interpretable dynamic mechanisms like mean reversion or state-dependent volatility.
• The Challenge:
  1. Intractability: Estimating the parameters of the latent SDE and performing posterior inference over the intensity paths are computationally intractable.
  2. Computational Cost: Existing methods rely on Markov Chain Monte Carlo (MCMC) for posterior inference. This is expensive, requiring repeated simulations for every new observation, making real-time or large-scale inference impractical.
  3. Variational Gaps: Previous variational approaches often rely on Gaussian approximations or heuristics that do not guarantee the true posterior is within the variational family, leading to a "variational gap."

2. Methodology: Neural Diffusion Intensity Models (NDIM)

The authors introduce a variational framework driven by Neural SDEs that enables amortized inference (fast, single-pass inference) while theoretically guaranteeing the variational family contains the true posterior.

A. Model Architecture

• Prior (Decoder): The latent intensity $Z_t$ follows a Markov diffusion:
  $$dZ_t = b_\theta(Z_t, t) dt + \sigma(Z_t, t) dB_t$$
  Here, the drift function $b_\theta$ is parameterized by a Neural Network (MLP), providing a flexible, nonparametric prior over intensity paths.
• Variational Posterior (Encoder): The posterior distribution $Q_\phi(Z|X)$ is modeled as the solution to a drift-corrected SDE:
  $$dZ_t = [b_\theta(Z_t, t) + \sigma(Z_t, t) u_\beta(Z_t, t, T', X)] dt + \sigma(Z_t, t) d\tilde{B}_t$$
  The term $u_\beta$ is a neural network that learns an observation-dependent drift correction. Crucially, the diffusion coefficient $\sigma$ remains unchanged.

B. Theoretical Foundation: Enlargement of Filtrations (EoF)

The core theoretical contribution is the use of Enlargement of Filtrations to characterize the true posterior.

• Key Insight: By conditioning on the observed point process trajectory $X$ (viewed as "future" information relative to the latent path), the posterior intensity process remains a diffusion with the same diffusion coefficient but a modified drift.
• The Result (Theorem 2.1): The true posterior drift is the prior drift plus a score-type correction term:
  $$h(z, t, X) = \frac{\partial}{\partial z} \log \mathbb{E} \left[ \exp\left(-\int_t^{T'} Z_s ds\right) \prod_{t < \tau_i \leq T'} Z_{\tau_i} \mid Z_t = z \right]$$
• Significance: This establishes a conjugacy at the level of path measures. If the neural network $u_\beta$ is sufficiently expressive, the variational family contains the true posterior. Consequently, maximizing the Evidence Lower Bound (ELBO) is equivalent to Maximum Likelihood Estimation (MLE), and the variational gap vanishes.

C. Amortized Inference & Architecture

• Amortization: Once trained, inferring the posterior for a new observation $X'$ requires only a single forward pass (simulating the SDE with the learned $u_\beta$), replacing hours of MCMC sampling with milliseconds of simulation.
• Deep Sets Architecture: Since the drift correction $u_\beta$ depends on the set of future event times $\{\tau_i\}$, the authors use a Deep Sets architecture (permutation-invariant pooling) to process variable-length event sequences without losing temporal structure.

3. Key Contributions

1. Theoretical Guarantee: Proved via EoF that conditioning a diffusion-driven Cox process on point process observations preserves the diffusion structure, identifying the exact functional form of the posterior drift correction.
2. Zero Variational Gap: Demonstrated that under sufficient model capacity, the variational family contains the true posterior, ensuring ELBO maximization coincides with MLE.
3. Amortized Inference: Developed a framework that replaces iterative MCMC with a single SDE simulation, achieving orders-of-magnitude speedups.
4. Neural SDE Priors: Combined flexible neural network parameterizations for drift functions with rigorous stochastic calculus to model complex intensity dynamics.

4. Experimental Results

The authors evaluated the method on synthetic data (CIR process) and a real-world dataset (U.S. Bank call center records).

• Prior Recovery: The model successfully learned the ground-truth drift function of a Cox-Ingersoll-Ross (CIR) process, capturing global dynamics even with local deviations.
• Posterior Accuracy:
  • The learned variational posterior paths closely matched "gold-standard" MCMC approximations.
  • The method generalized well to unseen data once the training set size exceeded a threshold (avoiding overfitting of the amortized encoder).
• Computational Efficiency:
  • Speed: The variational method was 10x to 100x faster than MCMC-based EM methods for posterior inference.
  • Scalability: While MCMC requires re-running for every new observation, the NDIM approach performs inference in a single forward pass.
• Real-World Application: On the bank call center data, the model captured complex temporal patterns (peak hours) and overdispersion, outperforming simple Poisson models.

5. Significance

This work bridges the gap between rigorous stochastic calculus and modern deep learning for point processes.

• Practical Impact: It makes Bayesian inference for complex, overdispersed point processes feasible for real-time applications (e.g., real-time fraud detection, dynamic resource allocation in call centers) where MCMC is too slow.
• Theoretical Advancement: It moves beyond heuristic variational approximations by providing a structural proof that the posterior remains a diffusion, offering a principled target for learning.
• Generalizability: The EoF-based approach is applicable beyond Cox processes to other latent diffusion models conditioned on observations, potentially extending to broader classes of stochastic models.

In summary, Neural Diffusion Intensity Models provide a scalable, theoretically grounded, and highly accurate framework for learning and inferring latent intensity dynamics in overdispersed point process data.