Multivariate Spatio-Temporal Neural Hawkes Processes

The Big Idea: Predicting the "When" and "Where" of Events

Imagine you are trying to predict the next big thing that will happen in a crowded city. Is it a traffic jam? A flash mob? A protest?

In the world of data science, these events are called Point Processes. The authors of this paper are trying to build a super-smart crystal ball that doesn't just guess when an event will happen, but also where it will happen and what kind of event it will be.

They call their new invention the Multivariate Spatio-Temporal Neural Hawkes Process (MSTNHP). That's a mouthful, so let's break it down.

1. The Problem: The "Blind" Crystal Ball

For a long time, scientists used models to predict events (like earthquakes, crimes, or tweets).

The Old Way: They used rigid rules. Imagine a recipe that says, "If an earthquake happens here, another one will happen exactly 5 miles away in 2 days." The problem? Real life isn't that rigid. Sometimes an earthquake triggers a landslide 10 miles away; sometimes it triggers nothing.
The "Deep Learning" Way: Recently, scientists tried using AI (Neural Networks) to learn these patterns. They built "Temporal" models.
- The Flaw: These AI models were like blindfolded detectives. They could tell you when the next event might happen, but they couldn't see where. They treated the whole world as a single line of time, ignoring the map.

The Paper's Discovery: The authors tested these "blind" AI models on fake data. They found that while the models got good scores on paper (mathematically), they were actually hallucinating. They couldn't figure out the real cause-and-effect relationships. It was like a student memorizing the answers to a test without understanding the subject.

2. The Solution: Giving the AI "Eyes" and a "Memory"

The authors propose a new model that fixes this by adding two superpowers to the AI:

A. The "Spatio-Temporal" Superpower (Eyes)

Instead of just looking at a timeline, the new model looks at a 3D movie (Time + Space + Event Type).

Analogy: Imagine a security guard watching a city.
- Old Model: The guard only hears a siren and knows "something happened."
- New Model: The guard sees the siren, knows it's a fire truck, sees it's in the North District, and knows that fires in the North District often cause traffic jams in the East District 10 minutes later.

B. The "Neural Hawkes" Superpower (Memory)

The model uses a special type of AI memory called a Continuous-Time LSTM.

Analogy: Think of a leaky bucket.
- Every time an event happens (like a tweet or a crime), someone drops a rock into the bucket, raising the water level (increasing the chance of another event).
- Over time, the water leaks out (the excitement fades).
- The Magic: The old models had rigid buckets with fixed holes. The new model has a smart bucket that can change the size of the hole and the shape of the bucket depending on where the rock fell. If a rock falls in a crowded area, the bucket might fill up faster. If it falls in a quiet area, it might leak faster.

3. The Real-World Test: Terrorists in Pakistan

To prove their model works, they tested it on real data: Terrorist attacks in Pakistan between 2008 and 2020.

They looked at four different terrorist groups:

TTP (The big, aggressive group).
BRA, BLA, BLF (Three smaller, separatist groups).

What they found:

The "Blind" Model (Old Way): When they ignored the map and just looked at the dates, the model got confused. It couldn't tell that Group A attacks in the mountains often trigger Group B to attack in the valleys a week later. It just saw a messy pile of dates.
The New Model (MSTNHP): It successfully mapped out the invisible connections.
- It noticed that when the TTP group attacks, it often suppresses (inhibits) the other groups for a while (maybe because they are scared or busy).
- It saw that the three smaller groups often trigger each other in specific regions.
- It created a "heat map" showing exactly where and when the next attack was likely to happen, based on the complex dance between these groups.

4. Why This Matters

The paper argues that accuracy isn't just about getting the right number; it's about understanding the story.

The Metaphor: Imagine two weather forecasters.
- Forecaster A predicts it will rain tomorrow with 90% accuracy. But they don't know where it will rain.
- Forecaster B predicts it will rain in the valley but not on the mountain, with 85% accuracy.
- Forecaster B is more useful. They understand the dynamics of the weather.

The authors show that their new model (Forecaster B) captures the true dynamics of how events influence each other across space and time, whereas the old models (Forecaster A) just guess the total volume of events without understanding the cause.

Summary in One Sentence

The authors built a smarter AI that doesn't just count events on a timeline, but actually "sees" the map and understands how one event in one place can trigger a chain reaction in another place, making it much better at predicting complex real-world chaos like terrorism, earthquakes, or viral trends.

1. Problem Statement

The paper addresses the challenge of modeling multivariate spatio-temporal point patterns where events exhibit complex, dynamic interactions (both excitation and inhibition) across time and space.

Limitations of Existing Models: Traditional parametric Hawkes processes rely on fixed triggering kernels (e.g., exponential or Gaussian decay), which lack flexibility to capture dynamic changes in interaction structures. While recent deep learning approaches (Neural Hawkes Processes) have improved temporal modeling, they often fail to effectively integrate spatial dimensions or capture cross-triggering structures between different event types in a spatio-temporal context.
The "Intensity Gap": The authors identify a critical flaw in current evaluation metrics. Many neural point process models achieve high log-likelihood scores but fail to recover the true conditional intensity structure (the underlying excitation/inhibition dynamics). Specifically, models often produce degenerate solutions where intensity collapses to zero between events to maximize likelihood, rather than learning meaningful temporal dynamics.
Real-World Motivation: The study is motivated by terrorism data (specifically in Pakistan), where different groups (TTP, BRA, BLA, BLF) exhibit complex, non-stationary interactions that vary over space and time, which cannot be captured by aggregating data into simple counts or ignoring spatial coordinates.

2. Methodology: Multivariate Spatio-Temporal Neural Hawkes Process (MSTNHP)

The authors propose MSTNHP, an extension of the continuous-time Neural Hawkes Process (NHP) that integrates spatial information into the latent state evolution.

Core Architecture:
- Latent State Evolution: The model utilizes a continuous-time LSTM to maintain a hidden state $h(s, t)$ that evolves continuously between events. This state serves as a sufficient statistic for the history of events.
- Intensity Function: The conditional intensity for event type $k$ at location $s$ and time $t$ is defined as:
  $\lambda_k(s, t) = f_k(w_k^T h(s, t))$
  where $f_k$ is a softplus activation function ensuring positivity, and $w_k$ are trainable weights.
Spatio-Temporal Memory Decay:
- The key innovation lies in the memory cell update rule. Unlike standard NHP which only decays over time, the memory cell $c_d(s, t)$ in MSTNHP decays based on both temporal lag and spatial distance:
  $c_d(s, t) = \bar{c}_{d, i-1} + (c_{d, i-1} - \bar{c}_{d, i-1}) e^{-\delta^{(t)}_{d, i-1}(t - t_{i-1}) - \delta^{(s)}_{d, i-1}\|s - s_{i-1}\|}$
- Here, $\delta^{(t)}$ and $\delta^{(s)}$ are learned decay rates for time and space, respectively. This allows the model to learn how the influence of a past event fades as time passes and as the distance from the event location increases.
Training: The model is trained by maximizing the log-likelihood of observed sequences. The intractable double integral over space and time required for the likelihood is approximated using Monte Carlo sampling.

3. Key Contributions

Novel Framework: Introduction of the first flexible, deep learning-based framework that explicitly models multivariate spatio-temporal dependencies with learned, non-parametric triggering kernels.
Identification of the Intensity Gap: The paper demonstrates that optimizing log-likelihood alone is insufficient for Hawkes processes. It shows that many state-of-the-art temporal models (like SAHP, THP) fail to recover the true shape of conditional intensities, often collapsing to unrealistic solutions.
Superior Performance over Temporal-Only Models: The authors prove that ignoring spatial dimensions (collapsing spatio-temporal data into purely temporal data) leads to model misspecification. A temporal-only model (MTNHP) fails to capture the true dynamics of spatio-temporal processes, producing jagged, oscillatory intensities that do not reflect reality.
Real-World Application: Successful application to a complex, real-world dataset of terrorist attacks in Pakistan, demonstrating the model's ability to disentangle the distinct and interacting dynamics of four different terrorist groups.

4. Experimental Results

The paper validates the model through extensive simulations and a real-world case study.

Simulation Studies:
- Setup: Four bivariate simulation settings (Biv 1–4) were created with varying triggering structures (pure excitation, mixed excitation/inhibition, separable vs. non-separable kernels).
- Intensity Recovery: MSTNHP successfully recovered the ground-truth temporal and spatial intensity structures. In contrast, existing temporal-only neural models (RMTPP, SAHP, THP, etc.) failed to match the true intensity curves, often showing degenerate behavior.
- Spatial Accuracy: MSTNHP accurately reconstructed spatial intensity maps, capturing clustering and inhibition patterns that temporal-only models completely missed.
- Ablation Study: When spatial triggering was minimized (making the process nearly purely temporal), the temporal-only model performed well. However, when spatial heterogeneity was present, the temporal-only model failed, confirming that the spatial component is essential for correct modeling.
Real-World Application (Pakistan Terrorism Data):
- Data: 1,991 attacks by four groups (TTP, BRA, BLA, BLF) from 2008–2020.
- Findings:
  - MSTNHP captured complex, time-varying interactions. For instance, it identified periods where the intensity of smaller groups (BRA) temporarily exceeded the dominant group (TTP), reflecting local fluctuations.
  - It revealed "down peaks" (inhibitory effects) where intense waves of violence were followed by strategic pauses.
  - Spatial maps showed distinct interaction patterns: similar groups (BRA/BLA) showed correlated spatial intensity, while contrasting groups (TTP/BLF) showed reversed patterns, suggesting joint modulation.
- Comparison: The temporal-only model (MTNHP) produced distorted, low-magnitude intensity curves that failed to capture the group-specific dynamics observed in the data.

5. Significance and Conclusion

Methodological Shift: The paper argues that for spatio-temporal point processes, structural fidelity (recovering the true intensity function) is more important than, or at least as important as, predictive likelihood. It highlights that deep learning models must be designed to respect the continuous-time and continuous-space nature of the data to avoid degenerate solutions.
Practical Impact: The MSTNHP provides a powerful tool for analyzing complex event data in fields like epidemiology, seismology, and criminology, where understanding where and when events trigger other events is crucial for prediction and intervention.
Future Work: The authors note that the current memory decay is spatio-temporally separable and stationary. Future research aims to extend the model to non-separable and non-stationary structures to handle even more complex real-world dynamics.

In summary, this paper bridges the gap between deep learning and spatio-temporal statistics by proposing a model that not only predicts events but also faithfully reconstructs the underlying generative mechanisms of complex, interacting event streams.