📊 statistics

Semi-parametric bulk and tail regression using spline-based neural networks

This paper introduces SPQRx, a novel semi-parametric quantile regression framework that integrates spline-based neural networks with a blended generalised Pareto distribution to enable flexible, interpretable density modeling that satisfies extreme value theory guarantees for reliable tail extrapolation.

Original authors: Reetam Majumder, Jordan Richards

Published 2026-02-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Reetam Majumder, Jordan Richards

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather, but you are only interested in two things: the average days and the catastrophic storms.

Most standard weather models are like a pair of glasses that work perfectly for sunny days and light drizzles. But when a massive hurricane hits, those glasses fog up and break. They can't see the storm clearly because they were never designed to look at the "extreme" end of the spectrum.

This paper introduces a new, super-powered pair of glasses called SPQRx. It's a smart system that can predict both the boring, average days and the terrifying, record-breaking storms, all in one go.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Two-World" Dilemma

In statistics, there are two main ways to look at data:

The Bulk (The Average): This is the middle of the data. Most wildfires are small or medium-sized. Standard AI models (like neural networks) are great at learning these patterns. They are flexible and can adapt to any shape.
The Tail (The Extremes): This is the rare, dangerous stuff. The "1-in-100-year" fires. Standard AI models fail here. If they haven't seen a fire bigger than 5,000 acres in their training data, they can't guess what a 50,000-acre fire looks like. They just stop guessing.

On the other hand, old-school "Extreme Value" theories are great at predicting the tail, but they are rigid. They force the data into a specific, boring shape that doesn't fit the messy reality of the "average" days.

The Analogy: Imagine trying to draw a map of a country.

Method A (Standard AI): Draws the cities and roads perfectly, but when you get to the edge of the map (the ocean), it just stops drawing.
Method B (Old Extreme Theory): Draws the ocean perfectly with a straight line, but the cities in the middle look like distorted blobs.

2. The Solution: The "Blended" Approach

The authors created a new method called SPQRx (Semi-Parametric Quantile Regression for Extremes). They solved the problem by "blending" the two methods together, like mixing two types of paint to get the perfect color.

They invented a new mathematical shape called the Blended Generalized Pareto (bGP) distribution. Think of this as a Seamless Bridge:

The Left Side (The Bulk): Uses a flexible, shape-shifting AI (called SPQR) to perfectly model the average fires. It learns the complex, wiggly patterns of normal data.
The Right Side (The Tail): Uses a rigid, mathematically proven rule (from Extreme Value Theory) to model the massive fires. This ensures that even if a fire is bigger than anything ever seen before, the model knows how to extrapolate (guess) it correctly.
The Middle (The Blend): They use a smooth, invisible transition zone where the flexible AI gently hands over control to the rigid math rule. There is no sharp cut-off; it flows naturally from "average" to "extreme."

3. Why This Matters: The Wildfire Example

The authors tested this on U.S. Wildfire data from 1990 to 2020.

The Challenge: Wildfires are tricky. Most are small, but a few are massive and devastating. The data is "heavy-tailed," meaning the "fat tail" of the distribution is very thick with dangerous outliers.
The Result:
- Old Models: Could predict the size of a typical fire well, but when asked about the biggest fires (like the 2017 or 2020 record-breakers), they failed completely. They couldn't even give a probability because those fires were "out of bounds."
- SPQRx: Successfully predicted the size of the massive fires. It didn't just guess; it used the mathematical rules of the tail to say, "Based on the pattern of the smaller fires, a fire of this size is possible, and here is the probability."

4. The "X-Ray Vision" (Interpretability)

Usually, deep learning models are "black boxes." You put data in, and a number comes out, but you don't know why.

SPQRx comes with X-Ray Vision. It can tell you which factors matter for the average fire versus the massive fire.

The Finding: The model discovered that dryness (low humidity) is the most important factor for both small and huge fires.
The Twist: It also showed that while temperature matters, the lack of rain over the previous year is a massive driver for the extreme fires. This is a nuance that simpler models might miss.

Summary

Think of SPQRx as a Swiss Army Knife for Data:

It has a screwdriver for the everyday, average data (flexible and adaptable).
It has a hammer for the rare, extreme disasters (rigid and mathematically guaranteed).
And it has a smooth hinge that connects them so you never have to switch tools.

This allows scientists to not only understand the "normal" world but also to prepare for the "worst-case scenarios" with much higher confidence, which is crucial for things like insurance, disaster planning, and climate change adaptation.

1. Problem Statement

The paper addresses a critical gap in extreme quantile regression (EQR) and conditional density estimation.

Limitations of Standard SPQR: Semi-parametric Quantile Regression (SPQR) uses neural networks and spline-based representations to model conditional densities flexibly without parametric assumptions. While excellent for in-sample interpolation and modeling the "bulk" of data, it assumes a finite support. Consequently, it fails to extrapolate beyond observed data ranges and cannot satisfy the asymptotic guarantees required for heavy-tailed data (Extreme Value Theory, or EVT).
Limitations of Parametric EVT Models: Traditional EVT approaches (e.g., Generalized Pareto (GP) regression) rely on a high threshold $u(x)$ $u (x)$ to model tail excesses. This introduces two major issues:
1. Threshold Selection: Choosing an appropriate threshold is difficult and introduces uncertainty.
2. Bulk Modeling: These models typically ignore the "bulk" (body) of the distribution or model it parametrically, lacking the flexibility to capture complex, non-linear covariate effects in the central part of the data.
The Goal: Develop a unified, flexible framework that models the entire range of conditional data (both bulk and tail) without requiring threshold selection, while maintaining EVT compliance for reliable tail extrapolation.

2. Methodology: SPQRx and the Blended GP (bGP)

The authors propose SPQRx (Semi-parametric Quantile Regression for extremes), a framework that combines the flexibility of SPQR with the theoretical guarantees of EVT via a new distributional construct.

A. The Blended Generalized Pareto (bGP) Distribution

The core innovation is the bGP distribution, which seamlessly blends a "bulk" distribution with a GP tail.

Construction: It combines a constituent bulk distribution $F(y|W)$ (supported on $[0, 1]$ ) with a GP distribution $FGP(y|u, \sigma_u, \xi)$ for the upper tail.
Blending Mechanism: A weighting function $p(y)$ $p (y)$ , based on a Beta distribution CDF, transitions smoothly between the bulk and the tail over an interval $[a, b]$ $[a, b]$ .
- $a$ and $b$ are defined as specific quantiles ( $p_a$ and $p_b$ ) of the bulk distribution.
- The GP parameters (scale $\tilde{\sigma}_u$ and threshold $\tilde{u}$ ) are analytically derived to ensure continuity of the distribution function and its first derivative at the boundaries $a$ and $b$ .
Properties:
- For $y \leq a$ , the distribution is purely the bulk model.
- For $y \geq b$ , the distribution is exactly the GP distribution (ensuring EVT compliance).
- No intermediate threshold estimation is required; the transition is data-driven via the quantiles of the bulk model.

B. The SPQRx Framework

SPQRx integrates the bGP with deep learning:

Bulk Modeling: The bulk distribution $F(y|x)$ is modeled using the SPQR framework. This uses a Multi-Layer Perceptron (MLP) to predict weights for a convex combination of M-spline basis functions. This allows the bulk to be arbitrarily flexible and non-parametric.
Tail Modeling: The shape parameter $\xi(x)$ of the GP tail is modeled jointly with the bulk weights using the same MLP.
Architecture: A single neural network takes covariates $x$ $x$ as input and outputs:
- The vector of spline weights $W(x)$ for the bulk.
- The shape parameter $\xi(x)$ for the tail.
- A softmax activation ensures weights sum to 1; a specific activation (e.g., tanh or exp) constrains $\xi(x)$ .
Inference:
- Training: Optimized via Stochastic Gradient Descent (Adam) minimizing the negative log-likelihood of the bGP density.
- Pre-training: To improve stability, the model is first trained as a standard SPQR (ignoring the tail) and then fine-tuned as SPQRx.
- Extrapolation: For quantiles $\tau > p_b$ , the quantile function is derived analytically from the GP component, allowing extrapolation beyond the observed data range.

C. Interpretability

The authors utilize Accumulated Local Effects (ALEs) to assess variable importance. Crucially, SPQRx allows for separate importance scores for:

The bulk (via quantile levels $\tau < p_a$ ).
The tail (via the shape parameter $\xi(x)$ or high quantiles $\tau \to 1$ ).

3. Key Contributions

Blended GP (bGP) Distribution: A new univariate distribution that models the full data range with exact GP upper-tails, eliminating the need for threshold selection.
SPQRx Framework: A semi-parametric density regression method that combines deep neural networks with EVT. It offers the flexibility of non-parametric bulk modeling and the extrapolation capabilities of parametric tail modeling.
Variable Importance Decomposition: A method to quantify how covariates differentially affect the body versus the tail of a distribution, addressing the "black box" nature of deep learning in extreme value contexts.

4. Results

Simulation Studies

Setup: Data generated from a log-normal distribution with covariate-dependent parameters.
Performance: SPQRx consistently outperformed standard SPQR and deep GP regression models in terms of the Integrated Conditional 1-Wasserstein Distance (IWD) and tail-calibrated IWD (tIWD).
Extrapolation: Unlike SPQR (which fails for $y > 1$ ) and deep GP (which cannot model the bulk), SPQRx accurately captured both the bulk density and the heavy tails.
Variable Importance: The method correctly identified that specific covariates had no effect on the distribution, while others had equal effects across all quantiles.

Application: U.S. Wildfire Burnt Areas (1990–2020)

Data: Burnt areas of >10,000 wildfires, conditioned on meteorological and spatial covariates.
Findings:
- Tail Behavior: The data exhibited extreme heaviness ( $\xi > 0.5$ ). SPQRx successfully modeled the two most extreme fires in the test set (occurring in 2017 and 2020), which were outside the range of the training data. Standard SPQR failed to provide probability estimates for these events.
- Variable Importance:
  - Dryness (minimum relative humidity, $r_{min}$ ) was the dominant driver for both the bulk and the tail.
  - Precipitation ( $pr_{curr}, pr_{prev}$ ) increased in importance over time, overtaking maximum temperature.
  - Temporal Trends: The shape parameter $\xi(x)$ showed a positive temporal trend, suggesting wildfires are becoming more extreme (heavier-tailed) over time.
- Spatial Variability: The model revealed distinct spatial patterns in the median vs. the 0.999-quantile, confirming that covariates impact the bulk and tail differently.

5. Significance and Conclusion

Theoretical Advancement: SPQRx bridges the gap between flexible non-parametric density estimation and rigorous Extreme Value Theory. It solves the "threshold problem" by creating a smooth, continuous transition from the bulk to the tail.
Practical Utility: The framework is particularly valuable for risk assessment in fields like insurance, natural hazards, and finance, where accurate extrapolation into the tail is critical, but the bulk of the data is complex and non-linear.
Interpretability: By decoupling bulk and tail importance, SPQRx provides deeper insights into risk drivers, revealing that factors driving average events may differ from those driving catastrophic extremes.
Future Directions: The authors suggest extending the model to multivariate settings (using copulas) and investigating theoretical approximation bounds for the blended distribution.

In summary, SPQRx represents a significant step forward in deep extreme quantile regression, offering a robust, interpretable, and flexible tool for modeling the full spectrum of conditional distributions, from the central tendency to the most extreme outliers.