📊 statistics

exdqlm: An R Package for Estimation and Analysis of Flexible Dynamic Quantile Linear Models

This paper introduces the R package `exdqlm`, which facilitates Bayesian estimation and analysis of flexible dynamic state-space quantile models using the extended asymmetric Laplace distribution, offering both MCMC and efficient variational inference methods alongside tools for static regression, forecasting, and model diagnostics.

Original authors: Antonio De Leon, Raquel Barata, Raquel Prado, Bruno Sansó

Published 2026-07-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Antonio De Leon, Raquel Barata, Raquel Prado, Bruno Sansó

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Weather Forecast for the Whole Sky, Not Just the Average

Imagine you are trying to predict the weather. Most traditional forecasting tools are obsessed with the "average" day. They tell you the average temperature, the average rainfall, and the average wind speed. But averages can be tricky. If the average temperature is a comfortable 70°F, it might hide the fact that some days are freezing and others are scorching. In the world of statistics, this is called "mean-based modeling," and it often misses the extremes—the heatwaves, the floods, and the droughts—that matter most when you're planning a picnic or building a dam.

To fix this, statisticians use a technique called quantile regression. Instead of asking, "What is the average?" they ask, "What will the temperature be on the hottest 5% of days?" or "What is the water level on the driest 1% of days?" This is like looking at the entire weather map instead of just the thermometer in the middle of the room. However, when data changes over time—like stock prices, river levels, or sunspot cycles—doing this gets incredibly complicated. The old tools for these "dynamic" situations were either too rigid (assuming the world is perfectly predictable) or too slow to run on a computer.

The New Toolkit: exdqlm

Enter exdqlm, a new software package for the R programming language, created by a team of researchers at the University of California Santa Cruz. Think of exdqlm as a high-tech, Swiss Army knife for statisticians who need to predict the "edges" of data over time. It doesn't just guess the average; it builds a flexible, moving model that can track the 95th percentile (the extreme highs), the 5th percentile (the extreme lows), and everything in between, as they evolve day by day.

The core magic of this package lies in a new mathematical shape it uses to describe uncertainty, called the extended asymmetric Laplace (exAL) distribution. To understand why this matters, imagine trying to draw a curve that fits a pile of sand. The old tools used a very specific, rigid shape (the standard Asymmetric Laplace) that worked well for simple piles but struggled when the sand was lopsided or had weird bumps. The exAL distribution is like a shape-shifting mold; it can stretch, squeeze, and twist to fit the weird, lopsided reality of real-world data much better than the old rigid molds, while still being fast enough for computers to handle.

The paper introduces this software and demonstrates how it works through several real-world stories. First, the team looked at the Lake Huron water levels. By fitting separate models for the high, medium, and low water levels, they could see how the lake's behavior changed over decades. They then combined these separate predictions into a single, coherent picture of the lake's future, showing how the software can stitch together different "slices" of the data to tell a complete story.

Next, they tested the software on sunspots, the dark spots that dance across the surface of the sun in an 11-year cycle. Here, they compared two versions of their tool: one that used the old, rigid shape (DQLM) and one that used the new, flexible shape (exDQLM). The results suggested that the flexible version was better at capturing the weird, skewed behavior of the sun's upper limits. They also showed how the software could automatically tune itself to find the best settings for these predictions, acting like a self-driving car that adjusts its sensitivity to the road.

In a third example, the researchers looked at river water flow in California. They wanted to see if they could predict low water levels by looking at climate signals like ocean temperatures. They tested three different ways of connecting these signals: ignoring them, adding them directly, and using a "transfer function" that accounts for how the effect of the ocean might ripple through time. The software helped them visualize exactly how these different methods changed the prediction, showing that the "transfer function" approach could capture the delayed effects of climate patterns better than a simple direct link.

Finally, the team ran a simulation where they knew the "true" answer beforehand. They created fake data with a hidden pattern and asked the software to find it. They found that the software's fast, approximate method (called LDVB) was incredibly quick—up to 20 times faster than the traditional, slow method (MCMC)—and still found the right answer most of the time. This suggests that for many problems, you don't need to wait hours for a result; you can get a very good answer in minutes.

The paper doesn't claim to have solved every problem in statistics. It explicitly notes that the software currently handles one "quantile" (one specific slice of the data) at a time, rather than modeling all slices simultaneously in a single giant model. It also admits that while the fast method is great for speed, the slower, traditional method is still the gold standard if you need to be absolutely certain about the very rare, extreme tails of the data.

However, the main takeaway is clear: exdqlm fills a crucial gap. It bridges the divide between tools that are good at handling time-based data but only look at averages, and tools that look at extremes but are too slow or rigid to handle time. By offering a flexible, fast, and user-friendly way to model the "edges" of changing data, it gives scientists and analysts a powerful new way to prepare for the unexpected, whether it's a flood, a market crash, or a solar storm.

Technical Summary of exdqlm: An R Package for Estimation and Analysis of Flexible Dynamic Quantile Linear Models

Problem Statement
While quantile regression is a standard tool for static modeling, dynamic quantile methods and their accompanying software remain limited. Existing Bayesian approaches often rely on the asymmetric Laplace (AL) distribution as a likelihood, which links to the check-loss function but imposes restrictive assumptions on error distributions. Conversely, more flexible nonparametric error models are often computationally prohibitive for dynamic state-space models and long time series. Furthermore, general quantile regression packages (e.g., quantreg, bayesQR) typically treat time as an observed predictor or lag rather than propagating coefficients as latent states, while general state-space packages (e.g., dlm) focus on Gaussian likelihoods and lack dedicated quantile workflows. There is a gap between broad quantile tools and Gaussian dynamic modeling software that requires a dedicated Bayesian workflow for dynamic quantile state-space models.

Methodology
The paper introduces the exdqlm R package, which implements Extended Dynamic Quantile Linear Models (exDQLMs). The core methodological components are:

Extended Asymmetric Laplace (exAL) Likelihood: The package utilizes the exAL distribution (Yan, Zheng, and Kottas, 2025), a parametric extension of the standard AL distribution. The exAL preserves computational advantages via a mixture representation involving latent variables ( $v_t$ and $s_t$ ) while relaxing restrictive assumptions through a skewness parameter $\gamma$ . The observation equation is defined as $y_t = F_t^\top \theta_t + \epsilon_t$ , where $\epsilon_t \sim \text{exAL}_{p_0}(0, \sigma, \gamma)$ , and the state evolution follows $\theta_t = G_t \theta_{t-1} + \omega_t$ with Gaussian innovations.
Inference Algorithms: The package provides two primary inference engines:
- MCMC: A Gibbs sampling routine (exdqlmMCMC) that updates latent variable sequences, dynamic states (via forward-filtering backward-sampling), and scale/skewness parameters. It supports slice sampling and Metropolis–Hastings kernels.
- Laplace–Delta Variational Bayes (LDVB): A fast approximate inference routine (exdqlmLDVB) based on the nonconjugate variational strategy of Wang and Blei (2013). It employs mean-field factorization and blockwise coordinate ascent. The non-conjugate joint scale–skewness block is approximated using a Laplace–delta method on transformed coordinates, providing deterministic convergence monitoring and approximate uncertainty summaries.
Model Extensions:
- Transfer Functions: The framework supports transfer-function state augmentation to model current and lagged input effects on a response quantile via an exponential decay rate $\lambda$ .
- Static Regression: The package handles static exAL and AL regression with shrinkage priors (specifically the Nishimura–Suchard regularized horseshoe prior) as special cases.
- Posterior Synthesis: A quantileSynthesis() function combines posterior predictive draws from separately fitted quantile levels into a single coherent distribution, applying monotonicity corrections if necessary.

Key Contributions
The paper's primary contribution is software-centric, delivering an integrated workflow for specifying, fitting, diagnosing, and forecasting Bayesian dynamic and static quantile analyses. Specific contributions include:

A composable model-construction interface for trend, seasonal, regression, and transfer-function components.
Coordinated inference interfaces for both MCMC and variational inference (LDVB and ISVB).
An S3 object workflow with standard methods (print(), summary(), plot(), predict(), diagnostics()) for fitted models, forecasts, and diagnostic objects.
Compiled implementations (via Rcpp) for computationally intensive operations (e.g., exAL density, Kalman recursions, posterior sampling) with R fallbacks for portability.
Diagnostic tools including Probability Integral Transform (PIT) analysis, Kullback–Leibler (KL) divergence for calibration, Continuous Ranked Probability Score (CRPS), and Posterior Predictive Loss Criterion (PPLC).

Results
The paper validates the package through four examples:

Lake Huron: Demonstrates dynamic model construction, MCMC inference, multi-step forecasting, and post-hoc synthesis of posterior predictive draws across three quantile levels (0.05, 0.50, 0.95).
Sunspots: Compares DQLM (fixed $\gamma=0$ ) and exDQLM (estimated $\gamma$ ) fits using LDVB. It illustrates model diagnostics (PIT, KL, CRPS) for selecting discount factors and compares the computational efficiency of LDVB against MCMC, showing LDVB is substantially faster while maintaining competitive predictive performance.
Big Tree Water Flow: Compares three specifications: a baseline without covariates, a direct regression model, and a transfer-function model. The example highlights the workflow for handling external covariates and conditional forecasting, showing how transfer functions capture lagged climate effects.
Simulated Sparse Benchmark: Evaluates static exAL regression with shrinkage priors. Results indicate that while MCMC provides more accurate posterior tail summaries, LDVB offers a rapid approximation (7–20x faster) with comparable signal recovery for sparse coefficients.

Significance and Scope
The paper positions exdqlm as a specialized tool filling the gap between general quantile regression software and Gaussian state-space packages. It explicitly states that it is not intended to replace broad general-purpose toolkits or packages designed for joint multi-quantile estimation. Instead, it provides a dedicated Bayesian workflow for dynamic quantile state-space models where coefficients are propagated as latent states.

The authors note current limitations: dynamic models are fitted one quantile level at a time (with synthesis occurring post-hoc), LDVB provides approximate rather than exact posterior draws (making MCMC the reference for tail behavior), and transfer-function lag rates are fixed or grid-selected rather than internally estimated. Future development directions mentioned include multivariate-output models, improved approximations for the nonconjugate scale–skewness block, and nonlinear dynamic modeling. The package is distributed under the MIT License and is available on CRAN.

The Weather Forecast for the Whole Sky, Not Just the Average

The New Toolkit: exdqlm

More like this