📊 statistics

HAL-MLE Log-Splines Density Estimation (Part I: Univariate)

This paper establishes a unified framework connecting the Highly Adaptive Lasso maximum likelihood estimator (HAL-MLE) with classical total variation-penalized density estimation methods in the univariate setting, while proving new theoretical results including asymptotic linearity, pointwise asymptotic normality, and uniform convergence rates for smoothness orders $k \geq 1$ .

Original authors: Yilong Hou, Zhengpu Zhao, Yi Li, Mark van der Laan

Published 2026-02-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yilong Hou, Zhengpu Zhao, Yi Li, Mark van der Laan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to reconstruct a hidden landscape based only on a few scattered footprints left by a traveler. Your goal is to draw a map (a density function) that shows exactly where the traveler was most likely to be and where they rarely went.

This paper introduces a new, super-smart detective tool called HAL-MLE (Highly Adaptive Lasso Maximum Likelihood Estimator) to solve this map-making problem. Here is how it works, explained without the heavy math jargon.

1. The Problem: The "Rigid" vs. The "Chaotic"

Traditional map-makers (statisticians) usually use one of two tools:

The Smooth Painter (Kernel Density Estimation): This tool tries to paint a smooth, flowing landscape. It's great for gentle hills, but if the terrain has a sudden cliff or a jagged rock, the painter smoothes it over, losing important details. It's like trying to draw a mountain range with a thick marker; you miss the sharp peaks.
The Rigid Builder (Standard Splines): This tool builds the map using straight lines or simple curves. It's better at handling sharp turns, but it often gets "jittery" or oscillates wildly, drawing fake hills and valleys where there are none.

The Challenge: Real-world data (like galaxy speeds or stock prices) is messy. It has smooth areas and sudden, sharp jumps. We need a tool that can be smooth when it needs to be and sharp when it needs to be, without getting jittery.

2. The Solution: The "Adaptive Lasso" (HAL)

The authors propose HAL-MLE, which is like a master sculptor with a magical chisel.

The "Lasso" (The Rope): Imagine you have a rope that limits how much "wiggle room" your map can have. This is the Total Variation (TV) penalty. It prevents the map from getting too crazy or jittery. If the map tries to wiggle too much, the rope pulls it back.
The "Highly Adaptive" Part: Unlike old tools that use a fixed grid of points (like a grid of squares on graph paper), HAL-MLE is data-adaptive. It looks at your footprints and says, "Hey, there's a huge jump in activity right here, so I'll put a knot (a pivot point) exactly there." It builds the map using a flexible set of building blocks that only appear where the data demands them.

3. The Secret Sauce: The "Log-Spline" Link

To make sure the map always makes sense (probabilities can't be negative!), the authors use a special trick called a log-spline link.

Think of this as a translator. The HAL-MLE doesn't draw the map directly. Instead, it draws a "log-map" (which can be any shape, positive or negative) and then runs it through a translator that turns it into a perfect, positive probability map. This ensures the math stays stable and the result is always a valid probability distribution.

4. Why This Paper Matters (The "Aha!" Moment)

The authors prove three big things:

It's Flexible: In the simple one-dimensional case (like a single line of data), this new method is mathematically equivalent to the best existing "TV-penalized" methods. It's not just a new toy; it's a new way of seeing the same powerful tools.
It's Fast and Accurate: They proved that as you get more data (more footprints), this map converges to the truth at a very fast speed. It's not just "getting closer"; it's getting closer quickly, even for complex, bumpy landscapes.
It Gives You Confidence: The paper provides a way to draw confidence intervals (a "fuzzy band" around the map) showing where the true landscape likely lies. This is crucial for scientists who need to know not just what the map looks like, but how sure they can be about it.

5. The "Targeting" Trick (HAL-TMLE)

Sometimes, you don't just want the whole map; you want to know a specific number, like the average speed of the galaxies or the median value.

The authors show that you can take your HAL-MLE map and give it a tiny, precise "nudge" (a Targeting Step).
Imagine you have a rough sketch of a face. You know the average eye position is wrong. You don't redraw the whole face; you just nudge the eyes to the right spot.
This HAL-TMLE method ensures that even if your map isn't perfect everywhere, your specific calculation (like the average) is statistically perfect (asymptotically efficient).

6. Real-World Test: The Galaxy Case Study

To prove it works, they tested it on Galaxy Velocity Data.

The Data: Astronomers measure how fast galaxies are moving. If they are clustered in groups, the speed data will have "humps" (modes) corresponding to those clusters.
The Result: HAL-MLE successfully identified the hidden clusters (the humps) and the gaps between them, drawing a much clearer picture than older methods. It handled the jagged, multi-peaked nature of the data perfectly.

Summary Analogy

If traditional methods are like using a cookie cutter (fixed shapes) or a blurry camera (smoothing everything out), HAL-MLE is like a 3D printer that listens to the data. It prints a model that is smooth where the data is smooth, sharp where the data is sharp, and it knows exactly how much "ink" (complexity) to use so it doesn't waste energy or create fake details.

The paper essentially says: "We have built a universal, mathematically proven, and computationally efficient tool that can map any complex, bumpy, or jagged distribution, and we can tell you exactly how confident you should be in that map."

1. Problem Statement

The paper addresses the problem of nonparametric maximum likelihood estimation (MLE) of a probability density function $p_0$ on a compact support $[a, b]$ . The goal is to estimate the density without assuming a specific parametric form, while handling complex features such as rapid variations, oscillations, and discontinuities.

The authors focus on the univariate setting as a foundational step to connect classical Total Variation (TV) penalized methods with the modern Highly Adaptive Lasso (HAL) framework. Key challenges in existing methods include:

Kernel Density Estimation (KDE): Suffers from the curse of dimensionality and poor performance in regions with rapid density changes.
Log-spline methods: Often rely on uniform knots and may struggle with oscillation issues or generalizing to multivariate settings without strong smoothness assumptions.
TV-Penalized methods (e.g., Trend Filtering, Local Adaptive Splines): While effective, they often lack a unified theoretical framework connecting them to the HAL estimator, particularly regarding asymptotic normality and efficiency for general statistical functionals.

2. Methodology: HAL-MLE with Log-Splines

The authors propose the HAL-MLE (Highly Adaptive Lasso Maximum Likelihood Estimator) using a log-spline link function.

A. Theoretical Foundation: HAL Assumptions

The method relies on the assumption that the log-density $f = \log p$ belongs to a class of càdlàg functions (right-continuous with left limits) with a Bounded Sectional Variation Norm (BSVN).

Univariate Equivalence: In the univariate case, the BSVN assumption is shown to be equivalent to the classical Bounded Total Variation (BTV) assumption used in Local Adaptive Splines (LAS) and Trend Filtering (TF).
Representation: Any function $f$ in this class can be represented exactly as an infinite linear combination of truncated power basis functions (splines). The HAL estimator approximates this using a finite-dimensional working model selected via Lasso.

B. The Estimator Construction

Link Function: The density is modeled as $p_f(x) = \frac{\exp(f(x))}{\int \exp(f(u))d\mu(u)}$ , where $f$ is a linear combination of basis functions. This ensures positivity and convexity of the optimization problem.
Basis System: The model uses a truncated power basis (including global polynomial terms and local truncated power functions at knot points).
Data-Adaptive Knots: Unlike traditional methods using uniform grids, HAL initializes knots at the observed data points ( $x_1, \dots, x_n$ ).
Optimization: The estimator $\hat{f}_n$ is obtained by minimizing the negative log-likelihood subject to an $L_1$ norm constraint on the basis coefficients:
$\hat{\beta}_n = \arg\min_{\|\beta\|_1 \leq M} \sum_{i=1}^n \left( -\log p_{f_\beta}(x_i) \right)$
The hyperparameter $M$ (controlling the complexity/variation) is selected via Cross-Validation (CV).
Undersmoothing: To achieve asymptotic efficiency for functionals (like means or medians), the authors employ an "undersmoothed" version of the estimator (selecting a larger $M$ than optimal for density estimation alone) to reduce bias.

C. Computational Algorithms

The paper develops specialized optimization algorithms tailored to the structure of the HAL-MLE problem, as generic solvers (like CVXPY) struggle with knot selection and scalability:

Proximal Newton: Combines Newton's method with proximal operators for fast convergence.
L-BFGS: Used for large-scale problems to avoid $O(n^2)$ memory usage associated with full Hessian matrices.
AdaGrad: A variant using diagonal Hessian approximations for computational efficiency.
Results: These specialized solvers effectively select the necessary knot points (reducing the active set to $O(n^{1/(2k+3)})$ ) and converge faster than generic solvers.

3. Key Contributions

The paper establishes three major theoretical results for the univariate HAL-MLE:

Asymptotic Linearity: The HAL-MLE is shown to be asymptotically linear, meaning it can be represented as an empirical mean of an influence function plus a negligible remainder term.
Pointwise Asymptotic Normality: For any fixed $x$ $x$ , the estimator $\hat{f}_n(x)$ $\hat{f}_{n} (x)$ (and consequently the density $\hat{p}_n(x)$ $\overset{p}{^}_{n} (x)$ ) follows a normal distribution asymptotically.
- The paper provides a Delta-method based variance estimator for the density, allowing for the construction of pointwise confidence intervals.
Uniform Convergence Rates: The estimator achieves a uniform convergence rate of:
$\|\hat{p}_n - p_0\|_\infty = O_P\left( n^{-(k+1)/(2k+3)} (\log n)^p \right)$
for smoothness order $k \geq 1$ . This improves upon previous HAL results which only guaranteed consistency without rates for $k=0$ .

Extension to Statistical Functionals:
The paper demonstrates that the HAL-MLE (and its Targeted Maximum Likelihood Estimation, HAL-TMLE, variant) yields asymptotically efficient estimators for pathwise differentiable statistical functionals (e.g., mean, median, survival probability, moments).

Plug-in HAL-MLE: Achieves efficiency if the estimator is sufficiently undersmoothed.
HAL-TMLE: A single targeting step is sufficient to solve the efficient influence curve equation, ensuring efficiency even without aggressive undersmoothing, while preserving the density estimate's structural properties.

4. Results

The authors validate their theory through extensive simulations and a real-world case study:

Simulation Study:
- DGPs: Tested on six diverse data generating processes (Truncated Normal, GMM with multiple modes/spikes, Step functions, Sinusoidal).
- Convergence: Empirical results confirm the theoretical convergence rates and pointwise asymptotic normality. The coverage of 95% confidence intervals constructed via the Delta method is close to the nominal level.
- Efficiency: The plug-in HAL-MLE and HAL-TMLE achieve Mean Squared Error (MSE) comparable to the optimal parametric estimators (e.g., sample mean for the population mean) across various DGPs.
- Comparison: HAL-MLE outperforms KDE and standard Log-splines in handling highly oscillatory and discontinuous densities. It performs comparably to Trend Filtering (TF) but with a unified theoretical framework and data-adaptive knot selection.
Case Study (Galaxy Velocities):
- Applied to the classic "Galaxies" dataset (velocities of galaxies in the Corona Borealis region).
- The HAL-MLE successfully captured the multimodal structure of the data (superclusters) and provided valid confidence intervals via the Delta method, which agreed well with nonparametric bootstrap results but were computationally cheaper.

5. Significance

Unification: The paper bridges the gap between classical TV-penalized methods (LAS, TF) and the modern HAL framework, proving they are theoretically equivalent in the univariate case under bounded variation assumptions.
Theoretical Advancement: It provides the first rigorous proof of uniform convergence rates and pointwise asymptotic normality for HAL-based density estimators with $k \geq 1$ , extending previous work that was limited to consistency.
Practical Utility: By providing a unified framework for density estimation and efficient inference for functionals (via HAL-TMLE), the method offers a powerful tool for nonparametric statistics.
Software: The authors release a Python package (HALDensity) implementing these specialized optimization algorithms, making the method accessible for practical applications.

Future Work: The authors note that this is Part I, focusing on the univariate case. Part II will extend these results to the multivariate setting, addressing the complexities of sectional variation norms in higher dimensions.