📊 statistics

neuralGAM: An R Package for Fitting Generalized Additive Neural Networks

The paper introduces **neuralGAM**, an R package that enhances the interpretability of deep learning by implementing Generalized Additive Neural Networks, which fit independent neural networks to estimate individual feature contributions while maintaining high accuracy and architectural flexibility.

Original authors: Ines Ortega-Fernandez, Marta Sestelo

Published 2026-08-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ines Ortega-Fernandez, Marta Sestelo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to predict the weather. You could build a giant, super-complex brain for the robot, stuffing it with millions of tiny connections. This "black box" brain might get the forecast right almost every time, but if you ask it, "Why do you think it will rain?" it might just stare back with a blank look. It knows what to do, but it can't explain how it figured it out. This is the famous "black-box problem" in artificial intelligence. On the other hand, you could build a simple, transparent machine where every single gear and lever is visible. This "white box" is easy to understand, but it might be too simple to handle the messy, complicated reality of a stormy sky. For years, scientists have been stuck trying to choose between a smart-but-mysterious brain and a clear-but-dumb machine. The big question is: Can we build a robot that is both a genius at predicting the future and a great teacher who can explain its thinking step-by-step?

This is exactly the puzzle tackled by a new tool called neuralGAM, introduced by researchers Ines Ortega-Fernandez and Marta Sestelo. They created a special R package (a toolbox for the R programming language) that builds a new kind of AI brain. Instead of one giant, tangled web of connections, their model builds a team of small, independent experts. Imagine a sports team where you have a separate specialist for every single player on the field: one expert studies the quarterback, another studies the wind, and a third studies the referee. Each expert learns their own specific job using a powerful neural network, but they all report their findings to a central coach who simply adds them all up. This "Generalized Additive Neural Network" (GANN) allows the AI to use the super-smart learning power of deep neural networks while keeping the model transparent enough to show you exactly how much each factor contributed to the final prediction.

The researchers tested this idea with two different scenarios. First, they created a fake, made-up world with perfect, known rules to see if their tool could find the hidden patterns. They found that neuralGAM was incredibly accurate, matching the performance of other complex methods, but with a huge bonus: it could draw clear, smooth pictures showing exactly how each variable changed the result. It even added "confidence bands," which are like fuzzy shadows around the lines, telling you how sure the model is about its answer. When they moved to a real-world challenge—predicting whether a flight would be delayed based on weather and flight data—the tool performed just as well as a "black box" neural network that couldn't explain itself. However, unlike the black box, neuralGAM could show you why a delay was likely. For instance, it revealed that very cold temperatures or high humidity increased delay risks in specific, smooth ways, and it could tell you when it was guessing because there wasn't enough data in that area.

The paper suggests that this approach offers a sweet spot between being a mysterious genius and a transparent teacher. While the "black box" models might squeeze out a tiny bit more accuracy in some cases, they lose the ability to show us the logic behind the decision. The neuralGAM tool, by contrast, proves that you don't have to sacrifice intelligence to get clarity. It shows that by breaking a complex problem into smaller, independent pieces and letting a neural network solve each piece, you can get a model that is both highly accurate and easy to understand. The researchers are careful to note that this is a new method that works well for specific types of data (like numbers, yes/no outcomes, and counts), and they are already dreaming up ways to make it even better by teaching it to handle more complex relationships between variables in the future. For anyone who wants to use AI but is tired of trusting a magic box they can't see inside, this package offers a fresh, clear, and powerful alternative.

Technical Summary: neuralGAM: An R Package for Fitting Generalized Additive Neural Networks

Problem Statement

Neural networks (NNs) are widely recognized for their superior predictive performance across diverse tasks, yet they suffer from the "black-box" problem, making it difficult to understand the decision-making process. While post-hoc interpretability methods exist, there is a growing demand for ante-hoc approaches that build inherently interpretable models without sacrificing predictive accuracy. Existing attempts to combine Generalized Additive Models (GAMs) with neural networks have faced limitations: early methods required manual tuning or were restricted to shallow architectures, while recent deep learning-based approaches (e.g., Neural Additive Models) can suffer from overfitting or increased computational costs. Furthermore, existing R packages that integrate additive models with neural networks (such as deepregression and bamlss) either focus on distributional regression with embedded spline bases or utilize Bayesian frameworks with high computational overhead, often lacking a dedicated implementation that leverages fully independent deep neural networks per feature within the R ecosystem.

Methodology

The paper introduces neuralGAM, an R package that implements a Generalized Additive Neural Network (GANN) topology. The core methodology trains an ensemble of independent neural networks—one for each feature—to estimate the contribution of that feature to the response variable. The overall model remains additive, adhering to the structure:

$E[Y | X] = m(X) = h^{-1}\left(\alpha + \sum_{j=1}^{p} f_j(X_j)\right)$

where $h(\cdot)$ is a link function and $f_j$ are smooth functions estimated by neural networks.

Estimation Algorithms

The package estimates the model parameters using a combination of two iterative algorithms:

Local Scoring (LS): Adapts the dependent variable ( $Z_i$ ) and weights ( $W_i$ ) based on the current fit and the specific distribution (Gaussian, Binomial, or Poisson).
Backfitting (BF): Iteratively updates the contribution of each covariate. In each step, a weighted additive model is fitted where each smooth term $f_j$ is estimated by training a feed-forward neural network on the partial residuals of that specific covariate.

This approach prioritizes interpretability by enforcing additive decomposition while retaining the flexibility of deep learning architectures. The framework supports semi-parametric models where some terms can be linear while others are non-parametric neural networks.

Uncertainty Quantification

To quantify epistemic uncertainty, neuralGAM employs Monte Carlo (MC) Dropout. Dropout layers are inserted into the neural networks and remain active during inference. By performing multiple stochastic forward passes ( $B$ ), the package generates a distribution of predictions. This allows for the calculation of confidence intervals for individual smooth terms and the additive predictor on both the link and response scales (using the delta method for the latter).

Software Architecture

The package relies on keras and TensorFlow for the underlying neural network implementation. Key features include:

Flexible Architecture: Users can define network depth, units, activation functions, and regularizers globally or on a per-term basis.
Custom Loss Functions: Supports standard Keras losses and user-defined custom loss functions.
Validation: Includes support for validation splits and cross-validation workflows.
Visualization: Provides diagnostic plots (residuals, QQ-plots) and partial effect plots with confidence intervals using both base R and ggplot2.

Key Contributions

Implementation: neuralGAM is presented as the only available R package to implement a GANN using fully independent deep neural networks for each feature, estimated via local scoring and backfitting.
Interpretability vs. Flexibility: It offers a practical balance, delivering competitive predictive performance while maintaining the "white-box" nature of GAMs, allowing for clear feature-level interpretation.
Uncertainty Estimation: It integrates MC Dropout directly into the training and inference pipeline to provide per-term and global epistemic uncertainty estimates, a feature not natively available in comparable R packages like deepregression without computationally expensive model ensembles.
Usability: The package provides a user-friendly interface similar to standard R regression packages, with helper functions for environment setup (install_neuralGAM) and comprehensive diagnostic tools.

Results

The paper evaluates neuralGAM through simulated data and a real-world application (flight delay prediction), comparing it against deepregression and a black-box Multi-Layer Perceptron (MLP) implemented in keras.

Simulated Data: In a Gaussian response scenario, neuralGAM achieved slightly better Mean Squared Error (MSE) and deviance explained compared to deepregression. Crucially, neuralGAM recovered the true functional forms (quadratic, linear, sinusoidal) with smoother curves and narrower, more stable confidence intervals. In contrast, deepregression exhibited more irregular, "wiggly" estimates, particularly at boundaries, and required model ensembles to approximate uncertainty.
Real Data (Flight Delays): In predicting flight delays (binary response), neuralGAM achieved an AUC-ROC of 0.819, comparable to deepregression (0.838) and the black-box MLP (0.838). While the black-box model offered slightly higher predictive accuracy, it lacked interpretability. neuralGAM successfully visualized partial effects (e.g., the non-monotonic relationship between air time and delay) with stable uncertainty bands, whereas deepregression produced more oscillatory effects that suggested overfitting to local noise.

Significance and Claims

The authors position neuralGAM as a valuable tool for researchers seeking interpretable neural network implementations within the R ecosystem. The paper claims that neuralGAM offers a strong compromise between the interpretability of additive models and the flexibility of deep learning.

The significance of the work lies in its ability to:

Provide a "white-box" alternative to black-box deep learning that does not require manual tuning of univariate perceptrors.
Offer native uncertainty quantification via MC Dropout, avoiding the computational cost of training multiple independent models (ensembles) required by other methods.
Deliver stable, smooth effect estimates that are easier to interpret than the spline-based estimates found in deepregression, particularly in regions with sparse data.

The authors modestly note that while neuralGAM is currently limited to Gaussian, Binomial, and Poisson distributions, it represents a significant step toward accessible, interpretable deep learning in R. Future work is identified as potentially extending the model to multinomial logistic regression, incorporating interaction terms, and exploring additional uncertainty estimation methods like conformal prediction.