📊 statistics

Interpretable Network-assisted Random Forest+

This paper introduces a family of interpretable network-assisted models built upon a generalized random forest framework that bridges the gap between high prediction accuracy and model transparency by leveraging network dependencies while providing comprehensive tools to quantify feature and network contributions.

Original authors: Tiffany M. Tang, Elizaveta Levina, Ji Zhu

Published 2026-09-09

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Tiffany M. Tang, Elizaveta Levina, Ji Zhu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of modern data science, computers are often taught to learn from examples, much like a student studying for an exam. For decades, the standard assumption has been that every example is independent, a solitary fact standing alone. But in the real world, things are rarely isolated. People are connected by friendships, cities by roads, and genes by biological pathways. These connections create a web of influence where one person's behavior or a city's condition can ripple out to affect their neighbors. When data points are linked in this way, treating them as independent creates a blind spot, missing the subtle but powerful forces that shape outcomes. The challenge for scientists is to build models that can use these connections to make better predictions without turning the model into a black box that no one can understand. If a model predicts a student's future or a city's crime rate based on their social circle, we need to know exactly how much that circle matters compared to their personal history.

This is the problem tackled by a new method called NeRF+, developed by researchers at the University of Notre Dame and the University of Michigan. They sought to create a tool that could harness the power of network connections to improve predictions while remaining transparent enough for humans to trust. Traditional approaches often force a choice: use complex, highly accurate systems that are impossible to interpret, or use simple, understandable systems that fail to capture the nuance of connected data. The researchers bridged this gap by building a flexible model that combines the predictive strength of decision trees with the clarity of linear equations. Their approach allows the computer to learn from the data's structure, identifying not just which personal traits matter, but how much the surrounding network influences the result.

The core of their work involves teaching the computer to recognize two distinct ways a network can shape reality. Sometimes, people who are close in a network are similar because they share a hidden, underlying quality, like belonging to the same social group. Other times, they are similar simply because they are directly connected and influence each other, like neighbors who adopt similar habits. The new method can detect both of these patterns. It does this by looking at the data in two ways simultaneously. First, it maps the network to find hidden groupings, much like sorting people into circles based on who they know. Second, it applies a smoothing rule that encourages the model to treat connected neighbors as similar, capturing the direct flow of influence between them. By combining these with the individual details of each person or place, the model creates a prediction that is both sharp and adaptable.

To ensure this tool is useful for real-world decision-making, the researchers equipped it with a built-in set of lenses to see how it works. Unlike other advanced systems that require separate, often unreliable tools to explain their choices, this model explains itself. It can tell a researcher exactly how much a specific feature, like a student's grade or a city's temperature, contributed to a prediction. More importantly, it can separate the influence of the network from the influence of personal traits. It can quantify how much of a prediction came from the fact that a person is connected to others, and how much came from their own unique characteristics. This ability to disentangle the web from the individual is crucial for understanding the true drivers of outcomes in complex systems.

The researchers tested their method using simulated data where they knew the exact rules of the game. They created scenarios where the network played a small role, a large role, or no role at all, and where the relationships between variables were simple or wildly complex. In every case, the new method performed as well as or better than existing techniques. When the network was a strong factor, the method used it to boost accuracy significantly. When the network was irrelevant, the method simply ignored it without losing its ability to predict. Crucially, it maintained its high performance even when the relationships in the data were jagged and unpredictable, a situation where simpler models often fail. The simulations also showed that the method could correctly identify which parts of the network were driving the results, distinguishing between the hidden groupings and the direct connections.

To prove the method worked in the messy reality of the real world, the team applied it to two distinct case studies. The first involved a large experiment on school conflicts, where researchers tried to predict how friendly a student would perceive their school to be by the end of the year. The data included the students' backgrounds and a map of their friendships. The new model successfully predicted these perceptions, and its internal analysis revealed something striking: for some schools, the overall social structure was the main driver, while for others, it was the tight-knit groups of friends that mattered most. In one specific school, the model identified a small, isolated group of eighth-graders whose friendships were strongly linked to a negative view of the school. Without the model's ability to zoom in on individual connections, this specific group might have been lost in the average noise of the data.

The second case study looked at crime rates across Philadelphia over fifteen years. Here, the data consisted of monthly crime statistics for hundreds of neighborhoods, linked by their physical adjacency. The goal was to predict crime rates using weather data and time. The results were clear: the network of neighboring neighborhoods was the single most important factor, far outweighing the weather or the time of year. The model's internal tools showed that the influence came almost entirely from the direct connection between neighbors, confirming that crime rates tend to smooth out across borders. The model could even pinpoint which specific neighborhoods were driving these patterns, showing a clear, smooth gradient of influence that matched the observed crime maps. In both the schools and the cities, the method proved that it could learn from the connections without getting confused by them.

The researchers emphasize that this tool is designed for situations where understanding the "why" is just as important as the prediction itself. Whether it is a doctor trying to understand a diagnosis, a policy maker evaluating an intervention, or a scientist studying social dynamics, the ability to see the contribution of the network is vital. The method does not claim to solve the mystery of human behavior or to prove that one thing causes another, but it provides a clear, honest map of how the data is being used. It offers a way to navigate the complexity of connected data with confidence, ensuring that the insights gained are not just accurate, but also transparent and trustworthy. By making the invisible web of influence visible, this work opens the door to more informed decisions in a world where everything is connected.

Technical Summary: Interpretable Network-assisted Random Forest+ (NeRF+)

Problem Statement
Machine learning algorithms typically assume training samples are independent. However, in many modern applications (e.g., social networks, spatial data), data points are connected via a network, inducing dependencies that reduce effective sample size and introduce bias. While leveraging these dependencies (network cohesion) can improve prediction, existing methods face a trade-off:

Graph Neural Networks (GNNs): Offer high prediction accuracy but are often "black boxes," lacking intrinsic interpretability. Post-hoc explainability tools for GNNs are often unstable and susceptible to adversarial attacks.
Network-assisted Linear Regression (e.g., RNC): Highly interpretable but often lacks the flexibility to capture complex nonlinear relationships, leading to suboptimal prediction performance.
Network-assisted Tree Methods (e.g., Network BART, BAMDT): Improve prediction but lack robust interpretability tools. Existing importance measures for these models often suffer from bias toward network features due to the large number of potential network splits, making it difficult to disentangle network importance from feature importance.

There is a need for a method that balances high prediction accuracy with intrinsic interpretability, specifically capable of disentangling the contribution of network connections from that of node features.

Methodology: NeRF+
The authors propose Network-assisted Random Forest Plus (NeRF+), a flexible family of models built upon Random Forest Plus (RF+). RF+ generalizes standard Random Forests by representing decision tree predictions through a linear model, combining nonlinear tree-based features with linear features.

NeRF+ incorporates network information through two complementary mechanisms:

Network Embedding Features ( $Z$ ): The original covariate matrix $X$ is augmented with network embedding features $Z$ (e.g., Laplacian spectral embeddings) learned solely from the network structure. This captures cohesion in the latent space.
Network Cohesion Penalty ( $\alpha$ ): The model includes a node-specific effect $\alpha$ regularized by a network cohesion penalty ( $\alpha^\top L \alpha$ ), where $L$ is the network Laplacian. This encourages predictions for connected nodes to be similar, capturing smoothness in the observed network.

Model Formulation:
For a forest of $T$ trees, NeRF+ predicts the response $\hat{y}$ as an ensemble average:
$\hat{y} = \frac{1}{T} \sum_{t=1}^T \left( \hat{\alpha}^{(t)} + \tilde{X}\hat{\beta}^{(t)} + \Psi_t(\tilde{X})\hat{\gamma}^{(t)} \right)$
Where:

$\tilde{X} = [X, Z]$ is the augmented feature matrix.
$\hat{\alpha}^{(t)}$ represents the node effects (network cohesion).
$\hat{\beta}^{(t)}$ are coefficients for the linear features.
$\Psi_t(\tilde{X})$ represents decision stump features derived from the tree splits.
$\hat{\gamma}^{(t)}$ are coefficients for the tree-based nonlinear features.

The parameters are estimated by minimizing a loss function (e.g., squared error) plus regularization penalties (e.g., $\ell_2$ on $\beta$ and $\gamma$ , and the cohesion penalty on $\alpha$ ).

Key Contributions: Interpretability Tools
A primary contribution of the paper is a suite of intrinsic interpretability tools derived directly from the model parameters, avoiding the need for unstable post-hoc approximations. These tools allow researchers to:

Global Feature Importance:
- Permutation Importance: Measures the drop in prediction accuracy when feature values (or network features) are permuted. Crucially, this allows for the separate quantification of the importance of network cohesion ( $\alpha$ ) versus network embeddings ( $Z$ ).
- MDI+ Importance: A generalization of Mean Decrease in Impurity adapted for RF+. It measures the similarity between the response and predictions made using only a subset of features (imputing others with column means), allowing for the disentanglement of network vs. feature contributions.
Local Feature Importance:
- Leveraging the linear form of NeRF+, the authors define local importance scores (similar to Shapley values) for individual samples. This quantifies how much a specific feature or the network contributes to a specific sample's prediction relative to the average.
Sample Influence:
- The authors derive closed-form approximate Leave-One-Out (LOO) estimates for the model coefficients. This enables the calculation of sample influence scores to identify outliers or influential observations without re-fitting the model $n$ times.

Results
The paper evaluates NeRF+ through simulations and two case studies:

Simulations: NeRF+ consistently achieves competitive or superior prediction accuracy (measured by $R^2$ $R^{2}$ ) compared to RNC, Network BART, standard RF+, and linear regression across various functional forms (linear, polynomial, locally spiky sparse) and network effect types (additive blockwise, network autocorrelation).
- NeRF+ correctly identifies signal features over non-signal features.
- Global importance measures accurately reflect the strength of the network effect and successfully distinguish between the contributions of network cohesion and embeddings.
- Sample influence scores effectively identify outliers, ranking them as the most influential samples as the perturbation strength increases.
Case Study 1 (School Conflict): In predicting student perceptions of school friendliness, NeRF+ performed competitively with RNC (the top performer on average) but with greater consistency across different schools. The interpretability tools revealed that while baseline friendliness was the strongest predictor, network effects varied by school: some were driven by community structure (embeddings), while others were driven by peer-to-peer cohesion. Local importance scores identified specific friend groups with strong negative peer influences that were not obvious from raw data.
Case Study 2 (Philadelphia Crime): Predicting monthly crime rates across census tracts. NeRF+ and RNC outperformed non-network methods. The results indicated that network cohesion was the dominant factor (far more important than weather covariates or time), reflecting the spatial smoothness of crime rates. Local network importance maps closely matched observed crime rate patterns, highlighting the model's ability to capture spatial dependencies.

Significance and Claims
The paper claims that NeRF+ bridges the gap between the flexibility of deep learning/tree ensembles and the interpretability of linear models in the context of networked data.

Intrinsic Interpretability: Unlike GNNs, NeRF+ provides model-based interpretations derived directly from parameters, avoiding the instability of post-hoc explainers.
Disentanglement: It uniquely allows for the quantification of how much prediction power comes from the network structure (cohesion vs. embedding) versus the node features.
Applicability: The method is particularly valuable for high-stakes domains (e.g., biomedical, social science) where transparency and understanding how a model makes predictions are as critical as accuracy.
Practicality: The authors provide an R package with a Shiny application to facilitate the use of these tools, and demonstrate that NeRF+ can be fit efficiently (under 3 minutes for datasets up to $n=1500$ ), significantly faster than Network BART.

The authors modestly note that interpretations are specific to the fitted model and do not constitute causal inference, which remains a complex, separate area of research. They suggest future work could involve joint learning of embeddings and the model, or extensions to dynamic networks.

More like this