Impact Range Assessment (IRA): An Interpretable Sensitivity Measure for Regression Modelling

Technical Summary: Impact Range Assessment (IRA) for Regression Modelling

Problem Statement
Regression models are widely used to model relationships between predictors and numeric response variables; however, interpreting the influence of individual predictors remains challenging. In linear regression, coefficients are scale-dependent and lose their original unit interpretation after standardization. In nonlinear models, particularly machine learning algorithms, the lack of interpretable parameters makes it difficult to compare predictor influences. Existing sensitivity analysis methods have limitations: local derivative-based approaches only capture effects within a restricted region of the input space, while global methods (e.g., Sobol, Morris) are often computationally intensive and mathematically complex. Furthermore, visual inspection methods like scatter plots lack a quantitative metric to rank predictor importance. There is a need for an intuitive, efficient, and model-agnostic method to quantify and rank the influence of predictors across their full observed range.

Methodology: Impact Range Assessment (IRA)
The authors propose the Impact Range Assessment (IRA), a model-agnostic sensitivity measure designed to quantify the maximal influence of each continuous numeric predictor on a regression model's response variable. The method operates by measuring the total potential change in the response variable when a specific predictor is varied across its observed data range, while other predictors are held constant based on background observations.

The IRA algorithm consists of five steps:

1. Selection: A single predictor is selected as the "focus predictor."
2. Interpolation: $M$ evenly spaced values are generated between the focus predictor's minimum and maximum observed values.
3. Background Sampling: $K$ observations are randomly drawn with replacement from the original training dataset to serve as background contexts.
4. Modification: For each background observation, the focus predictor is replaced by the $M$ interpolated values, creating $M$ modified input sets per background sample.
5. Calculation: The model predicts the response for all modified sets. For each background observation $k$, the range of predicted outputs ($\max \hat{y}_i^k - \min \hat{y}_i^k$) is calculated. The IRA value for predictor $i$ is the average of these ranges across all $K$ background observations:
   $$IRA_i = \frac{1}{K} \sum_{k=1}^{K} (\max \hat{y}_i^k - \min \hat{y}_i^k)$$

The method supports two modes of execution:

• Single-execution: Provides a point estimate of predictor impact.
• Repeated-execution: Repeats the process multiple times with different random background samples to generate a distribution of IRA values, allowing for the calculation of 95% confidence intervals (CIs) to assess stability and uncertainty.

Key Results
The method was evaluated using synthetic datasets (linear and nonlinear) and a real-world case study involving a feed mill dataset.

• Synthetic Linear Data: When applied to a linear regression model, the single-execution IRA successfully distinguished relevant predictors from irrelevant noise variables. Relevant predictors ($X_1, X_3, X_4, X_6, X_7$) yielded significantly higher IRA values than irrelevant ones ($X_2, X_5, X_8$). The results were robust and identical across various configurations of interpolated points (50–500) and background sample sizes (50–500), indicating that repeated analysis is unnecessary for simple linear models.
• Synthetic Nonlinear Data: Applied to a Random Forest regressor, the IRA again correctly identified relevant predictors with higher values than irrelevant ones. The study found that using 50 interpolated points slightly underestimated impacts in nonlinear relationships, whereas 100 or more points provided stable estimates. Similarly, increasing background observations from 50 to 100 improved stability, with diminishing returns beyond 100. A configuration of 100 interpolated points and 200 background observations was deemed optimal for balancing stability and computational cost. Repeated IRA analysis (50–90 repeats) confirmed the stability of the rankings and provided narrow confidence intervals.
• Case Study (Pellet Durability Index): In a multiple linear regression model predicting Pellet Durability Index (PDI) from feed manufacturing data, the IRA ranked predictors by influence. The results highlighted "Fat Content," "ADF Content," and "Indoor Humidity" as the most influential factors.
  • Comparison with Previous Methods: The authors contrasted IRA with a previous local sensitivity analysis (You et al.) that varied predictors by ±20% of their mean. While the local method identified "Expanding Temperature" as most influential based on a narrow range, the IRA, which considered the full observed range (62.5°C to 111.7°C), revealed that "Fat Content" had a larger total impact (approx. 5.5 unit change in PDI) compared to "Cumulative Production" (approx. 1.6 unit change) despite the latter's wider numerical range. This demonstrated IRA's ability to provide a global summary metric.

Significance and Claims
The paper claims that IRA offers a simple, intuitive approach to interpret and rank predictor influence, thereby improving model transparency and reliability. Its primary contributions include:

1. Interpretability: It provides a single, interpretable value representing the maximum possible change in the response variable due to a predictor's variation, accessible to both technical and non-technical users.
2. Model Agnosticism: The method applies to both linear and nonlinear models, including those with transformed variables or predictors on different scales.
3. Interaction Capture: By utilizing background observations, IRA partially captures interaction effects based on realistic combinations of predictors, unlike methods that vary one predictor in isolation without context.
4. Robustness: The method allows for repeated analysis to assess result stability and generate confidence intervals, addressing the uncertainty inherent in random sampling.

Limitations
The authors acknowledge several constraints:

• IRA is not well-suited for discrete numeric or categorical predictors without adaptation to unique observed values.
• It is challenging to apply to logistic regression due to the probabilistic nature of the output.
• IRA values are dataset-dependent and cannot be compared across datasets with different predictor ranges.
• Results may be sensitive to extreme values; the authors suggest restricting the analysis range (e.g., 25th–75th percentile) if outliers are present.
• Parameter tuning (number of interpolated points and background samples) may be required for different models.