📈 economics

SplitWise Regression: Stepwise Modeling with Adaptive Dummy Encoding

The paper introduces SplitWise, a novel R package that enhances stepwise regression by adaptively converting numeric predictors into threshold-based binary features via shallow decision trees only when they improve model fit according to AIC or BIC, thereby achieving a balance between capturing nonlinear relationships and maintaining model interpretability.

Original authors: Marcell T. Kurbucz, Nikolaos Tzivanakis, Nilufer Sari Aslam, Adam M. Sykulski

Published 2026-02-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Marcell T. Kurbucz, Nikolaos Tzivanakis, Nilufer Sari Aslam, Adam M. Sykulski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict how well a car will perform based on its features, like engine size or weight. Traditionally, statisticians use a simple "straight-line" approach: they assume that if you double the engine size, the performance changes by a fixed amount. It's easy to understand, but it's often wrong because real life isn't a straight line. Sometimes, a car only gets significantly worse once the engine gets too big, or a feature only matters after it crosses a specific threshold.

On the other hand, modern machine learning uses complex "black box" methods (like deep neural networks) that can find these weird, non-straight patterns. But they are so complicated that no one can explain why they made a prediction. It's like having a GPS that gives you the right turn but refuses to tell you the map.

Enter "SplitWise": The Best of Both Worlds

The paper introduces a new tool called SplitWise Regression. Think of it as a smart, flexible assistant that helps you build a simple, transparent model that can still handle the messy, non-straight realities of the real world.

Here is how it works, using a few everyday analogies:

1. The "Smart Switch" (Adaptive Dummy Encoding)

In a standard model, a variable like "Age" is treated as a continuous line. SplitWise asks: "Does this variable behave like a line, or does it have a 'tipping point'?"

If the data suggests that "Age" only starts affecting health outcomes after you turn 50, SplitWise doesn't force a straight line. Instead, it creates a smart switch. It turns the "Age" variable into a simple "Yes/No" question: "Is the person older than 50?"

If Yes, it applies one rule.
If No, it applies another (or no rule at all).

This is like a thermostat. You don't need to know the exact temperature curve of a room; you just need to know: "Is it above 70 degrees? If yes, turn on the AC." SplitWise automatically finds these "tipping points" (thresholds) in the data.

2. The "Strict Accountant" (Stepwise Selection with AIC/BIC)

You might worry: "If we keep adding these 'Yes/No' switches, won't the model get too complicated and confusing?"

That's where SplitWise's "Strict Accountant" comes in. Before it adds any new switch to the model, it checks a financial ledger called AIC or BIC. These are scores that balance two things:

How well the model fits the data (Accuracy).
How many rules the model has (Complexity).

If adding a new "switch" (like "Is blood pressure > 140?") doesn't improve the score enough to justify the extra complexity, the accountant says, "No thanks, we're keeping it simple." This ensures the final model remains easy to read and understand, avoiding the "overfitting" trap where a model memorizes the noise instead of learning the pattern.

3. The "Two Ways to Cook" (Iterative vs. Univariate)

The paper describes two ways SplitWise builds the model:

The "Team Huddle" (Iterative Mode): The algorithm looks at all the variables together. It asks, "If we already have 'Engine Size' in the model, does adding 'Is Weight > 2000lbs?' help us?" This is precise and accounts for how variables interact with each other.
The "Solo Scout" (Univariate Mode): The algorithm looks at each variable one by one, like a scout checking individual ingredients. It decides the best form for each ingredient independently, then assembles the final dish. This is faster for huge datasets with many variables.

What the Paper Found

The authors tested SplitWise on both made-up data (where they knew the "true" answer) and real-world data (like car fuel efficiency, house prices, and wine quality).

The Result: SplitWise consistently built models that were more accurate than traditional straight-line methods and simpler (fewer variables) than complex machine learning models.
The Sweet Spot: It managed to capture the "bumps" and "jumps" in the data (nonlinearity) without turning the model into an unreadable black box.
The Tool: They released this as a free software package (in the R programming language) so anyone can use it.

The Bottom Line

SplitWise is like upgrading from a rigid ruler to a flexible measuring tape that can snap into place at the exact points where the data changes behavior. It keeps the model transparent enough for a human to understand (great for doctors, policymakers, or engineers who need to explain their decisions) but smart enough to catch the complex, non-linear relationships that simple models miss.

What the paper does not claim:
The paper focuses entirely on the statistical performance and the software tool itself. It does not claim that this method has been tested in specific clinical trials, nor does it predict specific future medical outcomes or financial market crashes. It simply proves that the mathematical method works better than existing alternatives for building clear, accurate regression models.

Technical Summary: SplitWise Regression

Problem Statement

In regression modeling, a persistent challenge exists in capturing nonlinear relationships without sacrificing the interpretability inherent in linear models. Traditional linear regression offers transparency, where coefficients directly quantify predictor influence, but fails to model threshold effects or complex nonlinearities (e.g., health outcomes changing sharply after a specific age). Conversely, machine learning methods like decision trees and neural networks can model these complex relationships but often produce "black box" structures that lack global interpretability, which is critical in high-stakes domains such as healthcare, finance, and public policy.

Existing hybrid approaches, such as RuleFit or tree-based methods with linear components (e.g., M5, PILOT), attempt to bridge this gap. However, they often result in complex model structures with multiple nested splits or ensemble components that diminish transparency. Furthermore, current stepwise selection methods typically rely on p-value thresholds or information criteria but do not automatically detect optimal thresholds within numeric variables to enhance model robustness and reduce overfitting. There is a specific need for an approach that introduces nonlinearity in a controlled manner—creating binary threshold variables that directly enhance linear model performance while preserving the coherence of a global linear equation.

Methodology

The paper introduces SplitWise, a deterministic framework that integrates adaptive, threshold-based transformations into a stepwise regression process. The core innovation is the ability to automatically determine whether a numeric predictor should enter the model as a standard linear term or as one or more binary (dummy) indicators derived from data-driven cut-points.

Algorithmic Modes

SplitWise operates in two distinct modes, both utilizing the Akaike Information Criterion (AIC) or Bayesian Information Criterion (BIC) to balance model fit and complexity:

Iterative Mode:
- Builds the model via a stepwise procedure (forward, backward, or bidirectional).
- At each step, it evaluates excluded predictors to find their best form (linear or dummy-transformed) and current predictors to see if removal improves the model.
- Candidate transformations are generated using shallow decision trees (via the rpart package) with a maximum depth of two.
- The algorithm selects the action (add, remove, or switch form) that yields the lowest AIC/BIC.
- This mode accounts for variable interactions and joint effects, generally offering higher precision.
Univariate Mode:
- Evaluates each predictor independently to determine its optimal representation (null, linear, single-split dummy, or double-split dummy).
- Uses the same shallow tree logic to identify optimal split points.
- Selects the best transformation for each variable based on AIC/BIC before assembling a final model via a subsequent stepwise selection process.
- This mode is designed for scalability in high-dimensional data.

Software Implementation

The method is implemented as an open-source R package (SplitWise). It relies minimally on external dependencies, primarily using base R functions (lm, step) and the rpart package for tree-based splitting. The package outputs a standard regression model object (splitwise_lm) that retains a linear structure but includes metadata detailing the specific cut-points and dummy encodings (e.g., $I(\text{blood pressure} > 140)$ ), ensuring the final model remains human-readable and auditable.

Key Contributions

Adaptive Dummy Encoding: Unlike traditional stepwise regression which treats variables as fixed linear terms, SplitWise dynamically converts numeric predictors into binary threshold features if doing so improves the information criterion.
Interpretability Preservation: The resulting model is a global linear equation augmented by a small number of thresholded features. This avoids the complexity of full tree ensembles while capturing relevant nonlinearities.
Parsimony and Stability: The framework is designed to produce sparse models. By using AIC/BIC for selection, it aims to prevent overfitting and ensure that only meaningful transformations are retained.
Open-Source Tooling: The release of the SplitWise R package provides a reproducible, user-friendly tool for analysts to apply these methods without needing to implement complex tree-search algorithms manually.

Experimental Results

The authors evaluated SplitWise against classical stepwise regression, best subset selection, and penalized regression techniques (LASSO, Ridge, Elastic Net) using both synthetic and real-world datasets.

Synthetic Datasets

Performance: Across datasets with 15, 30, and 60 predictors, SplitWise (particularly in iterative configurations) consistently achieved the lowest Root Mean Squared Error (RMSE) and highest Adjusted $R^2$ .
Model Selection: It produced the lowest AIC and BIC values in most scenarios, indicating a superior trade-off between fit and complexity.
Sparsity: SplitWise identified very sparse models (e.g., 4–5 variables in the 15-predictor setting) while maintaining high accuracy, whereas traditional stepwise and penalized methods often selected larger models for similar or worse performance.
Stability: The method demonstrated high stability (proportion of times the same predictors were selected across resampling runs), often reaching 1.00, whereas traditional stepwise methods showed lower stability in higher dimensions.

Real-World Datasets

Evaluated on five datasets (Bodyfat, Boston Housing, Mtcars, Wine Quality Red/White):

Predictive Accuracy: SplitWise (iterative) consistently achieved the lowest RMSE in complex datasets like Boston Housing and Wine Quality.
Variable Selection: It achieved these results with fewer or comparable numbers of predictors compared to traditional methods. For instance, in the Wine Quality (Red) dataset, it outperformed others with only 7 variables.
Comparison: While traditional stepwise methods remained competitive in lower-dimensional datasets (e.g., Bodyfat), they struggled with efficiency and stability in more complex scenarios. Penalized methods were computationally efficient but generally lagged in RMSE and Adjusted $R^2$ .

Significance and Claims

The paper claims that SplitWise offers a practical solution for domains where both high predictive performance and interpretability are non-negotiable. By blending the rigor of model selection with adaptive thresholding, it allows analysts to model nonlinearities without resorting to opaque machine learning ensembles.

The authors position SplitWise as a tool that "consistently produces more parsimonious and generalizable models than traditional stepwise and penalized regression techniques." They emphasize that the method enables the discovery of meaningful breakpoints in continuous variables while maintaining the transparency of a global linear regression. The authors note that while the method is effective, it is a deterministic framework intended to enhance, not replace, the interpretability of linear modeling, making it suitable for healthcare, economics, and social sciences. Future work is identified as expanding the framework to Python, generalized linear models, and time series, as well as developing accelerated variants for ultra-high-dimensional settings.