Technical Summary: Model Instability in Software Engineering Optimization

1. Problem Statement

In software analytics, rerunning the same analysis on identical data often yields different models and conclusions. This phenomenon, termed model instability, undermines trust in software engineering (SE) tools and limits their practical adoption. While previous literature has acknowledged instability in specific domains (e.g., defect prediction, text mining, causal discovery), it is often treated as a "bug" to be fixed with more data or ensembling, or dismissed as unavoidable noise.

This paper argues that instability is not merely noise to be tolerated but a property that can be measured and managed. The authors distinguish between two types of instability:

• Structural Instability: Variation in model form, features used, and tree topology (e.g., different decision trees generated from the same data).
• Performance Instability: Variation in the outcomes or recommendations produced by the models (e.g., different predicted optimal configurations).

The paper posits that structural instability is often a fundamental property of the data (the "Rashomon Effect," where many structurally different models fit the data equally well) and may be irreducible. However, performance instability is the critical issue for practitioners, as it determines whether a single run of an optimizer can be trusted to provide consistent recommendations.

2. Methodology

The study employs a large-scale empirical approach using the MOOT repository, a collection of 127 multi-objective SE optimization problems.

Data and Scope

• Datasets: 127 datasets spanning software configurations, project health, defect prediction, testing, and behavioral analysis.
• Scale: 12,700 held-out test cases across all datasets.
• Learner: The primary learner is EZR, a state-of-the-art active-learning optimizer that constructs decision trees from a limited labeling budget.
• Experimental Design: Each configuration is repeated 20 times with different random seeds to measure variance.

Metrics

1. Performance Stability: Measured by the agreement rate (percentage of test cases where 20 repeated runs produce sufficiently similar predictions) and the standard deviation of optimization error (distance-to-heaven).
2. Structural Stability: Measured using weighted Jaccard similarity of feature sets across decision trees, weighting features by their depth in the tree.
3. Optimization Quality: Assessed via "distance-to-heaven" (d2h) and statistical top-rank counts across datasets.

Interventions

The authors systematically tested six factors to determine their impact on instability:

1. Labeling Budget: The number of labeled examples available (10 to 200).
2. Acquisition Strategy: Rules for selecting which data to label next (e.g., near, Xplor, Xploit, adapt).
3. Model Complexity: Minimum leaf size in decision trees.
4. Splitting Criterion: Entropy vs. Gini impurity.
5. Causal-Inspired Interventions: Confounder filtering and Gain-Ratio splitting.
6. Data Locality: Using clustering (KMeans, HDBSCAN, CURE) to partition heterogeneous data before modeling.

3. Key Contributions and Results

RQ0: Prevalence of Performance Instability

Under default settings, repeated runs of the optimizer agreed on only 2.9% of test cases (364 out of 12,700). Even with a highly permissive threshold, agreement rarely exceeded 31.7%. This confirms that performance instability is the norm, not the exception, rendering single-run recommendations unreliable.

RQ1: Nature of Instability (Structural vs. Performance)

The study found that structural and performance instability are decoupled.

• Finding: Configuration changes that drastically improved performance stability (increasing agreement from 2.9% to 13.7%) had almost no effect on structural similarity. The trees remained structurally diverse.
• Implication: Practitioners should not expect the same model structure every time. Trust should be based on consistent recommendations, not identical model forms. Structural instability is largely irreducible due to the Rashomon effect.

RQ2: Drivers and Fixes

The authors identified four key configuration factors that drive performance instability and proposed a refined configuration that improves stability without sacrificing quality:

1. Labeling Budget: Increasing from 20 to 50 labels improved stability, but gains saturated beyond 50.
2. Acquisition Strategy: Switching from Xploit (greedy) to near (selecting candidates closest to the current best centroid) yielded the most significant stability gains (top-rank stability increased from <50% to >80%) with no cost.
3. Model Complexity: A minimum leaf size of 3 offered the best balance between stability, performance, and interpretability.
4. Splitting Criterion: Switching from Entropy to Gini impurity provided a "free" win in stability, as Entropy is highly sensitive to rare events under tight budgets.

Results of Refined Configuration:

• Agreement: Increased from 2.9% to 13.7% (a 4.8× improvement).
• Error Variance: Reduced the standard deviation of optimization error by 22% (mean std from 17.4 to 13.6).
• Quality: The refined settings achieved statistically top-ranked optimization quality on 119 of 127 datasets, compared to 74 for the defaults.

RQ3: Limits of Stability

The study investigated whether remaining instability was due to the learner or the data.

• Causal Interventions: Adding confounder filters and causal reasoning helped only on specific datasets where confounded features drove instability, but offered no significant global improvement.
• Data Locality: Clustering methods (the most stable model families) agreed on fewer than 51% of test cases.
• Conclusion: A "stability floor" exists, set by the data itself (noise, scarce labels, proxy objectives, and the existence of many near-equivalent models). No learner-side intervention can fully eliminate this residual instability.

4. Significance and Claims

The paper claims that model instability should be treated as a standard evaluation axis in Search-Based Software Engineering (SBSE), routinely measured and reported alongside performance metrics.

• Reframing the Goal: The objective is not to force repeated runs to produce identical trees (which is often impossible due to the Rashomon effect) but to ensure they produce consistent recommendations.
• Actionable Baseline: The proposed refined settings (50 labels, near acquisition, min leaf=3, Gini splitting) provide a baseline against which future efforts to reduce SBSE instability can be judged.
• Practical Guidance:
  1. SE tools should move away from unstable defaults (Entropy splitting, greedy acquisition).
  2. Empirical papers must report agreement rates, not just mean performance, to reveal cross-run disagreement.
  3. Researchers should avoid chasing structural stability for its own sake and recognize that instability has multiple causes requiring compounded fixes rather than a single "silver bullet."

The authors conclude that while instability cannot be fully eliminated, it can be managed to a level where practitioners can calibrate their trust in any single run of an optimization tool.