💻 computer science

Response-balanced sampling relocates rather than improves accuracy in slope-stability surrogates under a fixed simulation budget

Under a fixed simulation budget, response-balanced sampling for slope-stability surrogates fails to systematically improve overall accuracy and instead merely redistributes predictive precision within the critical decision-making range—yielding a gain in the rare unstable zone [0.5, 1) at the expense of the [1, 2) zone—without providing a net gain over standard Latin hypercube sampling.

Original authors: Iancu-Bogdan Teodoru, Alessia Amelio, Mircea Aniculăesi, Irina Lungu

Published 2026-10-30✓ Author reviewed ⓘ

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Iancu-Bogdan Teodoru, Alessia Amelio, Mircea Aniculăesi, Irina Lungu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of civil engineering, keeping a hillside from sliding down is a matter of life and death. When a slope becomes unstable, it can bury roads, destroy homes, or trigger catastrophic landslides. To prevent this, engineers calculate a number called the "factor of safety." Think of this number as a margin of safety: a value of one means the slope is on the very edge of collapsing, while a value of two means it is twice as strong as it needs to be to stay put. The higher the number, the safer the ground. However, calculating this number requires running physics simulations that model how soil, rock, and gravity interact. While some highly complex models can be very time-consuming, any engineering project operates under a finite computational budget. Because of this limited budget, engineers cannot test every possible variation of a slope's shape or soil type. They need a way to allocate their available simulations as efficiently as possible.

This is where machine learning steps in. Engineers train computer programs, known as "surrogates," to mimic the physics simulations. Once trained, these programs can predict the safety factor in milliseconds, allowing for rapid testing of thousands of scenarios. But for a machine learning program to be smart, it needs to be fed good data. The central question researchers have been asking is: how should we choose which specific slope scenarios to run the simulations on in the first place? If we pick the wrong ones, the computer program might become very good at predicting safe slopes but terrible at predicting dangerous ones, or vice versa. The standard method for picking these scenarios has been a technique called Latin hypercube sampling, which spreads the test points evenly across all the input variables, like soil weight or slope angle, without looking at the final safety result.

A team of researchers from Romania and Italy decided to test a different idea. They wondered if it would be better to pick the test points based on the safety result itself. Their strategy, which they call "response-balanced sampling," involves running a quick, rough guess first to see what safety factors a large pool of potential slopes might have. Then, they would deliberately choose the simulations to fill in the gaps where the results were rare. For example, if very few of their initial guesses resulted in a safety factor between 0.5 and 1.0 (a dangerous zone), they would force the computer to run more simulations specifically in that range to ensure the machine learning model learned it well. The goal was to create a perfectly balanced training set where every level of safety was represented equally, hoping this would make the final model more accurate everywhere.

To test this, the researchers set up a controlled experiment using a simple, uniform soil slope. They started with a standard set of 300 simulations generated by the traditional method. They then had a budget of 300 more simulations to spend. One group of computer models was trained using the traditional method, adding another 300 random simulations. The other groups used the new "response-balanced" strategy, using a preliminary model to guide them toward the rare safety values. They ran this experiment multiple times with different random draws to ensure the results were not just luck, and they tested the final models against two different sets of unseen data.

The results were surprising and overturned the initial intuition. The researchers found that the new strategy did not make the models more accurate overall. Instead, it simply moved the accuracy from one place to another. By forcing the model to learn more about the rare, unstable slopes (those with safety factors below 1.0), the strategy inadvertently took training points away from the zone where safety factors are between 1.0 and 2.0. This entire range—from 0.5 to 2.0—is the critical decision-making zone for engineers, as it covers both unstable slopes awaiting remediation and the range where they must decide if a slope is safe enough for construction.

In their specific test environment, the traditional method actually produced a lot of data in that 1.0 to 2.0 range naturally. When the researchers tried to "balance" the data by adding more points to the low-safety zone, they ended up thinning out the data in the higher-safety part of the decision range. Consequently, the models trained with the new strategy became slightly better at predicting the rare, unstable slopes but slightly worse at predicting the slopes in the 1.0 to 2.0 range. The accuracy was not improved; it was just relocated along the scale of safety. Even when the researchers used a perfect "oracle" system that knew the exact safety values in advance and could allocate points perfectly without any guessing errors, the same shift occurred. The model still gained accuracy in the rare zone and lost it in the other part of the decision-making zone.

The study concludes that trying to force a balanced distribution of results does not automatically lead to a better model, especially when the budget for running simulations is fixed. In this case, the "rare" data points were not the ones where the model needed the most help; the model was already quite good at the common values. The researchers suggest that before trying to reshape how data is collected, engineers should first check what their current method is already producing. If the standard method already provides plenty of data in the zone where decisions are made, trying to balance the distribution might actually hurt the model's performance where it matters most. The lesson is not that balancing is always bad, but that it is not a magic fix. It changes where the model is smart, and without careful planning, that change can move the intelligence away from the very place that is needed.

Technical Summary: Response-balanced sampling relocates rather than improves accuracy in slope-stability surrogates under a fixed simulation budget

Problem Statement
Machine learning (ML) surrogates are increasingly used to replace computationally expensive physics-based analyses in geosciences, particularly for slope stability where they predict the Factor of Safety (FOS) in milliseconds. While model architectures have matured, the focus has shifted to data-centric machine learning, where the composition of the training set is often more critical than model refinement. A fundamental constraint in this domain is that FOS values cannot be measured experimentally; they exist only as outputs of a solver. Consequently, the training data must be generated by running the solver on chosen parameter combinations.

The standard approach for generating this data is Latin Hypercube Sampling (LHS) in the input space, valued for its space-filling properties. However, because the mapping from input parameters (strength, unit weight, geometry) to FOS is strongly nonlinear, an input-space design that covers parameters evenly results in an uneven distribution of the response (FOS). In imbalanced regression, models typically perform poorly in regions where training data is sparse. The prevailing intuition suggests that "response-balanced sampling"—allocating the remaining simulation budget to even out the FOS distribution—should improve overall accuracy. This paper investigates whether this intuition holds when the total simulation budget is fixed.

Methodology
The study employs a controlled testbed: a homogeneous, drained soil slope with no water table, governed by slope angle, friction angle, and the stability ratio $\gamma H/c$ . The ground truth is generated using the simplified Bishop method via the xslope solver.

The experimental design compares sampling strategies under a fixed budget of 600 analyses per arm:

Baseline (Arm A & B): Arm A uses a 300-point LHS seed. Arm B adds a second, independent 300-point LHS set (total 600), serving as the continued LHS baseline.
Response-Balanced Sampling (Guided Arms): These arms use a "screener" (a random forest trained on the 300-point seed) to predict the FOS of a pool of 200,000 candidate parameter combinations. The remaining 300 budget is allocated to candidates chosen to fill logarithmic bins in the FOS selection range $[0.5, 6]$ $[0.5, 6]$ to match a target distribution.
- C-log: Allocates points in a single batch to fill deficits relative to the seed's distribution.
- C-seq: A sequential version spending the budget in three batches, retraining the screener after each batch.
- C-logG: Uses a gradient-boosting screener instead of a random forest.
Oracle and Contrast Arms: To isolate the effect of the screener's prediction error from the sampling strategy itself, the study uses a reference pool of 22,000 pre-solved analyses.
- C-oracle: Bins candidates based on their solved FOS, meeting the intended density exactly.
- C-logP: Bins candidates based on the screener's predicted FOS, drawn from the same solved pool.
Evaluation: The study uses two tree-ensemble families (Random Forest and Gradient Boosting) as independent measuring instruments. Accuracy is evaluated on two test sets (natural LHS and stratified by predicted FOS) using log-scale Root-Mean-Square Error (log-RMSE). The primary criterion is performance on the "decision range" $FOS \in [0.5, 2)$ , which encompasses unstable slopes and the zone where design decisions are made. The study includes five replicated realizations of the added points to ensure robustness.

Key Results
The results challenge the intuition that balancing the response distribution improves accuracy under a fixed budget:

Relocation, Not Improvement: Response-balanced sampling does not systematically improve accuracy over continued LHS. Instead, it relocates accuracy along the FOS axis. In all tested variants (C-log, C-seq, C-logG), accuracy improved in the rare zone $FOS \in [0.5, 1)$ but degraded in the zone $FOS \in [1, 2)$ . Within the total decision range of $[0.5, 2)$ , there was no demonstrated net gain.
Mechanism of Relocation: Under LHS in this specific parameter domain, the zone $[1, 2)$ is already over-represented (35–37% of points) compared to a log-uniform target (27.9%), while $[0.5, 1)$ is under-represented (11–12%). Evening out the distribution therefore forces the allocation of new points into the rare $[0.5, 1)$ zone at the expense of withdrawing them from the $[1, 2)$ zone.
No Demonstrated Gain: On the primary decision range $[0.5, 2)$ , no design that selects points via a screener showed a demonstrated gain over continued LHS. In one specific configuration (C-log with GBM on the natural test set), the guided design was significantly worse.
Oracle Confirmation: The "Oracle" arm (C-oracle) also exhibited the same relocation effect. This shows that the relocation is present even when the intended density is met exactly (observed in three of the four learner/test-set cells).
Ablations: Designs trained only on the added points (without the seed) performed poorly overall, confirming that the seed is essential. However, even these ablations showed the relocation pattern: they were more accurate in the rare $[0.5, 1)$ zone but less accurate elsewhere, consistent with density-driven learning.

Significance and Claims
The paper claims to provide a "budget-fair" protocol for testing sampling strategies, distinguishing between the value of a larger dataset and the value of a specific allocation rule. Its primary contributions are:

Empirical Evidence: It demonstrates that for slope-stability surrogates, response-balanced sampling relocates accuracy rather than improving it. The gain in the rare region is offset by a loss in the region where engineering decisions are actually made.
Diagnostic Protocol: The authors propose a diagnostic approach: before reshaping a training response, practitioners should compare the FOS distribution provided by standard LHS against the target distribution and the specific decision range of interest. If the target distribution withdraws points from the decision range (as it did here), the strategy is likely to be detrimental.
Methodological Rigor: The study isolates the effect of the sampling strategy from model architecture and hyperparameter tuning by using replicated realizations, ablation studies, and an oracle reference. It shows that the relocation effect persists even when the intended density is met exactly and across different tree-ensemble learners.

The authors conclude that the region that is rare under LHS need not be the region where decisions are made. For a surrogate intended to support design decisions, accuracy in the decision range is the priority, and strategies that trade off density in that range to balance the overall distribution may yield no net benefit. The study does not claim these effect sizes transfer to other domains or parameter spaces, but rather offers a protocol and a diagnostic for evaluating such strategies in future applications.

More like this