🤖 machine learning

Probabilistic indirect models for undrained shear strength: addressing significant data missing and variability with advanced imputation and machine learning techniques

This study proposes a robust probabilistic indirect model for predicting undrained shear strength by integrating multiple imputation techniques to handle missing data with a multi-head attention-enhanced neural network, demonstrating superior accuracy and uncertainty quantification compared to conventional methods.

Original authors: Haibin Xiong, Shaoheng Dai, Peng Lan, Xuzhen He, Chenxi Tong, Sheng Zhang, Daichao Sheng

Published 2026-08-17

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Haibin Xiong, Shaoheng Dai, Peng Lan, Xuzhen He, Chenxi Tong, Sheng Zhang, Daichao Sheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to build a skyscraper on a patch of ground you've never seen before. You can't just guess; you need to know how strong the soil is, or the whole thing might sink or slide. This is the job of geotechnical engineers, who study the "undrained shear strength" of soil—basically, how much force the dirt can take before it slips when water is trapped inside it. It's a bit like trying to figure out how much weight a wet sponge can hold before it squishes flat. The problem is, getting this information is messy. Sometimes you can't test the soil directly because it's too deep or too expensive, so engineers have to guess based on other clues, like the soil's stickiness or how a cone penetrates it. But these guesses are often full of holes, literally and figuratively. The data is missing, the numbers vary wildly from one spot to another, and the old ways of guessing are often just too uncertain to be safe.

This paper is like a team of detectives trying to solve the case of the missing soil data. They grabbed a massive, global "missing persons" file for clay soil called the CLAY/10/7490 database. This file has information from 30 different countries, but it's a disaster: only about 34% of the information is actually there. The rest is blank, like a puzzle with most of the pieces ripped out. The researchers wanted to see if they could use advanced computer tricks to fill in those blanks and then use those filled-in puzzles to predict soil strength more accurately than ever before. They tested three different ways to guess the missing pieces and built two new types of "super-forecasters" to see which combination worked best.

The Great Missing Piece Hunt

The team started by looking at their giant database, which contained 7,490 rows of data about clay. But here's the catch: most of the rows were incomplete. Some columns, like the "Atterberg limits" (which measure how sticky and plastic the clay is), were only missing about 40% of the time. But the columns about "CPTU" measurements (which come from pushing a cone into the ground) were missing over 90% of the time! It was like trying to solve a crime where the witness forgot 90% of what they saw.

First, they tried a simple, old-school method called the Multivariate Normal (MN) model. Think of this like a teacher who knows that students who are good at math are usually good at science, too. If a student is missing a science grade, the teacher guesses it based on their math grade. The researchers used this logic to fill in the missing soil data. They found that while this method kept the general "shape" of the data correct, it didn't actually make the final predictions much better on its own. It was like filling in the puzzle with pieces that looked right but didn't quite fit the picture.

Next, they tried two more sophisticated methods: MICE and Miss Forest (MF).

MICE is like a group of detectives who take turns guessing the missing info. One detective guesses a value based on the clues, then the next detective uses that guess to make their own, and they keep going back and forth until they all agree.
Miss Forest is like a team of super-smart AI detectives who use a "forest" of decision trees to figure out the missing pieces. They look at complex patterns that simple math might miss.

When they compared these methods, they found that MICE was the best at keeping the data looking like the original, real-world soil. It didn't create weird outliers or fake patterns and was the most faithful to the original statistical distribution. Miss Forest was okay, but it sometimes got a little too creative with its guesses. The simple MN method was the least accurate at preserving the true nature of the data, often producing results that were "overly concentrated" compared to the original messy reality.

The Super-Forecasters

Once they had their "filled-in" databases, the researchers built two new types of prediction models to see if they could guess the soil strength better than the old ways.

The PXGB (Probabilistic Extreme Gradient Boosting): Imagine a team of 100 experts who vote on the answer. If one expert is wrong, the others correct them. This model is great at handling messy data, but it's still just a standard "voting" machine.
The MHA-PNN (Multi-Head Attention Probabilistic Neural Network): This is the star of the show. Think of this model as a detective with a superpower: Attention. Just like how you can focus on a specific detail in a crowded room while ignoring the noise, this model has "heads" that can focus on the most important clues in the soil data and ignore the useless ones. It doesn't just look at the data; it understands the relationships between the different clues.

The Big Reveal

The results were clear. When the researchers tested these models, the MHA-PNN model crushed the competition, but the choice of how to fill in the missing data mattered in a specific way.

Accuracy: The MHA-PNN model made mistakes that were about 72% smaller than the PXGB model when using the best-filled data.
Confidence: It didn't just guess the number; it gave a range of likely values (uncertainty). The MHA-PNN model's guesses were much tighter and more reliable. Its "confidence interval" (the range of possible answers) was about 96% narrower than the PXGB model's, meaning it was much more sure of its answer without being wrong.
The "Missing" Problem: They also discovered something important: if you only have a few clues (less than four pieces of data), even the best model struggles. It's like trying to guess a movie plot with only one scene. But once they had enough clues, the MHA-PNN model shined.

Here is the twist: While MICE was the clear winner at preserving the statistical shape of the data (making the filled-in puzzle look most like the real thing), it wasn't the perfect partner for the final prediction. Surprisingly, the MHA-PNN model achieved its absolute best performance when paired with the MN imputation method. The researchers found that MN offered the best trade-off for this specific advanced model. Even though MN didn't preserve the data distribution as perfectly as MICE, it provided the most accurate foundation for the MHA-PNN to build its predictions upon. So, the "best" combination wasn't about picking the single best imputer or the single best model, but finding the specific pairing where the advanced "super-attention" model could do its best work.

Why This Matters

This study suggests that we can build much safer and more reliable models for predicting soil strength, even when our data is incomplete and messy. By using these advanced "attention" techniques and finding the right data-filling partner, engineers might be able to make better decisions about building foundations, tunnels, and slopes without needing to test every single inch of soil. However, the authors are careful to say this isn't a magic wand that replaces real-world testing. It's a powerful tool for getting a good first guess, especially when data is scarce, but for the most critical projects, you still need to check the ground yourself.

In short, the paper shows that by teaching computers to pay attention to the right clues and intelligently filling in the blanks—even if the filling method isn't statistically perfect on its own—we can turn a messy, incomplete puzzle into a clear picture of the ground beneath our feet.

Technical Summary: Probabilistic Indirect Models for Undrained Shear Strength

Problem Statement
Accurate prediction of undrained shear strength ( $s_u$ ) is fundamental to geotechnical design, particularly for transient loading scenarios like seismic events, tunnelling, and slope stability. However, traditional empirical methods are hampered by significant uncertainties, including inherent soil variability, statistical uncertainty due to limited data, measurement errors, and transformation uncertainties. While data-driven indirect models offer a computationally efficient alternative to Bayesian Markov Chain Monte Carlo (MCMC) methods, their development is often constrained by the scarcity of site-specific data. Furthermore, existing global databases, such as the CLAY/10/7490 database (containing 7,490 rows from 251 studies), suffer from high rates of missing data (completion rate ~34.1%) and substantial variability. The challenge lies in effectively utilizing these incomplete, highly variable datasets to build robust probabilistic models without discarding valuable information or introducing bias through simple data handling.

Methodology
The study proposes an integrated framework addressing data incompleteness and model uncertainty through a three-stage approach:

Data Imputation Strategies:
The authors evaluate three distinct imputation methods to handle missing values in the CLAY/10/7490 database (specifically for Atterberg limits and CPTU parameters, excluding the target variable $s_u$ ):
- Multivariate Normal (MN): A statistical baseline method assuming a multivariate normal distribution after transforming variables using the Johnson system (specifically the SU family) and specific transformation functions (logarithmic, inverse sigmoid, inverse hyperbolic).
- Multiple Imputation by Chained Equations (MICE): A statistical approach that iteratively models each variable's missing values based on its own distribution without assuming multivariate normality.
- Miss Forest (MF): A machine learning-based approach using Random Forests to capture non-linear relationships between variables.
  The effectiveness of these methods was first assessed by comparing the statistical distributions (histograms, Kolmogorov-Smirnov tests) of the imputed datasets against the original data.
Probabilistic Modeling Frameworks:
Two probabilistic indirect models were developed and trained on the imputed datasets (and the original incomplete dataset for comparison):
- Probabilistic Extreme Gradient Boosting (PXGB): An extension of XGBoost that assumes the target variable follows a normal distribution. It uses Negative Log-Likelihood (NLL) as the loss function to enable uncertainty quantification.
- Multi-Head Attention-based Probabilistic Neural Network (MHA-PNN): A novel architecture integrating a Multi-Head Attention (MHA) mechanism into a standard Artificial Neural Network (ANN). The MHA layer allows the model to focus on relevant input features and extract information from multiple perspectives, addressing the challenge of limited input data. The output layer predicts the mean and standard deviation of the target distribution, also optimized via NLL.
Evaluation Protocol:
The models were evaluated using stratified 5-fold cross-validation. Performance was measured using:
- Accuracy Metrics: Root Mean Square Error (RMSE), Coefficient of Determination ( $R^2$ ), and Mean Absolute Percentage Error (MAPE).
- Uncertainty Metrics: Conditional Interval Width ( $w_{CI}$ ) and Coverage Rate (CR) for 95% confidence intervals.
  The study specifically excluded cases with three or fewer input features, as preliminary analysis showed these yielded unreliable predictions regardless of the method used.

Key Contributions

Systematic Imputation Comparison: The paper provides a comparative analysis of MN, MICE, and MF imputation techniques on a large-scale, highly incomplete geotechnical database. It demonstrates that while MICE best preserves the original data distribution (lowest K-S test statistics), the choice of imputation method significantly impacts downstream model performance.
Novel MHA-PNN Architecture: The authors introduce the MHA-PNN model, which leverages attention mechanisms to enhance feature extraction from sparse and incomplete geotechnical data.
Probabilistic Framework Integration: The study moves beyond point estimation by developing models that output full probability distributions, explicitly quantifying uncertainty alongside predictions.
Benchmarking: The work establishes a rigorous benchmark comparing traditional statistical prediction (MN-based), ensemble learning (PXGB), and deep learning (MHA-PNN) under varying data completeness conditions.

Results

Imputation Impact: Imputation generally improved model performance compared to training on the original incomplete data. The MICE method yielded the most statistically consistent imputed datasets, while the MF method introduced some outliers.
Model Performance:
- The PXGB model showed significant improvement when trained on imputed data (e.g., MICE dataset) compared to the original data, reducing MAPE by ~42% and RMSE by ~35%.
- The MHA-PNN model consistently outperformed both the PXGB and the traditional MN-based prediction models across all datasets.
- Specifically, the MHA-PNN trained on the MN-imputed dataset achieved the best overall results: MAPE of 0.34, RMSE of 0.37, and $R^2$ of 0.55. This represented a ~72% reduction in MAPE and a ~53% reduction in RMSE compared to the PXGB model on the same dataset.
- In terms of uncertainty quantification, the MHA-PNN model produced significantly narrower confidence intervals ( $w_{CI} \approx 1.03$ ) while maintaining high coverage rates (CR $\approx$ 0.94), indicating well-calibrated uncertainty estimates.
Feature Dependency: The study confirmed that prediction accuracy is highly sensitive to the number of available input features. Models performed poorly when fewer than four features were available, highlighting the necessity of informative inputs (particularly CPTU parameters) for reliable $s_u$ estimation.

Significance and Claims
The authors claim that the proposed integrated strategy—combining advanced imputation with the MHA-PNN architecture—offers a robust solution for building probabilistic indirect models in geotechnical engineering, particularly when dealing with sparse and incomplete datasets. The findings suggest that this approach substantially outperforms conventional empirical correlations and standard machine learning models in both prediction accuracy and uncertainty quantification.

However, the paper maintains a modest stance regarding the practical application of these models. The authors explicitly state that the global database is intended for "first-order estimates" and that the model outputs should be treated as "preliminary guidance." They emphasize that the model is not a substitute for site-specific predictions and that engineers must supplement these results with local testing, especially for geologically distinct or sensitive conditions (e.g., structured or soft clays). The study concludes that while the MHA-PNN framework effectively captures complex feature interactions and improves reliability under data-scarce conditions, site-specific investigations remain essential for reliable decision-making.

The Great Missing Piece Hunt

The Super-Forecasters

The Big Reveal

Why This Matters

Technical Summary: Probabilistic Indirect Models for Undrained Shear Strength

More like this