💻 computer science

Sparse-Selective Similarity Regularisation for Enhanced Explanations

The paper proposes Sparse-Selective Similarity ( $S_3$ ), a novel regularisation method that dynamically balances sparsity and feature similarity to enhance the faithfulness, stability, and robustness of local explanations in linear models without compromising predictive accuracy.

Original authors: Ismail Alarab, Mohammad Naiseh, Simant Prakoonwit

Published 2026-10-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ismail Alarab, Mohammad Naiseh, Simant Prakoonwit

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, there is a growing demand for machines that do not just make decisions, but can explain why they made them. This field, known as explainable AI, is crucial in high-stakes areas like healthcare and finance, where a user needs to understand the reasoning behind a diagnosis or a loan approval to trust the system. At the heart of many of these systems are mathematical models that look at a list of characteristics—such as age, blood pressure, or income—and assign a weight to each one to determine the final outcome. Ideally, these weights reflect the true importance of each factor. However, a common problem arises when the model becomes too reliant on just one or two of these factors, assigning them massive importance while ignoring other relevant details. This imbalance can create a distorted view of how the decision was reached, making the explanation misleading even if the final prediction happens to be correct.

Researchers at Bournemouth University have developed a new method to fix this specific problem of unbalanced importance. They call their approach Sparse-Selective Similarity. The goal is to train computer models so that they distribute their attention more fairly among the factors that actually matter, rather than letting a single dominant factor shout over the rest. To test this, the team used a well-known medical dataset containing thirty different measurements related to breast tumors, with the goal of predicting whether a tumor is malignant or benign. They started with a standard computer model that had already learned to make these predictions with high accuracy, but which tended to let a few features dominate the explanation. They then applied their new method to adjust how the model learned, specifically targeting the situation where multiple features were contributing significantly to a decision.

The new method works by gently nudging the model to treat important features more equally. If the model decides that three different measurements are all playing a major role in a prediction, the method encourages the mathematical values representing those measurements to be similar in size. Crucially, it does this only when necessary. If the model truly relies on just one feature, the method leaves it alone, avoiding unnecessary changes. This selective approach ensures that the model does not lose its ability to make accurate predictions while it is being adjusted for better explanations. The researchers found that after applying this technique, the model's ability to predict whether a tumor was malignant or benign remained exactly the same, with an accuracy rate of 98 percent. The model had not been weakened; it had simply been reorganized.

The real test, however, was whether this reorganization made the explanations more trustworthy. The team used three different standard tools designed to explain how a model thinks, checking how well the top few features could reproduce the model's decision and how much the decision would change if those features were removed. The results showed a clear improvement in the quality of the explanations. The models trained with the new method showed a stronger reliance on a broader set of features. When the researchers removed the most important features, the model's confidence dropped more significantly than it did with the standard model, indicating that the model was genuinely using a richer set of information rather than leaning on a single crutch. This suggests that the explanations generated by these models are more faithful to the actual decision-making process.

The study also looked at how stable these explanations were when the input data was slightly tweaked. The models using the new method produced more consistent results, showing less erratic behavior when the data changed slightly. This stability is vital for building trust, as it means the explanation is not a fluke of a specific data point but a reflection of a robust reasoning process. The researchers confirmed these findings with statistical tests, which showed that the improvement in how the models distributed importance was not a random occurrence but a consistent effect of their method. By balancing the weights assigned to different features, the new technique allows the model to provide a more complete and honest picture of why a decision was made, without sacrificing the accuracy of the decision itself. This work offers a practical way to make inherently understandable models even more transparent, ensuring that when a machine explains its reasoning, that explanation reflects a balanced view of the evidence.

Technical Summary: Sparse-Selective Similarity Regularisation for Enhanced Explanations

Problem Statement
The paper addresses a specific challenge within Explainable AI (XAI) known as the imbalanced feature importance problem. While XAI is critical for building trust in fields like healthcare and finance, existing models often suffer from skewed coefficient distributions. In linear models such as logistic regression, a few features with large coefficients can dominate the decision boundary, causing other relevant features to be overlooked in local explanations. This imbalance distorts the output of post-hoc explanation techniques like LIME and SHAP, leading to explanations that are dominated by a single feature rather than reflecting a balanced contribution from multiple relevant variables. The authors argue that this is not merely a data quality issue but a modelling problem where even well-curated datasets can yield models with compromised local interpretability.

Methodology
To mitigate this issue, the authors propose a novel regularisation technique called Sparse-Selective Similarity (S3). Unlike traditional regularisers:

L1 (Lasso) enforces sparsity but does not address the relative magnitude of non-zero coefficients.
L2 (Ridge) promotes smoothness but does not enforce sparsity.
ElasticNet combines both but does not specifically target the similarity of active coefficients.

The S3 regulariser is designed to balance sparsity with context-aware similarity. It operates on the following logic:

Selective Application: The penalty is applied only to a subset of "active" features ( $A$ ) that contribute significantly to the prediction. A feature $j$ is included in set $A$ if its relative contribution $|\beta_j| / \sum |\beta_k|$ exceeds a threshold $\tau$ .
Pairwise Similarity Penalty: For features within the active set $A$ , the regulariser applies a penalty based on the absolute difference between coefficients ( $|\beta_i - \beta_j|$ ). This encourages similar magnitudes among dominant features.
Conditional Logic: The penalty is only enforced if $|A| \ge 2$ . If only one feature dominates, the similarity penalty is not applied, avoiding degenerate penalties in truly sparse scenarios.

The objective function is defined as minimizing the logistic loss plus the S3 regularisation term ( $\lambda \cdot R_{S3}(\beta)$ ).

Experimental Setup
The authors evaluated S3 using the Breast Cancer dataset from scikit-learn (30 features, binary classification).

Baseline: A logistic regression model with standard L1 regularisation.
Proposed Model: A logistic regression model trained with the S3 regulariser (initialized with the L1 coefficients), using a regularisation weight $\lambda = 5$ and a feature importance threshold $\tau = 0.1$ .
Evaluation Metrics: The study assessed local explanation faithfulness using three post-hoc techniques: LIME, SHAP, and Feature Influence. The quality of explanations was measured via:
- Sufficiency: How well top-K features preserve the original prediction.
- Necessity: How much model confidence drops when top-K features are removed.
- Prediction Variability: The sensitivity of predictions to perturbations in top-K features.

Key Results

Predictive Performance: The S3-regularised model maintained classification performance identical to the baseline (98% Accuracy and F1-score), demonstrating that the new regularisation does not sacrifice predictive power.
Coefficient Distribution: Visual analysis showed that S3 successfully encouraged similar magnitudes among dominant features while leaving smaller coefficients largely unaffected, effectively mitigating feature imbalance.
Explanation Quality:
- Necessity: The S3 model achieved consistently higher necessity scores across all explanation methods (LIME, SHAP, Feature Influence). Statistical analysis (paired t-test) confirmed these improvements were significant ( $p < 0.01$ ), indicating the model relies on a richer set of features rather than a single dominant one.
- Sufficiency: Sufficiency scores remained comparable between the baseline and S3 models, indicating that the most important features in the regularised model were still effective at reproducing predictions.
- Stability: Prediction variability was stable or slightly reduced, suggesting more robust explanations.

Significance and Claims
The paper claims that the S3 regulariser offers a principled solution to the imbalanced feature importance problem in inherently interpretable models. By selectively penalising disparities among significant coefficients, S3 enhances the robustness and informativeness of local explanations without degrading model accuracy. The authors position this work as a response to calls for more rigorous XAI methodologies, demonstrating that model regularisation can directly improve the trustworthiness of post-hoc interpretability techniques. The study concludes that S3 is a practical tool for improving explanation quality in critical applications, with future work suggested for extending these principles to neural network architectures.

Technical Summary: Sparse-Selective Similarity Regularisation for Enhanced Explanations

More like this