A transferable explainable and uncertainty-aware machine learning framework for water quality classification in data-scarce regions

Technical Summary: A Transferable Explainable and Uncertainty-Aware Machine Learning Framework for Water Quality Classification in Data-Scarce Regions

Problem Statement
Water quality assessment in data-scarce regions faces critical challenges, including fragmented monitoring networks, heterogeneous datasets, variable label quality, and uncertainty in decision thresholds. While Water Quality Indices (WQIs) offer a simplified communication tool, they often obscure local hydrochemical complexity and uncertainty, particularly when thresholds are transferred from different contexts. Existing machine learning (ML) approaches frequently prioritize predictive accuracy over task definition, often conflating independent classification problems with rule-recovery exercises based on index-derived labels. Furthermore, these models often lack explainability and uncertainty estimation, leading to overconfident predictions in environments where physicochemical parameters may not fully separate water quality classes. The study addresses the need for a cautious, reproducible decision-support framework that explicitly handles data scarcity, task heterogeneity, and the risk of information leakage.

Methodology
The authors propose a transferable framework that integrates open-access datasets, WQI reasoning, leakage-aware task construction, supervised learning, explainable AI (XAI), and uncertainty estimation. The workflow proceeds through the following stages:

1. Data Curation and Harmonization: Five open-access datasets were audited and harmonized. This involved standardizing column names, converting variables to numerical formats, handling missing values via median imputation (using training-set statistics only), and removing duplicate records. Crucially, a leakage-aware protocol was enforced: variables directly encoding target labels or WQI scores were excluded from the predictor matrix to prevent circular reasoning.
2. Task Definition: The study explicitly separates three distinct task families to avoid misinterpreting model performance:
   • Binary Water Potability Classification: The primary "difficult" task, aiming to distinguish potable from non-potable samples using physicochemical descriptors.
   • Structured Groundwater Quality Classification: A hydrochemical classification task based on parameters like salinity and sodicity.
   • WQI Rule-Recovery: A secondary task where models attempt to reproduce classifications derived from WQI logic, serving as a test of rule approximation rather than independent field validation.
3. Label Construction: For WQI tasks, labels were generated using various formulations (equal-weight, weighted arithmetic, entropy-weighted, CRITIC-weighted, and hybrid entropy–CRITIC). These were strictly used for rule-recovery tasks, not as ground truth for potability.
4. Modeling and Optimization: A broad set of algorithms (Random Forest, Extra Trees, HistGradientBoosting, XGBoost, LightGBM, SVM, KNN, MLP) was evaluated. Hyperparameter optimization focused on F1-macro and balanced accuracy to address class imbalance. Resampling techniques (SMOTE, ADASYN) were applied only within the training fold to prevent leakage.
5. Explainability and Uncertainty:
   • Explainability: SHAP (Shapley Additive Explanations) and permutation importance were used to identify physicochemical drivers.
   • Uncertainty: The framework employs probability calibration, confidence scoring, decision margins, entropy analysis, and split-conformal prediction. This allows the system to output a single class when evidence is strong, or a set of possible classes (or a flag for expert review) when uncertainty is high.

Key Results

• Task-Specific Performance:
  • Binary Potability: The Random Forest model achieved moderate performance (Accuracy: 0.6890, Balanced Accuracy: 0.6627, F1-macro: 0.6655, ROC-AUC: 0.6902). This confirms that available physicochemical descriptors only partially separate potable and non-potable samples.
  • Groundwater Classification: Models achieved near-perfect scores (Accuracy: 1.0000) on Telangana datasets, indicating that hydrochemical classes in these curated datasets are highly structured and separable.
  • WQI Rule-Recovery: Models achieved very high performance (Accuracy > 0.98), confirming that ML can effectively approximate index-based decision rules. However, the authors caution that this reflects rule recovery, not independent field validation.
• Uncertainty Analysis: For the difficult potability task, the uncertainty layer revealed that only ~6.25% of predictions were highly reliable. However, this reliable subset achieved an accuracy of ~0.8780. Conversely, ~63.11% of samples were flagged for expert review, preventing overconfident decisions on ambiguous data.
• Explainability: The models identified scientifically meaningful drivers. For potability, key variables included sulfate, pH, solids, chloramines, and hardness. For groundwater, electrical conductivity, sodium adsorption ratio (SAR), and total hardness were dominant.

Key Contributions

1. Methodological Distinction: The study establishes a critical separation between independent classification tasks and WQI rule-recovery tasks, preventing the overstatement of model capabilities when labels are derived from the same features used for prediction.
2. Uncertainty-Aware Decision Support: Rather than acting as a "blind automatic classifier," the framework functions as a decision-support tool that explicitly flags uncertain samples for expert review, a crucial feature for data-scarce regions.
3. Transferable Framework: The workflow provides a reproducible methodology for auditing, cleaning, and modeling water quality data that can be adapted to local contexts once field data becomes available.
4. Cautious Interpretation: The paper refrains from claiming universal generalization, instead positioning the work as a foundational phase for future local calibration.

Significance and Claims
The authors claim that this framework offers a "cautious and reproducible decision-support framework" rather than a definitive automatic classifier. Its primary significance lies in its applicability to data-scarce regions (specifically citing the context of Guinea) where local monitoring data is limited or fragmented. By integrating explainability and uncertainty estimation, the framework ensures that predictions are accompanied by an assessment of reliability, thereby supporting public health and environmental management without replacing necessary laboratory confirmations.

The paper explicitly states that the current results are based on open-access datasets and do not constitute direct operational predictions for Guinean water systems. The framework is designed to be adapted, recalibrated, and validated in a second phase using local field data (wells, boreholes, surface waters) and expanded variables (microbiological indicators, heavy metals, seasonal effects). The work serves as a methodological foundation for future deployment in African contexts where robust, transparent, and uncertainty-aware modeling is required before field-level implementation.