An End-to-End Explainable AI Framework with Automated LLM-Based Natural Language Explanation Generation for Energy Systems

Technical Summary: An End-to-End Explainable AI Framework for Energy Systems

Problem Statement
While deep neural networks (DNNs) offer high performance in critical domains like energy systems, their "black box" nature hinders deployment in high-stakes scenarios where transparency, trust, and accountability are paramount. Existing Explainable AI (XAI) research often focuses on isolated techniques (either local or global) or relies heavily on visualizations that require domain expertise to interpret. Furthermore, there is a lack of unified frameworks that quantitatively evaluate explanation quality (via fidelity and stability) and translate technical attributions into natural language accessible to non-technical stakeholders.

Methodology
The paper proposes a reusable, end-to-end XAI framework designed to be dataset-agnostic and applicable to both classification and regression tasks. The pipeline consists of four integrated stages:

1. Data Preprocessing and Model Training:

   • Raw tabular data undergoes standardization (z-score for numerical features) and one-hot encoding (for categorical features) via a unified pipeline to prevent data leakage.
   • A Multi-Layer Perceptron (MLP) serves as the black-box model. The architecture includes two hidden layers (192 neurons) for classification and three hidden layers (448 neurons) for regression, utilizing ReLU activation.
   • Optimization is performed using the Adam optimizer with L2 regularization to prevent overfitting.

2. Explainability Generation (LIME and SHAP):

   • Local Explanations: Local Interpretable Model-agnostic Explanation (LIME) approximates the model locally using a surrogate model (logistic regression for classification, weighted ridge regression for regression) to generate instance-specific feature importance.
   • Global and Local Explanations: SHapley Additive exPlanations (SHAP) utilizes KernelSHAP (due to the non-tree nature of neural networks) to compute Shapley values, quantifying feature contributions globally and locally.
   • Evaluation: Explanation quality is rigorously assessed using Fidelity (how well the explanation approximates the model's behavior) and Stability (consistency of explanations under small input perturbations).

3. LLM-Based Natural Language Generation:

   • To bridge the gap between technical outputs and human understanding, structured data (model type, prediction, top features, influence direction, and reliability metrics) is fed into a Large Language Model (LLM).
   • Using a fixed prompt template, the LLM generates concise, five-point natural language summaries covering plot overviews, key findings, feature importance, model behavior, and potential issues.

4. Validation Datasets:

   • Power System Fault Detection: A binary classification task using 12,001 records of three-phase current and voltage measurements.
   • Building Energy Prediction: A regression task using 2,503 records of building attributes to predict site energy use (kBtu).

Key Contributions

• Unified Framework: A modular, end-to-end pipeline integrating black-box prediction, dual-mode explainability (LIME/SHAP), quantitative evaluation, and LLM-based text generation.
• Quantitative Evaluation: The incorporation of fidelity and stability metrics to objectively measure explanation reliability, moving beyond qualitative visual inspection.
• Accessibility: The automation of converting complex numerical attributions into natural language explanations suitable for non-technical stakeholders.
• Cross-Task Validation: Demonstration of the framework's efficacy across both classification (fault detection) and regression (energy prediction) domains.

Experimental Results

• Fault Detection (Classification):
  • The MLP achieved 99% testing accuracy and a ROC-AUC of 1.00.
  • Explanation quality was exceptional, with Fidelity at 1.00 and Stability at 0.998, indicating highly reliable and consistent explanations.
  • SHAP identified phase currents (A and B) and voltages as critical features.
• Building Energy Prediction (Regression):
  • The MLP achieved an R² score of 0.67 on the test set.
  • Fidelity remained high (0.998), but Stability dropped to 0.719. The authors attribute this lower stability to the complex, non-linear interactions between building characteristics (e.g., type, size) and the sensitivity of SHAP values to small input variations in this domain.
• Comparative Analysis (Neural Network vs. Random Forest):
  • In fault detection, the Neural Network (NN) slightly underperformed Random Forest (RF) in accuracy (0.990 vs. 0.997) but achieved higher SHAP fidelity (1.00 vs. 0.985).
  • In energy regression, RF outperformed NN in predictive metrics (R² 0.70 vs. 0.67) and significantly outperformed in explanation stability (0.982 vs. 0.719). The authors note that while NNs generalize well, their explanations are less robust for regression tasks involving categorical features compared to tree-based models.

Significance and Claims
The paper claims that the proposed framework successfully enhances the interpretability of black-box models for high-stakes engineering applications. By combining rigorous quantitative evaluation with LLM-driven natural language generation, the system provides "stable and faithful" explanations that are accessible to diverse stakeholders. The authors assert that while the framework performs exceptionally well in safety-critical classification tasks (fault detection), the lower stability observed in regression tasks highlights the inherent complexity of explaining non-linear relationships in energy prediction. Ultimately, the work provides a model-agnostic solution to improve transparency, though it acknowledges that explanation stability varies depending on the underlying data complexity and model architecture.