🤖 machine learning

Towards Explainable Federated Learning: Understanding the Impact of Differential Privacy

This paper proposes Federated EXplainable Trees with Differential Privacy (FEXT-DP), a federated learning framework that combines decision trees for explainability with differential privacy for enhanced data protection, while analyzing the trade-off where the added privacy layer negatively impacts the model's explainability as measured by SHAP and MDI metrics.

Original authors: Júlio Oliveira, Rodrigo Ferreira, André Riker, Glaucio H. S. Carvalho, Eirini Eleni Tsilopoulou

Published 2026-04-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Júlio Oliveira, Rodrigo Ferreira, André Riker, Glaucio H. S. Carvalho, Eirini Eleni Tsilopoulou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you and your neighbors are trying to build a super-smart weather forecaster for your entire neighborhood. You all have your own private weather stations (your data), but you don't want to share your raw data with anyone because it might reveal when you're home, what your habits are, or other private details.

This is the problem this paper solves. It introduces a new way to build a smart model that respects your privacy and explains why it makes its predictions.

Here is the story of FEXT-DP, broken down into simple concepts.

1. The Setting: The Neighborhood Potluck (Federated Learning)

Usually, to train a smart AI, you gather everyone's data into one giant central computer. But that's risky; if that computer gets hacked, everyone's secrets are out.

Instead, this paper uses Federated Learning. Think of it like a neighborhood potluck where everyone brings a dish (their local model) to a central table, but they don't bring their secret family recipes (their raw data).

The Neighbors (Clients): Each house trains a small "Decision Tree" (a simple flowchart that asks questions like "Is it raining? Yes/No") using only their own data.
The Host (Server): The server collects these flowcharts, mixes them together to make a better "Global Flowchart," and sends it back. No one ever sees anyone else's private data.

2. The Problem: The "Too-Smart" Black Box

While this keeps data safe, modern AI (like Neural Networks) is often a "Black Box." You put data in, and an answer comes out, but no one knows how the AI decided that. It's like a magic trick where the magician won't tell you how the rabbit got out of the hat.

The authors wanted a model that is Explainable. They chose Decision Trees because they are like a simple "Choose Your Own Adventure" book. You can trace the path: "It predicted rain because the humidity was high AND the wind was blowing." It's transparent.

3. The Twist: The Noise Machine (Differential Privacy)

Even with the potluck method, a sneaky neighbor (a hacker) might be able to reverse-engineer the shared flowcharts to guess what your specific data looked like.

To stop this, the authors add Differential Privacy (DP).

The Analogy: Imagine you are trying to guess the average height of people in a room. To protect privacy, you ask everyone to roll a die. If they roll a 6, they lie and say they are a giant. If they roll a 1, they lie and say they are a dwarf.
The Result: The final average is still very accurate, but no one can tell if you specifically were tall or short.
In the Paper: They add "mathematical noise" to the decision-making process of the trees. This makes it impossible for a hacker to figure out exactly what data a specific person contributed.

4. The Catch: The Foggy Glass

Here is the main discovery of the paper: Privacy comes with a cost to clarity.

When you add that "noise" (Differential Privacy) to protect the data, it's like putting a foggy filter over the Decision Tree.

Without Privacy: The tree is crystal clear. It knows, "Humidity is the #1 reason for rain."
With High Privacy (The "Fog"): The tree gets confused. It might say, "Well, humidity is important, but so is the barometer, and maybe the wind, and maybe the time of day." The importance of features gets "spread out" like butter on toast.
The Trade-off: The more you protect privacy (by adding more noise), the harder it is to explain exactly why the model made a decision. The "fog" makes the explanation less sharp.

5. The Experiment: Testing the Fog

The researchers tested this on a dataset about home energy usage (predicting how much electricity a house uses). They compared three things:

Standard AI (FedAVG): A complex, hard-to-explain model.
Clean Trees: A simple, explainable tree with no privacy protection.
FEXT-DP: Their new "Foggy Tree" with privacy protection.

What they found:

Performance: The "Foggy Tree" (FEXT-DP) was actually better at predicting energy use than the complex AI, even with the privacy noise!
Explainability: As they increased the privacy (made the fog thicker), the tree's ability to point out the single most important factor got weaker. The features became a blur of "all of them matter a little bit" rather than "this one matters a lot."

The Bottom Line

This paper presents a new tool called FEXT-DP. It's a way to build smart, simple, and transparent AI models that work across many different computers without sharing private data.

However, it teaches us an important lesson: Privacy and Clarity are a balancing act. If you want maximum privacy, your model's explanation might get a little "fuzzy." If you want a perfectly clear explanation, you might have to accept slightly less privacy protection. The authors show that you can find a "sweet spot" where you get strong privacy without completely losing the ability to understand how the AI thinks.

1. Problem Statement

Modern Machine Learning (ML) systems face a dual challenge: ensuring data privacy and providing explainability (XAI).

Privacy: While Federated Learning (FL) allows models to be trained on decentralized data without transferring raw data to a central server, it remains vulnerable to attacks (e.g., Gradient Inversion, Membership Attacks) that can reconstruct private user data from shared model parameters.
Explainability: Complex models like Neural Networks (NN) are often "black boxes." To meet regulatory and industry demands for transparency, models need to be interpretable, typically requiring simpler architectures and fewer features.
The Conflict: Adding Differential Privacy (DP) to FL enhances privacy by injecting noise, but this noise degrades model accuracy and, crucially, distorts the internal logic of the model, thereby harming its explainability.

The paper addresses the gap in understanding how DP specifically impacts the explainability of tree-based FL models and proposes a solution that balances these competing goals.

2. Methodology: FEXT-DP

The authors propose Federated EXplainable Trees with Differential Privacy (FEXT-DP), a framework designed to achieve high privacy and explainability simultaneously.

Core Architecture

Base Model: Unlike NN-based FL (e.g., FedAvg), FEXT-DP uses Decision Trees (DT). DTs are chosen for their lightweight nature and inherent interpretability (decision rules are trackable).
Federated Workflow:
1. Local Training: Clients train local Decision Trees on their private datasets.
2. Server Aggregation: The FL server receives trees and filters them based on an accuracy threshold ( $K$ ). Trees failing this threshold are discarded to prevent poisoning attacks.
3. Global Update: The server aggregates the valid trees and sends the updated global model back to clients.
4. Selection Logic: Clients compare the global model's Mean Squared Error (MSE) against their local model. If the global model is worse, the client retains its local model; otherwise, it adopts the global one.

Differential Privacy Integration

The core innovation lies in Algorithm 1, which modifies the standard Decision Tree splitting process to incorporate DP:

Standard Approach: A tree deterministically selects the feature-threshold split with the highest Information Gain.
FEXT-DP Approach: Instead of a deterministic choice, the algorithm uses the Exponential Mechanism.
- It calculates the Information Gain for all possible splits.
- It assigns a probability to each split based on the gain and the privacy budget ( $\epsilon$ ).
- A split is selected probabilistically ("roulette wheel" selection).
Effect: This ensures that the selected split is not directly tied to any single data point, satisfying DP guarantees and protecting against Membership Attacks.

3. Key Contributions

Novel Framework: Introduction of FEXT-DP, a Federated Learning solution that combines Decision Trees (for explainability) with Differential Privacy (for privacy).
Privacy-Explainability Trade-off Analysis: A comprehensive study quantifying how varying levels of DP ( $\epsilon$ ) impact the explainability of the model.
Algorithmic Innovation: A specific DP-compliant method for finding optimal splits in Decision Trees using the exponential mechanism within a federated setting.

4. Experimental Evaluation

The authors evaluated FEXT-DP using the Appliance Energy Prediction Data (AEPD) dataset (19,735 entries regarding energy consumption in Belgian homes).

Setup: 20 FL clients, 1 server.
Comparators:
- FEXT-DP with varying privacy budgets ( $\epsilon = 1, 0.1, 0.05, 0.01$ ).
- Federated Decision Trees without DP.
- FedAvg: A traditional Neural Network-based FL approach (6 layers, 600 neurons).
Metrics:
- Performance: Mean Squared Error (MSE) and Pearson Correlation.
- Explainability: Mean Decrease in Impurity (MDI) and SHAP (SHapley Additive exPlanations) values.

Key Results

Performance (MSE & Correlation):
- FEXT-DP significantly outperformed FedAvg. At round 1, FEXT-DP had 35% lower MSE than FedAvg.
- Even with the strictest privacy ( $\epsilon = 0.01$ ), the MSE increase was minimal (only 1.18% higher than non-private Decision Trees).
- FedAvg failed to converge as effectively as the tree-based approaches over 40 rounds.
Explainability Impact (MDI):
- Feature Importance Shift: Applying DP caused a significant redistribution of feature importance.
- Strict Privacy ( $\epsilon = 0.01$ ): The MDI values for all features decreased and became more uniform. This "flattening" makes it harder to identify the single most dominant feature, slightly reducing the clarity of the model's logic.
- Non-Strict Privacy: Higher $\epsilon$ values preserved the original feature ranking (e.g., Room Humidity "RH 1" remained the most important feature).
Explainability Impact (SHAP):
- SHAP analysis revealed that while the order of feature contributions shifted slightly under strict DP, the internal feature operations remained largely consistent.
- The model's ability to explain why a prediction was made was preserved, though the specific ranking of feature dominance changed.

5. Significance and Conclusion

The paper demonstrates that it is possible to build Federated Learning systems that are both private and explainable, challenging the notion that privacy mechanisms must completely sacrifice model interpretability.

Practicality: FEXT-DP offers a lightweight alternative to heavy Neural Network FL systems, achieving better convergence speeds and lower initial error rates.
Trade-off Management: The study provides empirical evidence that a strict privacy budget ( $\epsilon = 0.01$ ) introduces only minor perturbations to explainability. While the ranking of feature importance shifts, the model remains interpretable.
Future Work: The authors plan to integrate pruning algorithms, client selection mechanisms, and further analyze system metrics like latency and network traffic to optimize scalability.

In summary, FEXT-DP successfully bridges the gap between regulatory compliance (privacy) and the need for transparent AI (explainability), proving that Decision Trees enhanced with Differential Privacy are a viable, high-performance solution for sensitive, distributed data environments.