Tackling Federated Unlearning as a Parameter Estimation Problem

Technical Summary: Tackling Federated Unlearning as a Parameter Estimation Problem

Problem Statement
The increasing prevalence of privacy regulations necessitates the ability to erase specific data from trained deep learning models, a process known as machine unlearning. This challenge is significantly amplified in the context of Federated Learning (FL). In FL, data remains decentralized on client devices, making traditional unlearning strategies—such as full model retraining or coordinated global updates—often infeasible due to high computational costs and communication overhead. Furthermore, the server lacks direct access to raw client data, complicating the identification of which parameters are most influenced by the data to be forgotten.

Methodology
This work proposes an efficient Federated Unlearning framework grounded in information theory. The core innovation lies in modeling the data leakage associated with specific samples as a parameter estimation problem. Rather than attempting to invert the training process or retrain the entire model, the method employs the following technical approach:

1. Second-Order Sensitivity Analysis: The framework utilizes second-order Hessian information to mathematically identify the specific model parameters that are most sensitive to the data designated for removal.
2. Selective Reset: Based on this sensitivity analysis, the method selectively resets only the identified critical parameters, leaving the remainder of the model intact.
3. Minimal Federated Retraining: Following the selective reset, the framework performs a minimal federated retraining phase to restore model convergence.
4. Model-Agnostic Design: The approach is designed to be model-agnostic and supports both categorical unlearning (removing specific classes) and client-level unlearning (removing all data from a specific client). Crucially, it operates without requiring the server to access raw client data after the initial information aggregation phase.

Key Contributions

• Theoretical Formulation: The paper introduces a novel perspective by framing federated unlearning as a parameter estimation problem driven by information theory.
• Efficient Algorithm: It provides a practical algorithm that leverages Hessian-based sensitivity to minimize the scope of model modification, thereby reducing computational and communication costs compared to full retraining.
• Dual Capability: The framework effectively handles both class-level and client-level unlearning scenarios within the FL setting.
• Security Extension: The method is demonstrated to function as a defense mechanism, capable of neutralizing malicious triggers in targeted backdoor attack scenarios.

Experimental Results
Evaluations conducted on benchmark datasets yield the following findings:

• Privacy: The framework achieves strong privacy guarantees. Membership Inference Attack (MIA) success rates drop to levels near random guessing, and categorical knowledge regarding the forgotten data is effectively erased.
• Performance: The unlearned models maintain high utility, achieving a Normalized Accuracy against fully re-trained benchmarks of approximately 0.9.
• Efficiency: The approach demonstrates increased efficiency over complete retraining, validating the premise that selective parameter resetting is a viable alternative to full model regeneration.
• Robustness: In targeted backdoor attack scenarios, the framework successfully restores model integrity by neutralizing the malicious trigger.

Significance and Claims
The paper positions this work as a practical solution for the "right to be forgotten" within Federated Learning environments. By avoiding the prohibitive costs of full retraining and the privacy risks of centralizing raw data, the proposed framework offers a scalable and efficient pathway for data forgetting. The authors claim that this approach balances the competing demands of regulatory compliance, model utility, and system efficiency, providing a robust mechanism for maintaining model integrity even in the face of adversarial backdoor attacks.