💻 computer science

Closed-Loop Fairness and Privacy Optimization for Federated Text Classification under Non-IID Data

This paper proposes FedMOJO, a closed-loop collaborative framework that integrates low-rank representation learning, differential privacy, and fairness-privacy optimization to effectively balance utility, group fairness, and privacy preservation in federated text classification under non-IID data.

Original authors: Lihong Guo, Mingkai Dai, Luogang Zhang, Chao Ma

Published 2026-07-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Lihong Guo, Mingkai Dai, Luogang Zhang, Chao Ma

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a group of neighbors who all want to build a super-smart weather prediction machine together. However, they have a few strict rules:

No Sharing Secrets: They cannot share their personal diaries (raw data) with each other.
Fairness: The machine must predict the weather equally well for everyone, whether they live in the city or the countryside, and not be biased against any specific group.
Privacy: They need to make sure no one can peek into their diaries by looking at the machine's blueprints (model updates).

The problem is that their local weather patterns are all different (some days are rainy in one town, sunny in another). If they just try to combine their notes, the final machine might become confused, unfair, or accidentally leak secrets.

This paper introduces a new team-building strategy called FedMOJO to solve this. Here is how it works, using simple analogies:

1. The "Low-Rank" Compression (The Sketch Artist)

Usually, when neighbors send their notes to the central hub, they send the whole diary page. This is bulky and risky.
FedMOJO asks each neighbor to act like a sketch artist. Instead of sending the whole page, they only send a simplified "sketch" (low-rank representation) that captures the essential weather patterns but leaves out the messy, sensitive details (like specific names or addresses).

Why it helps: It reduces the "noise" and strips away sensitive info before it even leaves the house, making the data safer and easier to process.

2. The "Dual-Stream" Team (The Fairness & Privacy Duo)

Inside each neighbor's house, the learning process is split into two parallel teams working together:

Team Fairness: Focuses on making sure the sketch treats all groups equally.
Team Privacy: Focuses on adding a little bit of "static" or "fog" (mathematical noise) to the sketch so no one can reverse-engineer the original diary.
The Gatekeeper: A smart manager (a "gated mechanism") decides how much weight to give to each team's sketch for every single prediction. This ensures the final result is both fair and private without ruining the accuracy.

3. The "Closed-Loop" Feedback (The Conductor)

In old methods, the central hub just collected sketches, averaged them, and sent them back. It was a one-way street.
FedMOJO acts like a symphony conductor.

Listening: The hub listens to the "stats" neighbors send back (not the data itself, but reports like "Group A is being treated poorly" or "We are running out of our privacy budget").
Adjusting: Based on this feedback, the conductor changes the rules for the next round. If fairness is slipping, the conductor tells everyone to focus more on fairness. If privacy is getting too weak, the conductor tells them to add more "fog."
The Loop: This creates a continuous cycle where the global rules adapt to the local reality, ensuring the whole group stays balanced.

4. The "Robust Aggregation" (The Filter)

Sometimes, a neighbor might send a sketch that is way off because their local weather is weird (Non-IID data).
FedMOJO has a quality filter. It checks if a neighbor's sketch is drifting too far from the group norm. If it is, the filter gives that sketch less weight so it doesn't ruin the final model. This keeps the team stable even when everyone's local conditions are different.

The Results: What Did They Find?

The authors tested this method on three different "weather scenarios" (datasets):

Twitter Sentiment: Analyzing feelings in tweets (checking for bias against different dialects).
Bias in Bios: Predicting jobs from short bios (checking for gender bias).
Adult Income: Predicting income levels (checking for gender bias).

The Verdict:
Compared to other methods, FedMOJO was the "Goldilocks" solution.

Old methods were either accurate but unfair, or fair but inaccurate.
FedMOJO managed to be accurate (predicting well), fair (treating groups equally), and private (keeping secrets safe) all at the same time.
Specifically, it improved accuracy by up to 4.89% compared to the next best privacy-focused method, while simultaneously reducing unfairness and privacy leaks.

In a Nutshell

FedMOJO is a new way for computers to learn together without sharing private data. It uses sketches instead of full photos, splits the work between fairness and privacy teams, and uses a smart conductor to constantly adjust the rules. This ensures the final result is smart, fair, and safe, even when everyone starts with very different information.

Technical Summary: Closed-Loop Fairness and Privacy Optimization for Federated Text Classification under Non-IID Data

Problem Statement
Federated text classification under non-independent and identically distributed (non-IID) data faces three coupled challenges: utility degradation, group fairness mismatch, and privacy leakage. While federated learning (FL) allows collaborative training without centralizing raw data, it does not inherently guarantee fairness or privacy. Existing approaches often treat fairness and privacy as isolated regularizers or rely on static trade-off weights, which fail to address the dynamic interplay between these objectives as they evolve across communication rounds. Furthermore, statistical heterogeneity, label skew, and unstable client participation in text classification can amplify fairness mismatches and privacy risks, particularly when differential privacy (DP) noise weakens learning signals for minority groups.

Methodology: FedMOJO
The paper proposes FedMOJO, a closed-loop collaborative optimization framework designed to unify low-rank representation learning, differentially private local training, and multi-objective coordination within a single pipeline. The framework operates through a cross-round closed loop between client-side local execution and server-side global coordination.

Client-Side Local Optimization:
- Dual-Stream Representation: Clients employ a dual-stream architecture consisting of a fairness-enhanced stream and a privacy-preserving stream. These are adaptively fused via a gated mechanism to balance task-relevant semantics with sensitive information suppression.
- Low-Rank Adaptation: To reduce redundancy and suppress sensitive interference, clients utilize low-rank adapters ( $A_k, B_k$ ) for parameter updates. This compresses the representation space and facilitates efficient aggregation.
- Differentially Private Training: Local optimization utilizes DP-SGD with gradient clipping and Gaussian noise injection. Privacy accounting is performed using Rényi Differential Privacy (RDP).
- Uploadable Statistics: Clients compute and upload perturbed fairness statistics (e.g., group-wise true positive rates) and local state summaries (e.g., loss changes, gate stability, privacy budget consumption) to the server without exposing raw data.
- Objective Function: The local loss combines classification loss, fairness regularization (based on TPR disparity), low-rank structural loss, consistency stabilization between streams, and a proximal term to mitigate local drift.
Server-Side Global Coordination:
- State-Driven Control: The server aggregates client packets to estimate a global state vector comprising fairness gaps, update drift, and privacy levels. Based on this state, a control generator dynamically updates hyperparameters (fairness weights, noise scales, regularization strengths) for the next round.
- Low-Rank Structural Alignment: The server performs low-rank structural consistency maintenance, aligning local low-rank factors with global reference factors before aggregation to ensure stability.
- Robust Aggregation: To handle non-IID data and system heterogeneity, the server employs a robust aggregation mechanism that assigns reliability scores to clients based on update drift and participation frequency, down-weighting unstable or divergent updates.
- Fairness-State Fusion: The server reconstructs a global fairness state from uploaded statistics, enabling global regulation even when individual clients have limited group coverage.

Key Contributions

Unified Framework: The proposal of a closed-loop framework that integrates low-rank representation learning, DP local training, and multi-objective coordination, moving beyond loose combinations of fairness and privacy constraints.
Client-Server Cooperative Mechanism: A novel mechanism for global fairness-state control utilizing uploadable fairness statistics, low-rank structural alignment, and cross-round control-signal generation, transitioning from isolated local optimization to globally coordinated learning.
Empirical Validation: Demonstration of a robust balance among classification utility, group fairness, and privacy preservation across multiple heterogeneous federated settings.

Experimental Results
Experiments were conducted on three datasets: Twitter Sentiment (binary sentiment), Bias in Bios (28-class occupation prediction), and Adult Income (binary income prediction). The proposed method was compared against FedAvg, FairFed, DPFed, and DPFairFed.

Performance Trade-offs: FedMOJO achieved a superior trade-off compared to baselines. On the Twitter Sentiment dataset, it improved accuracy by 4.89 percentage points over DPFairFed (73.32% vs. 68.43%) while simultaneously reducing the fairness gap (TPR-gap) to 4.95 and lowering privacy leakage.
Consistency: Across all datasets, FedMOJO consistently reduced fairness disparity and privacy leakage while maintaining or improving classification accuracy and Minimum Description Length (MDL) metrics.
Ablation Study: Removing key components (low-rank representation, global fairness-statistics fusion, dynamic regulation, or robust aggregation) resulted in noticeable degradation in at least one of the three objectives (utility, fairness, or privacy), confirming that the performance gain stems from the collaboration of all modules within the closed-loop framework.

Significance and Claims
The paper claims that FedMOJO provides a promising solution for privacy-sensitive federated learning scenarios where raw data cannot be shared. The authors assert that the closed-loop client-server coordination effectively balances classification performance, group fairness, and privacy preservation. Specifically, the work demonstrates that optimizing fairness and privacy in isolation is insufficient for stable three-objective coordination in heterogeneous federated environments. Instead, a jointly structured approach involving low-rank representation compression, global state fusion, and dynamic control generation is necessary to mitigate the intrinsic conflicts between utility, fairness, and privacy under non-IID conditions.

Limitations and Future Work
The authors note that the current study focuses on federated classification tasks and has not yet been validated on multimodal data or large-scale generative language models. Future work aims to extend the framework to multimodal scenarios, investigate complex sensitive-attribute structures (including intersectional attributes), and explore more communication-efficient interaction schemes.