🤖 machine learning

Influence-Oriented Personalized Federated Learning

This paper proposes FedC^2I, an influence-oriented personalized federated learning framework that quantifies client-level and class-level influences via dedicated vectors and matrices to enable adaptive, personalized parameter aggregation and improve performance under non-IID settings.

Original authors: Yue Tan, Guodong Long, Jing Jiang, Chengqi Zhang

Published 2026-09-10

📖 5 min read🧠 Deep dive

CC0 1.0

Original authors: Yue Tan, Guodong Long, Jing Jiang, Chengqi Zhang

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, machines learn by finding patterns in vast amounts of data. For decades, this learning happened in one place: a massive computer center where all the information was gathered. But as our digital lives become more personal and our data more sensitive, a new approach has emerged called federated learning. Imagine a group of people who want to solve a puzzle together but cannot show their individual pieces to anyone else. Instead of bringing the pieces to a central table, they keep them in their own hands. They pass only the rules they have learned from their pieces to a coordinator, who mixes these rules together and sends them back. This allows the group to build a smarter, shared understanding without ever exposing their private information. However, a problem arises when the pieces are too different. If one person is looking at handwritten numbers on a whiteboard and another is looking at digital text on a screen, the rules they learn might clash, making the final shared model confusing and less effective for everyone.

Researchers have long tried to fix this by averaging the rules from all participants, assuming that a middle ground works best for everyone. But this approach often fails when the data is highly varied, leaving some participants with a model that doesn't fit their specific needs. A team of scientists has now proposed a different way to handle this collaboration, one that recognizes that not all partners are equally helpful to every other partner. They developed a system called FedC2I, which allows each participant to decide, based on their own specific situation, which other participants are most worth listening to. Instead of forcing everyone to follow the same set of instructions, this new method lets each client build a personalized version of the shared model by selectively borrowing the most useful insights from others.

The core of this new approach lies in measuring "influence." In a standard setup, every participant contributes to the final model with a fixed weight, often based simply on how much data they have. The researchers argue that this is like asking everyone in a room to speak at the same volume, regardless of whether their voice is actually helpful to the person trying to listen. In their system, each client performs a small experiment before accepting new rules. They temporarily remove the contribution of one other participant and see how much their own performance drops. If removing a specific partner causes a significant drop in accuracy, that partner is deemed highly influential and is given more weight in the next round of learning. If removing a partner makes little difference, their influence is reduced. This process happens for every single participant, creating a unique map of who helps whom.

To make this even more precise, the researchers realized that influence isn't just about the whole person; it is also about specific topics. In a task like recognizing digits, one participant might be excellent at identifying the number seven but terrible at the number three, while another might be the opposite. The new system separates the learning into two parts: the general ability to see shapes and features, and the specific rules for labeling them. It measures influence separately for these two layers. For the general features, it looks at how much one client helps another understand the overall style of the data. For the specific labels, it checks how much help a client provides for each individual category. This allows a client to learn the general shape of a "7" from one neighbor and the specific way to draw a "7" from a different neighbor, combining the best of both worlds without the confusion of conflicting rules.

The team tested this method on two different sets of data involving handwritten digits and images of everyday objects from various domains. In these tests, the data was intentionally made difficult, with different fonts, backgrounds, and lighting conditions to mimic the messy reality of the real world. The results showed that this influence-oriented approach consistently outperformed existing methods. On the digit recognition task, the new system improved the average accuracy by a noticeable margin compared to the best previous techniques. On the object recognition task, it also achieved higher scores across the board. The researchers found that the system was particularly good at identifying which clients shared similar characteristics, such as the same handwriting style or background color, and prioritizing those connections.

A key finding was that the influence between clients is not static; it changes as the learning progresses. In the early stages, the connections are somewhat fluid, but as the models learn more, the system naturally settles into a stable pattern where clients consistently rely on the most relevant partners. The researchers also tested how sensitive the system is to the settings used for these measurements. They found that a specific setting worked best for both datasets, allowing the system to be selective enough to ignore unhelpful noise while still remaining open enough to learn from diverse sources. The experiments confirmed that by measuring influence at both the client level and the class level, the system could create a more robust and personalized learning environment.

This work suggests that the future of collaborative machine learning may not be about finding a single perfect model for everyone, but about creating flexible systems that adapt to individual needs. By quantifying how much one participant helps another, and by distinguishing between general knowledge and specific expertise, the researchers have shown that machines can learn together more effectively without compromising their privacy. The method does not require sharing raw data or complex statistics; it relies entirely on how the models perform when tested against each other. This makes it a practical solution for scenarios where data is diverse and privacy is paramount, offering a way for artificial intelligence to grow smarter through cooperation that respects the unique context of every participant.

Technical Summary: Influence-Oriented Personalized Federated Learning (FedC2I)

1. Problem Statement

Federated Learning (FL) enables collaborative model training across distributed clients without sharing private data. However, a core challenge in FL is statistical heterogeneity (the non-IID problem), where clients possess data from different domains with varying feature attributes. Traditional FL methods, such as FedAvg, rely on fixed weighting schemes (typically proportional to dataset size) for parameter aggregation. This approach neglects the mutual influence among clients.

In real-world scenarios, clients with similar preferences, backgrounds, or data characteristics (e.g., similar font styles or handwriting habits in digit recognition) can provide more useful knowledge to one another. Conversely, aggregating models from clients with divergent data distributions can lead to negative transfer. Existing personalized FL methods often focus on global model robustness or use centralized reweighting based on loss/gradient similarity, but they fail to explicitly quantify and leverage client-level and class-level influence for adaptive, personalized aggregation. Specifically, they do not address how a client can selectively acquire knowledge from others based on its specific data characteristics and learning needs.

2. Methodology: FedC2I

The authors propose FedC2I (Influence-Oriented Federated Learning), a framework that explicitly quantifies and utilizes Client-level Influence and Class-level Influence to guide parameter aggregation. The core intuition is that a client's usefulness should be evaluated from the perspective of each target client, rather than by a globally shared criterion.

A. Model Decomposition

FedC2I decouples the local model $w_m$ into two components:

Feature Representation Layers ( $\theta_m$ ): Capture transferable patterns (shape, texture, style) shared across domains.
Classifier ( $\phi_m$ ): Maps representations to class predictions, containing class-specific decision boundaries.

B. Influence Measurement via Leave-One-Out

To quantify influence without accessing raw data, FedC2I employs a leave-one-out principle evaluated on local mini-batches.

Client-Level Influence ( $\lambda_m$ ):
- Goal: Measure how much client $i$ contributes to the overall representation learning of client $m$ .
- Mechanism: Client $m$ computes the loss $l^{-i}_m$ on a local batch using an aggregated representation model $\theta_{-i}$ (where client $i$ 's parameters are excluded).
- Calculation: A large $l^{-i}_m$ implies client $i$ is highly influential. The influence vector $\lambda_m = [\lambda^1_m, \dots, \lambda^M_m]$ is computed as:
  $\lambda^i_m = \frac{[l^{-i}_m]^\gamma}{\sum_{j=1}^M [l^{-j}_m]^\gamma}$
  where $\gamma$ is a hyperparameter controlling sensitivity.
- Aggregation: The personalized feature representation $\bar{\theta}_m$ is formed by weighting all clients' $\theta_i$ by $\lambda^i_m$ .
Class-Level Influence ( $\Lambda_m$ ):
- Goal: Capture fine-grained disagreements or agreements on specific classes, as clients may share domain features but differ in class-specific decision boundaries (e.g., different interpretations of the digit "7").
- Mechanism: For each class $c$ , client $m$ evaluates the loss $l^{-i, -c}_m$ when the class-specific weight vector $\phi_{m,c}$ is replaced by an aggregated version excluding client $i$ ( $\phi^{-i}_{m,c}$ ).
- Calculation: An influence matrix $\Lambda_m \in \mathbb{R}^{M \times C}$ is constructed where element $\Lambda^{i,c}_m$ represents the influence of client $i$ on class $c$ for client $m$ :
  $\Lambda^{i,c}_m = \frac{[l^{-i, -c}_m]^\gamma}{\sum_{j=1}^M [l^{-j, -c}_m]^\gamma}$
- Aggregation: The personalized classifier $\bar{\phi}_m$ is constructed class-by-class. For class $c$ , the weights are aggregated using the $c$ -th column of $\Lambda_m$ .

C. Training Procedure

In each communication round:

The server broadcasts the latest local parameters from all clients.
Each client $m$ performs local influence estimation using a sampled mini-batch to compute $\lambda_m$ and $\Lambda_m$ .
Client $m$ constructs a personalized initialization $\{\bar{\theta}_m, \bar{\phi}_m\}$ via the weighted aggregation described above.
Client $m$ performs local training (SGD) on its private dataset starting from this personalized initialization.
Updated parameters are sent back to the server.

3. Key Contributions

The paper outlines three primary contributions:

Novel Influence Quantification: It is the first work to explicitly explore and quantify both client-level and class-level influence within an FL system, moving beyond fixed or globally shared aggregation weights.
FedC2I Framework: The proposal of a concrete, feasible framework that uses influence vectors and matrices to enable clients to personalize the model aggregation procedure. This allows for selective knowledge acquisition based on data characteristics.
Empirical Validation: Extensive experiments on benchmark datasets (Digit-5 and Office-10) demonstrate the method's superiority over baselines in terms of effectiveness, robustness, and interpretability under non-IID settings.

4. Experimental Results

The authors evaluated FedC2I against eight baselines (including Local, FedAvg, FedProx, FedRep, FedRoD, FedProto, FedCALM, and FedIC) on two heterogeneous datasets:

Digit-5: Five domains (MNIST, SVHN, USPS, SynthDigits, MNIST-M) with 10 digit classes.
Office-10: Four domains (Amazon, Caltech, DSLR, WebCam) with 10 overlapping categories.

Key Findings:

Performance: FedC2I outperformed the best baseline methods in at least half of the clients/domains.
Accuracy Gains: It achieved an average test accuracy improvement of 0.78% to 2.62% over the best baselines.
- On Digit-5, FedC2I achieved 85.50% accuracy compared to the next best (FedCALM at 83.95%).
- On Office-10, FedC2I achieved 66.70% accuracy compared to the next best (FedCALM at 65.40%).
Ablation Studies: Removing either the client-level influence vector ( $\lambda$ ) or the class-level influence matrix ( $\Lambda$ ) resulted in significant performance drops, confirming that both components are necessary. Specifically, client-level influence provided the primary boost, while class-level influence refined performance further.
Hyperparameter Sensitivity: The sensitivity parameter $\gamma$ was tuned, with $\gamma=5$ yielding optimal results. The authors note that influence relationships are dynamic, evolving from smooth distributions in early rounds to more stable, selective relationships as training progresses.

5. Significance and Claims

The paper claims that FedC2I addresses the gap in existing FL methods by shifting the aggregation paradigm from server-side fixed weighting to client-side adaptive, influence-aware aggregation.

Privacy Preservation: The method preserves the standard FL privacy protocol. Influence measurements are derived solely from local loss evaluations on private mini-batches; no raw data, gradients, or distribution statistics are transmitted.
Decentralization Potential: By moving the aggregation logic to the client side, the framework reduces the server's burden and offers a pathway toward decentralized FL systems.
Interpretability: The influence vectors and matrices provide an interpretable mechanism to understand which clients contribute to specific features or classes, offering insights into cross-client relationships that are otherwise opaque in standard FL.

The authors conclude that by explicitly modeling inter-client influence, FedC2I allows clients to efficiently and selectively acquire useful knowledge, significantly improving local performance in heterogeneous environments without compromising data privacy.