Technical Summary: CRAFT (Conflict-Resolved Aggregation for Federated Training)

1. Problem Statement

Federated Learning (FL) enables collaborative model training across decentralized clients without sharing raw data. However, a fundamental bottleneck arises from data heterogeneity (non-IID distributions), which causes divergent local updates. Conventional aggregation methods, such as FedAvg, compute weighted averages of these updates. The paper argues that naive averaging can produce a global update direction that improves the global objective while conflicting with specific clients (i.e., the inner product between the global update and a client's local update is negative). This conflict leads to performance degradation for those specific clients, resulting in high disparities in per-client accuracy and poor fairness, even if the mean global accuracy is high.

Existing approaches attempt to mitigate this through regularization (FedProx), normalization (FedNova), fairness-aware objectives (AFL, q-FedAvg), or personalized models. Conflict-aware methods (FedFV, FedLF, ConFIG) have emerged to directly manipulate update geometry, but they often rely on heuristic rules or lack the integration of temporal information to guide resolution.

2. Methodology: The CRAFT Framework

CRAFT reframes the aggregation problem not as a weighted average, but as a geometric correction problem. The core objective is to find a global update vector $g$ that is as close as possible to a stable reference direction while satisfying strict conflict-free alignment constraints.

2.1 Core Formulation

Instead of reconstructing an update from scratch (as in zero-prior methods) or simply smoothing (as in momentum methods), CRAFT solves a reference-anchored constrained least-squares problem:

$$\min_{g \in \mathbb{R}^d} \frac{1}{2} \|g - \hat{g}_t\|^2 \quad \text{s.t.} \quad U_t g = \rho_t$$

Where:

• $\hat{g}_t$ is the reference direction, defined as the normalized global update from the previous round ($\hat{g}_t = U(g_{t-1})$). For the first round, $\hat{g}_0 = 0$.
• $U_t$ is the matrix of normalized local updates from active clients ($u_i = g_i / (\|g_i\| + \epsilon)$).
• $\rho_t$ is the vector of positive alignment targets, typically derived from client data proportions ($\rho_i = |D_i| / \sum |D_j|$).
• The constraint $U_t g = \rho_t$ ensures that the global update $g$ has a positive inner product with every participating client's normalized update, thereby guaranteeing a conflict-free direction.

2.2 Closed-Form Solution

The problem is underdetermined (model dimension $d \gg$ number of clients $m$). CRAFT derives a closed-form solution using the Moore-Penrose pseudoinverse, avoiding the computational overhead of iterative solvers or quadratic programming. The update is computed as a correction $\Delta_t$ to the reference:

$$g_t = \hat{g}_t + \Delta_t, \quad \text{where} \quad \Delta_t = U_t^\dagger (\rho_t - U_t \hat{g}_t)$$

This formulation ensures that if the reference direction already satisfies the constraints, no correction is applied. Otherwise, the smallest Euclidean correction is added to enforce alignment.

2.3 Layer-Wise Adaptation

Recognizing that deep neural networks exhibit layer-specific optimization dynamics (e.g., shallow layers may suffer from input distribution shifts while deeper layers suffer from label skew), CRAFT implements a layer-wise adaptation. The optimization problem is decomposed into $Q$ independent subproblems for each layer $q$, solving the correction separately:
$$g^q_t = \hat{g}^q_t + (U^q_t)^\dagger (\rho_t - U^q_t \hat{g}^q_t)$$
This prevents severe conflicts in one layer from suppressing useful components in others.

2.4 Theoretical Properties

The paper provides a theoretical analysis showing that CRAFT promotes a common-descent structure. By enforcing non-negative alignment with all client updates, the method mitigates conflicts. The authors establish finite-time convergence guarantees for non-convex objectives under standard smoothness and bounded gradient assumptions, demonstrating a convergence rate of $O(1/\sqrt{T})$ comparable to standard non-convex optimization, provided the local learning rate decays appropriately.

3. Key Contributions

1. Problem Reformulation: The authors propose reformulating federated aggregation as a reference-anchored constrained least-squares problem rather than a weighted averaging rule. This directly enforces positive alignment between the global update and every client.
2. Efficient Closed-Form Solution: Unlike iterative conflict-resolution methods, CRAFT derives an explicit solution using the Moore-Penrose correction, keeping server-side computation lightweight ($O(dm^2 + Qm^3)$).
3. Momentum-Like Reference: The use of the previous round's global update as a reference direction allows CRAFT to preserve temporal consistency and stabilize updates against sampling noise, distinguishing it from methods that project toward a zero vector.
4. Layer-Wise Granularity: The framework addresses conflicts at the granularity of model layers, making it suitable for deep neural networks where global vector constraints might be too coarse.
5. Plug-in Extensibility: CRAFT is designed as a generic aggregation operator that can replace weighted averaging in existing personalized FL pipelines (e.g., Ditto) without modifying local adaptation mechanisms.

4. Experimental Results

Extensive experiments were conducted on heterogeneous benchmarks (FEMNIST, CIFAR-10, CIFAR-100) using various models (MLP, CNN, ResNet-20/56/110) and client populations (up to 1000 clients).

• Global Performance: CRAFT consistently outperformed state-of-the-art baselines in mean accuracy. On CIFAR-10 with ResNet-20, it achieved a 53.5% relative improvement over ConFIG and a 7.8% improvement over FedAvgM on FEMNIST.
• Fairness and Disparity: CRAFT significantly reduced the standard deviation of per-client accuracy. It improved the performance of "tail" clients (worst 10%) without sacrificing "head" clients. For instance, on CIFAR-10, it improved the worst-10% accuracy by 221.5% over FedLF.
• Convergence: CRAFT demonstrated faster convergence and higher final accuracy, particularly on deeper networks where other methods often saturated early or suffered from client drift.
• Personalized FL Integration: When integrated into the Ditto framework, CRAFT improved mean test accuracy by 19.7% on CIFAR-10 and 98.8% on CIFAR-100 compared to vanilla Ditto, demonstrating its ability to enhance personalized pipelines.

5. Significance and Claims

The paper claims that CRAFT provides a new perspective on aggregation design, shifting from heuristic conflict resolution to a geometrically principled, constraint-based approach. Its significance lies in:

• Mitigating Heterogeneity: It directly addresses the root cause of fairness issues in FL (conflicting updates) rather than just adjusting objectives or regularizing local training.
• Balancing Accuracy and Fairness: It achieves high mean accuracy while simultaneously reducing performance disparity, a dual goal often difficult to satisfy with existing methods.
• Practicality: The method is computationally efficient, requires no additional communication, and is easily extensible to existing personalized FL frameworks.

The authors acknowledge limitations, noting that the current reference direction is a simple momentum-like prior and that the alignment targets are based on data proportions. They suggest future work could explore adaptive references or targets based on training progress and fairness requirements. However, the current work establishes CRAFT as a robust, conflict-free aggregation mechanism for heterogeneous federated training.