🤖 machine learning

Closing the Alignment-Maturity Gap in Federated Prototype Learning

This paper introduces FedSAP, a federated learning framework that addresses the alignment-maturity gap in prototype-based methods by combining a deterministic alignment curriculum with a geometry-driven proxy separation loss to stabilize representation learning and significantly improve performance under heterogeneous data conditions without adding communication overhead.

Original authors: Mario Casado-Diez, Alejandro Dopico-Castro, Verónica Bolón-Canedo, Bertha Guijarro-Berdiñas

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mario Casado-Diez, Alejandro Dopico-Castro, Verónica Bolón-Canedo, Bertha Guijarro-Berdiñas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a group of students (the "clients") trying to learn how to recognize different animals, but they are all in separate classrooms and cannot share their notebooks or see each other's work. They can only send a single, tiny summary note to a teacher (the "server") once in a while. This is the world of Federated Learning.

The problem is that each student has a very different set of pictures. One student only has pictures of cats and dogs; another has only birds and fish. If they try to learn together too quickly, they get confused.

The Problem: The "Rush to Judgment"

In previous methods, the teacher would immediately try to force all students to agree on what a "cat" looks like, right from the first day of class.

The paper calls this the "Alignment-Maturity Gap." Here is the analogy:

The Immature Student: At the start of the year, a student's understanding of a cat is messy and vague. They might think a cat is just "a furry thing."
The Noisy Summary: When this student sends their messy idea to the teacher, the teacher combines it with other messy ideas from other students to create a "Global Cat Definition."
The Pressure Cooker: The teacher then immediately yells, "You must change your drawing to match this Global Cat Definition!"
The Result: Because the student's drawing was already shaky, this sudden, forceful correction doesn't help them learn. Instead, it scrambles their brain, making them forget how to draw a cat at all. They end up with a messy, confused drawing that looks like nothing.

The paper argues that forcing students to align with the group before they have learned the basics is counterproductive. It creates a "gradient pressure" that is too strong too soon, destroying the local learning process.

The Solution: FedSAP (The Patient Teacher)

The authors propose a new framework called FedSAP. Think of it as a teacher who uses a "Patient Curriculum" and a "Smart Map" instead of just shouting orders.

1. The "Warm-Up" Schedule (Progressive Alignment)

Instead of forcing agreement on Day 1, FedSAP acts like a patient coach:

Early Rounds: For the first few weeks, the teacher tells the students, "Just focus on learning your own animals. Don't worry about matching the group yet." This lets the students build a solid, clear understanding of cats, dogs, etc., on their own.
Gradual Introduction: Slowly, over time, the teacher starts asking them to compare their drawings with the group's.
Full Alignment: By the time the students are confident in their own skills, the teacher says, "Okay, now let's all agree on the standard."
Why it works: By the time the pressure to "align" arrives, the students' drawings are already good. The group agreement helps refine them, rather than destroying them.

2. The "Smart Map" (Geometry-Driven Proxy Separation)

Usually, to keep different animals (cats vs. dogs) from getting mixed up, you need a special set of "reference cards" (learnable parameters) that take up extra space and require extra communication.

FedSAP is clever: it says, "We don't need new cards!"

The teacher already sends the students the "Global Cat Definition" (the prototype).
FedSAP uses these existing definitions as the "reference cards."
It creates a rule on a "mental map" (a hypersphere): "If you are drawing a cat, move your drawing toward the Cat Definition. If you are drawing a dog, push your drawing away from the Cat Definition."
The Benefit: This keeps the different animal groups neatly separated on the map without needing any extra communication or complex new tools. It just uses the tools they already have.

The Bonus: Learning Without Labels

The paper also shows that this method works well even when some students have pictures without names (unlabeled data).

Once the students have learned the basics (during the "warm-up" phase), they can look at a new, unlabeled picture and guess, "This looks most like the Cat Definition."
Because the teacher waited until the students were ready before introducing the group rules, these guesses are much more accurate. The system can then use these guesses to learn even more, without needing a human to label every single picture.

The Bottom Line

The paper tested this on three different "classrooms" (datasets) with varying levels of confusion (heterogeneity).

The Result: FedSAP consistently outperformed the old methods.
The Gain: It improved accuracy by up to 4 percentage points.
The Big Win: The improvement was biggest when the classrooms were the most different from each other (severe non-IID conditions).

In short, FedSAP fixes the problem of "rushing to consensus" by teaching students to stand on their own feet first, and then gently guiding them to walk together. It creates a cleaner, more organized mental map of the world without needing to send more data back and forth.

Technical Summary: Closing the Alignment-Maturity Gap in Federated Prototype Learning

Problem Statement
Federated Learning (FL) faces a fundamental challenge in learning discriminative visual representations from distributed, heterogeneous data. While prototype-based methods (e.g., FedProto, FedNH) address statistical heterogeneity by sharing class-level representations (prototypes) rather than full model weights, they introduce a critical instability known as the alignment-maturity gap.

In existing prototype-based FL, global prototypes are aggregated from local client embeddings and used as semantic anchors from the very first training round. However, in early training stages, local feature extractors have not yet learned discriminative structures, resulting in noisy, unrefined local representations. Consequently, the aggregated global prototypes are unreliable. When a static alignment weight is applied immediately, the large Euclidean distance between immature local embeddings and these noisy global prototypes generates massive alignment gradients. These gradients dominate the cross-entropy signal, suppressing the emergence of local discriminative structure and destabilizing optimization, particularly under severe non-IID conditions.

Methodology: FedSAP
The authors propose FedSAP (Federated Scheduled and Adaptive Prototypes), a framework designed to stabilize federated representation learning by decoupling the timing of global alignment from the strength of inter-class separation. FedSAP operates without modifying the underlying communication protocol or introducing additional learnable parameters. It relies on two complementary mechanisms:

Deterministic Alignment Curriculum:
Instead of applying full alignment pressure from the start, FedSAP employs a linear warm-up schedule for the prototype alignment weight ( $\lambda_t$ ).
- Mechanism: The weight increases linearly from 0 to a maximum value $M$ over a fixed number of rounds ( $t_w$ ).
- Effect: This delays the introduction of global alignment pressure, allowing local representations to consolidate and form discriminative structures before being pulled toward a global consensus. The schedule is deterministic, requiring no auxiliary signals or per-round communication overhead.
Geometry-Driven Proxy Separation Loss:
To enforce inter-class structure without adding parameters, FedSAP repurposes the existing global prototype bank as a shared "proxy bank."
- Mechanism: Local embeddings and global prototypes are projected onto a unit hypersphere. A cosine-softmax objective is applied, pulling representations toward their target class prototype while repelling them from non-target class prototypes.
- Effect: This enforces compact, well-separated class clusters on the hypersphere. By using the server-distributed prototypes as proxies, the method avoids the complexity and communication costs associated with maintaining separate learnable proxy parameters (as seen in methods like ProxyNCA or ProxyAnchor).

Extension to Semi-Supervised Learning
The paper demonstrates that FedSAP naturally extends to semi-supervised federated settings (SSFL). By establishing a stable embedding space during an initial supervised phase, the framework enables reliable pseudo-labeling of unlabelled data in a subsequent phase. Unlabelled samples are assigned pseudo-labels based on nearest-neighbor distance to the global prototype bank, with confidence thresholds calibrated automatically from the supervised phase to mitigate confirmation bias.

Key Contributions

Identification of the Gap: The authors explicitly identify and motivate the "alignment-maturity gap," where premature global alignment on immature representations suppresses local learning.
FedSAP Framework: The proposal of a framework that resolves this gap through a progressive alignment curriculum and a geometry-driven proxy separation loss applied to the existing prototype bank.
Performance and Generality: Empirical validation showing that FedSAP achieves well-separated embeddings at the same communication cost as FedProto, with significant performance gains. The framework is shown to extend naturally to semi-supervised settings, underscoring the generality of scheduled alignment as a design principle.

Experimental Results
FedSAP was evaluated across three benchmarks (FEMNIST, CIFAR-10, CIFAR-100) under varying degrees of statistical heterogeneity.

Performance: The framework achieved gains of up to 4 percentage points over prototype-based baselines (FedProto, FedMPS) and the parameter-aggregation baseline (FedAvg).
Heterogeneity: Improvements were most pronounced under severe non-IID conditions, where the instability of early-round prototypes is most severe.
Ablation: The results confirm that the progressive alignment schedule and proxy separation loss are critical for stabilizing the embedding space without increasing communication overhead.

Significance and Claims
The paper claims that FedSAP addresses a specific, previously unaddressed limitation in prototype-based FL: the temporal mismatch between the reliability of global anchors and the maturity of local representations. By treating federated prototype learning fundamentally as a representation learning problem, FedSAP produces a learning trajectory that is stable in early rounds and increasingly discriminative in later rounds. The authors position the "scheduled alignment" principle not merely as a tuning strategy for a specific protocol, but as a general design principle applicable to both fully supervised and semi-supervised federated environments.