🤖 machine learning

FIRMA: FIbonacci Ring Model Aggregation for Privacy-preserving Federated Learning

The paper introduces FIRMA, a family of server-free federated learning protocols that utilize Fibonacci-weighted ring aggregation, adaptive peer suppression, and optimized ring permutations to simultaneously achieve privacy-preserving classification heads, robust convergence under label skew, and superior accuracy compared to existing decentralized methods.

Original authors: Rachid Hedjam

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rachid Hedjam

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a group of friends trying to solve a giant jigsaw puzzle together, but they have a strict rule: no one is allowed to show their own puzzle pieces to anyone else. They can only share the ideas they've formed about how the pieces fit, not the pieces themselves.

This is the world of Federated Learning. Usually, to solve the puzzle, everyone sends their ideas to a "Team Captain" (a central server) who mixes them all together and sends back a new plan. But this Captain has two big problems:

Privacy Risk: If the Captain is a bit nosy (or gets hacked), they can look at the ideas and figure out exactly what puzzle pieces you were holding.
Bottleneck: If there are 1,000 friends, the Captain gets overwhelmed trying to listen to everyone at once.

The Problem with Existing Solutions

Some groups tried to solve this by having friends talk only to their neighbors in a circle (a "Ring"). But they had a flaw: they treated every neighbor the same. If you were standing next to someone who was terrible at puzzles, they would drag your ideas down just as much as a genius neighbor would lift them up. Also, in many of these circle methods, you still had to share your "special finishing touches" (the classification head), which leaked private info.

The Solution: FIRMA

The authors propose a new family of methods called FIRMA (FIbonacci Ring Model Aggregation). Think of it as a smarter way for the friends to talk in a circle without a Captain.

Here is how it works, broken down into three levels of sophistication:

Level 1: The Golden Ratio Rule (FIBFL)

Instead of treating neighbors equally, FIRMA uses a special math rule called the Golden Ratio (about 1.618).

The Analogy: Imagine you are in a circle. You have a neighbor on your left and one on your right. The Golden Ratio says: "Listen to your left neighbor 62% of the time, and your right neighbor 38% of the time."
Why? This isn't random. It creates a natural "flow" of ideas around the circle that is mathematically proven to be more efficient than listening to both equally.
The Privacy Win: You keep your "special finishing touches" (the part of the model that decides what the image is) completely to yourself. You only share the "thinking process" (the feature extractor). No one ever sees your private data.

Level 2: The "Smart Filter" (FIBFL+)

Sometimes, a neighbor might be having a bad day and their ideas are just wrong.

The Analogy: If your left neighbor is confused, the Golden Ratio rule still says "listen to them 62%." That's silly.
The Fix: FIBFL+ adds a "Smart Filter." It checks: "Is this neighbor actually good at the puzzle right now?" If they are struggling, it lowers their weight. If they are doing great, it listens more. But it still keeps the Golden Ratio bias (it never completely ignores the left neighbor), ensuring the circle keeps flowing smoothly.

Level 3: The "Perfect Seating Chart" (FIBFL++)

This is the full, ultimate version.

The Analogy: Imagine the friends are sitting in a circle. If you sit next to someone who has the exact same puzzle pieces as you, you don't learn much from each other. But if you sit next to someone with different pieces, you learn a lot.
The Fix: Before the game starts, FIBFL++ rearranges the seating chart (using a method called 2-opt) to make sure people with different puzzle pieces sit next to each other.
The Result: This maximizes the diversity of ideas flowing around the ring. It also runs the "message passing" multiple times in one round so that everyone's ideas eventually reach everyone else, just like a central server would, but without the server.

What the Paper Found

The authors tested this on four different "puzzles" (datasets) with different levels of difficulty and different ways of splitting the pieces (some friends had only cats, others only dogs).

The "Underdog" Victory: In the hardest scenarios (where friends had very different data), the standard "Team Captain" method (FedAvg) and the "Personalized" method (FedRep) often failed or were unfair. FIBFL++ was the only method that consistently performed well, sometimes even beating the Captain method.
Fairness: FIBFL++ was the fairest. It made sure no single friend was left behind with a terrible score, which happened often with other methods.
Stability: The simpler versions (Level 1 and 2) were fast to start but sometimes got "wobbly" and unstable if the data was very messy. The full version (Level 3) was the most stable and reliable.
Privacy: All versions of FIRMA successfully kept the "special finishing touches" private, meaning no one could reverse-engineer your private data from the shared ideas.

The Bottom Line

FIRMA is a new way for computers to learn together without a boss, without sharing private secrets, and without treating everyone the same. It uses a bit of ancient math (the Golden Ratio) and a smart seating chart to make sure the group learns faster and more fairly than ever before, especially when everyone has very different data.

Note: The paper focuses entirely on computer science simulations and mathematical proofs. It does not claim these methods are currently used in hospitals, self-driving cars, or specific real-world products, though it suggests they are ready for such environments.

Technical Summary: FIRMA (Fibonacci Ring Model Aggregation for Privacy-Preserving Federated Learning)

1. Problem Statement

Federated Learning (FL) currently faces a structural trilemma regarding privacy, decentralization, and model personalization:

Server Dependency and Gradient Inversion: Canonical protocols like FedAvg rely on a central server to aggregate full model updates. This creates a single point of failure and exposes the server to gradient inversion attacks, where a semi-honest server can reconstruct client training data.
Head Exposure in Personalized FL: Personalized methods (e.g., FedRep) improve performance under non-IID data by keeping the classification head local while sharing the feature extractor. However, they still require a central server for aggregation, retaining the single point of failure and gradient inversion risks.
Limitations of Decentralized Ring Topologies: Decentralized alternatives (e.g., RDFL) eliminate the server using peer-to-peer ring gossip. However, existing ring protocols typically transmit the full model (including the classification head) to neighbors, exposing sensitive discriminative information. Furthermore, they rely on uniform, uninformed weights (e.g., 0.5/0.5) for neighbor blending, ignoring model quality or data diversity.

The Gap: No existing protocol simultaneously achieves: (i) server-free operation, (ii) permanently private classification heads (never transmitted), (iii) a ring topology, and (iv) principled, asymmetric neighbor weighting.

2. Methodology: The FIRMA Family

The paper proposes FIRMA (FIbonacci Ring Model Aggregation), a family of three progressively enhanced protocols designed to fill the "Top-Right" quadrant of the FL design space (Server-Free + Private Head).

Core Mathematical Foundation

The protocols utilize the Golden Ratio ( $\phi \approx 1.618$ ) to define a parameter-free, asymmetric weight pair for ring gossip:
$\alpha = 1/\phi \approx 0.618, \quad \beta = 1/\phi^2 \approx 0.382$
Since $\alpha + \beta = 1$ , this forms a valid convex combination. Unlike uniform weights, this asymmetry introduces a directed bias (left-bias) that theoretically yields a tighter spectral gap than uniform gossip, accelerating consensus.

Protocol Variants

The family is structured as an ablation study where each variant adds exactly one structural component:

FIBFL (Basic Protocol):
- Architecture: Server-free ring topology where clients only exchange feature extractor parameters ( $\theta$ ); classification heads ( $\eta$ ) remain permanently local and private.
- Aggregation: Uses the fixed Fibonacci weights ( $\alpha, \beta$ ) to blend left and right neighbors.
- Training: Employs a two-phase local training procedure adapted from FedRep: Phase 1 adapts the local head (freezing the extractor), and Phase 2 updates the extractor (freezing the head). Both phases use persistent Adam optimizers (state carried over across rounds) to improve convergence stability.
FIBFL+ (Accuracy-Gated):
- Improvement: Introduces an accuracy-gated interpolation.
- Mechanism: It blends the Fibonacci prior with an accuracy-derived posterior. If a neighbor's local accuracy falls below a threshold $\tau$ , its weight is suppressed.
- Result: This adaptively down-weights poorly converged peers while preserving the Fibonacci directional bias, preventing low-quality updates from degrading the global model.
FIBFL++ (Full System):
- Improvement: Addresses the limitation of static ring ordering and single-pass gossip.
- Fix A (2-Opt Ring Permutation): Before training, a 2-opt algorithm reorders clients on the ring to maximize the cosine dissimilarity between adjacent clients' class distributions. This ensures that neighbors have complementary data, maximizing gradient diversity.
- Fix B (Global Coverage): Instead of a single gossip pass, the protocol performs $K_g = \lceil N/2 \rceil$ sequential passes per round. This ensures that information from every client reaches every other client within a single round, equivalent to a central server's broadcast radius.
- Fix C (Calibrated Retention): Uses cosine-annealed self-retention coefficients and a FedAvg-style warmup phase to stabilize convergence during the initial rounds before the ring topology fully engages.

3. Key Contributions

The paper claims the following specific contributions:

C1. FIBFL: The first ring FL protocol combining head privacy, asymmetric Fibonacci weighting, and two-phase Adam training.
C2. FIBFL+: An original accuracy-gated interpolation mechanism that suppresses under-performing neighbors without abandoning the Fibonacci bias.
C3. FIBFL++: A full system integrating three novel structural fixes: 2-opt diversity ordering, multi-pass global coverage, and cosine-annealed retention.
C4. Theoretical Analysis: Establishes convergence bounds and proves that the Fibonacci weight pair provides a strictly tighter spectral gap than uniform weights, and that the 2-opt ordering maximizes class diversity.
C5. Empirical Evaluation: A systematic evaluation across 168 configurations (4 datasets $\times$ 7 heterogeneity regimes $\times$ 6 methods).

4. Experimental Results

Experiments were conducted on CIFAR-10, Fashion-MNIST, MNIST-60k, and MNIST-1797 under IID, Dirichlet (various $\alpha$ ), and Label-Skew ( $K$ ) regimes.

Performance under Label-Skew: FIBFL++ significantly outperforms FedAvg in all 12 label-skew configurations. On CIFAR-10 with $K=1$ , FIBFL++ achieved a peak advantage of +20.7 percentage points over FedAvg.
Pareto Dominance: Under Dirichlet heterogeneity, FIBFL++ is the Pareto-dominant method among server-free protocols, achieving the highest accuracy in 17 of 28 configurations.
Fairness: FIBFL++ consistently achieves lower Gini coefficients (higher fairness) than FedAvg in label-skew scenarios, effectively navigating the accuracy-fairness duality.
Small Data Regimes: On the small dataset MNIST-1797 ( $N=5$ ), FIBFL++ was the only proposed method to outperform FedAvg (0.9695 vs. 0.9691) in the Label-Skew $K=1$ scenario.
Ablation Findings:
- The 2-opt ring ordering (Fix A) was the dominant contributor to performance gains, particularly in high-heterogeneity settings (e.g., +5.2 pp on MNIST-60k).
- Adaptive selection (FIBFL+) provided marginal gains compared to the structural benefits of the ring topology.
- FIBFL and FIBFL+ suffered from plateau instability (high variance) in small-data regimes, whereas FIBFL++ maintained stability due to ring structural regularization.

5. Significance and Claims

The paper positions FIRMA as a solution to the "structural trilemma" of federated learning. Its primary significance lies in occupying a previously unexplored quadrant of the design space: Peer-to-Peer Ring Topology with Permanently Private Heads.

Privacy: By keeping classification heads local and using a server-free ring, FIRMA eliminates the single point of failure and prevents gradient inversion attacks on the discriminative head.
Efficiency vs. Robustness: While FIBFL++ incurs higher communication costs ( $K_g$ passes) and computational overhead (2-opt ordering) compared to FedAvg, the paper argues this is justified in heterogeneous environments where FedAvg fails to converge or personalizes poorly.
Self-Calibration: The 2-opt ring ordering is highlighted as "self-calibrating," providing near-zero savings in homogeneous (IID) data but massive savings (up to 95.5%) in highly skewed (non-IID) data without manual tuning.
Deployment Guidance: The authors conclude that FIBFL++ is the recommended choice for federations with moderate-to-strong heterogeneity and fairness requirements, while FedAvg remains the default for homogeneous data due to its communication efficiency. FedRep is cautioned against in data-scarce regimes due to collapse in fairness.

The paper does not claim to solve all FL challenges (e.g., concept drift or large-scale $N > 50$ ) but establishes a robust foundation for privacy-preserving, decentralized optimization in non-IID settings.