🤖 machine learning

ERIS: Enhancing Privacy and Scalability in Federated Learning via Federated Shard Aggregation

ERIS is a novel federated learning framework that enhances privacy and scalability for billion-parameter models by introducing Federated Shard Aggregation to distribute client-side updates across multiple aggregators, thereby eliminating central bottlenecks and limiting information leakage while maintaining convergence and model utility without relying on heavy cryptography.

Original authors: Dario Fenoglio, Pasquale Polverino, Jacopo Quizi, Martin Gjoreski, Akash Dhasade, Marc Langheinrich

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Dario Fenoglio, Pasquale Polverino, Jacopo Quizi, Martin Gjoreski, Akash Dhasade, Marc Langheinrich

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you and a group of friends want to build a giant, super-smart robot brain together. The problem is, everyone has their own secret diary of personal experiences (data) that they can't share because of privacy laws or personal trust.

Federated Learning is the idea that you can train this robot brain without anyone ever handing over their diaries. Instead, everyone trains a small piece of the brain on their own computer and sends only the "lessons learned" (mathematical updates) to a central teacher. The teacher combines these lessons to improve the robot.

However, the paper points out two big problems with this current method:

The Bottleneck: If you have millions of friends, the central teacher gets overwhelmed trying to collect and sort all the lessons. It's like one person trying to sort a million letters at a post office; it takes forever.
The Privacy Leak: Even though you aren't sending your diary, the "lessons learned" can sometimes be reverse-engineered by a sneaky hacker to guess what was in your diary.

The authors introduce ERIS, a new system designed to fix both problems at once without making the robot brain less smart. Here is how it works, using simple analogies:

1. The "Jigsaw Puzzle" Strategy (Federated Shard Aggregation)

In the old system, everyone sends their entire lesson to one central teacher. In ERIS, the system uses a technique called Federated Shard Aggregation (FSA).

Imagine every lesson is a giant, complex jigsaw puzzle.

The Old Way: Everyone sends their whole puzzle to one person. That person sees everything.
The ERIS Way: Before sending, everyone cuts their puzzle into 50 separate, non-overlapping pieces (shards).
- Person A sends Piece 1 to Teacher 1, Piece 2 to Teacher 2, Piece 3 to Teacher 3, and so on.
- Teacher 1 only sees Piece 1 from everyone. Teacher 2 only sees Piece 2.
- The Magic: Because the pieces are cut in a specific, coordinated way, the teachers can still put their pieces together to form the exact same final picture as the old system. The robot brain learns just as well.
- The Privacy Win: No single teacher ever sees a whole puzzle. If a hacker hacks Teacher 1, they only get a tiny, useless fragment of the information, not the whole secret.

2. The "Compressed Postcard" (Distributed Shifted Compression)

Even with the puzzle pieces, sending 50 pieces to 50 different teachers can still be a lot of mail. To make it faster, ERIS adds a second layer called Distributed Shifted Compression (DSC).

Think of this as writing your lesson on a postcard instead of a full book.

Before cutting the puzzle, the system summarizes the most important parts and throws away the fluff (compression).
It also uses a "shift" trick (like a secret code) to make sure that even though information is thrown away, the math still adds up perfectly when the teachers put it back together.
This makes the "mail" (data transmission) tiny, speeding up the whole process significantly.

3. The Result: A Faster, Safer, Smarter System

The paper claims that by combining these two ideas, ERIS achieves three things that usually fight against each other:

Privacy: Because no single observer sees the full picture, it is much harder for hackers to steal private data. The paper proves mathematically that the more "teachers" (aggregators) you have, the safer it gets.
Scalability: Because the work is split among many teachers instead of one, the system doesn't get clogged up. It can handle huge models (like the ones used for large language models) without slowing down.
Utility (Smartness): This is the most important part. Usually, when you try to make things private or faster, the robot brain gets "dumber" (less accurate). ERIS claims to be the first system that keeps the brain just as smart as the original, unmodified version, while still being private and fast.

The Bottom Line

The authors tested ERIS on various tasks, from recognizing images of cats and dogs to understanding human language. They found that:

It was just as accurate as the standard method.
It was much harder for hackers to steal data or guess what people were training on.
It was much faster to run, especially when dealing with massive models.

In short, ERIS is like a new way of organizing a group project where everyone splits their work into tiny, secret fragments, sends them to different team leaders, and reassembles them perfectly at the end—ensuring no one person sees the whole picture, but the final result is perfect.

Technical Summary: ERIS – Enhancing Privacy and Scalability in Federated Learning via Federated Shard Aggregation

Problem Statement

Scaling Federated Learning (FL) to billion-parameter models (e.g., Large Language Models) creates a critical trade-off between privacy, scalability, and model utility. Existing solutions typically address these challenges in isolation, often sacrificing one for the others:

Privacy vs. Utility: Perturbation-based methods like Differential Privacy (DP) or gradient pruning reduce information leakage but frequently degrade model accuracy, particularly in large models or low-data regimes. Cryptographic protocols (e.g., Secure Aggregation) hide updates but introduce significant computational overhead and hardware requirements, hindering scalability.
Scalability vs. Utility: Traditional centralized FL suffers from a single-server bottleneck, leading to network congestion and communication inefficiencies as client numbers grow. Fully decentralized architectures distribute communication but often rely on neighborhood-only aggregation, which can alter the optimization trajectory, slow convergence, or reduce final model utility compared to centralized FedAvg.
The Core Conflict: Current methods often introduce approximations, perturbations, or deviations from centralized aggregation that alter the learning trajectory. There is a persistent need for a framework that simultaneously preserves the utility of centralized collaboration, ensures information-theoretic privacy, and scales efficiently without relying on heavy cryptography or utility-degrading noise.

Methodology: The ERIS Framework

The authors propose ERIS, a novel FL framework centered on Federated Shard Aggregation (FSA), optionally enhanced with Distributed Shifted Compression (DSC).

1. Federated Shard Aggregation (FSA)

FSA is the core mechanism designed to remove the central aggregation bottleneck while preserving the exact update trajectory of centralized FL.

Sharding: Instead of sending a full client update to a central server, each client partitions its update vector into $A$ disjoint, non-overlapping shards using binary masks.
Distributed Aggregation: Each shard is sent to a distinct, client-side aggregator. Aggregators independently compute the average of the shards they receive across all clients.
Reassembly: Aggregators broadcast their partial model updates back to clients. Clients reassemble the full global model by summing the received shards.
Key Property: Because the shards are disjoint and complete (their masks sum to the all-ones vector), the reassembled global model is mathematically identical to the one produced by centralized FedAvg. No coordinate is discarded, and no approximation error is introduced in the aggregation step.

2. Integration with Distributed Shifted Compression (DSC)

To further enhance scalability and privacy, ERIS integrates DSC as a pre-processing layer before sharding.

Mechanism: Clients apply a shifted compression operator to their local gradients before partitioning. This reduces the number of transmitted parameters and the coordinates exposed in each shard.
Shift Compensation: To maintain convergence, both clients and aggregators maintain reference vectors to track and compensate for the compression shift, ensuring the aggregated update remains unbiased.
Modularity: This design allows ERIS to incorporate other transformations (e.g., DP) before partitioning, provided the aggregation side applies corresponding corrections.

Key Contributions

Federated Shard Aggregation (FSA): A distributed aggregation mechanism that shards client updates across multiple aggregators. It eliminates the central server bottleneck, limits the information visible to any single observer (enhancing privacy), and preserves the exact centralized FL update trajectory (ensuring utility).
Seamless DSC Integration: The framework naturally supports DSC, reducing transmitted payloads and exposed coordinates without compromising the utility guarantees provided by FSA.
Theoretical Guarantees:
- Convergence: The authors prove that FSA incurs no information loss compared to centralized aggregation. With DSC, they establish convergence bounds showing that utility depends on gradient estimator variance and is independent of the number of rounds $T$ (unlike some prior DP-compressed methods).
- Privacy: They derive an information-theoretic upper bound on mutual information leakage. The leakage is shown to scale with $1/A$ (where $A$ is the number of aggregators) and the compression retention probability $p$ . This demonstrates privacy amplification through sharding and compression.
Empirical Validation: Extensive experiments across image (MNIST, CIFAR-10, ImageNet, LFW) and text (IMDB, CNN/DailyMail) tasks, ranging from small models to 1.3B parameter LLMs, validate the framework against six state-of-the-art baselines.

Experimental Results

The paper evaluates ERIS across three dimensions:

Utility: ERIS achieves performance on par with non-private FedAvg across all datasets and model sizes, including large language models. In contrast, baselines like FedAvg-LDP, PriPrune, and SoteriaFL often suffer significant accuracy degradation, especially in large models or low-data regimes.
Privacy:
- Membership Inference Attacks (MIA): ERIS significantly reduces MIA accuracy compared to FedAvg, approaching the "Min. Leakage" baseline (where no gradients are shared). Increasing the number of aggregators ( $A$ ) and the compression strength ( $\omega$ ) further reduces leakage.
- Data Reconstruction Attacks (DRA): Against white-box attacks (DLG, iDLG, ROG, GGL), ERIS renders reconstructions highly distorted. Even with a minimal configuration ( $A=2$ ), the system effectively obfuscates gradients, whereas compression alone is often insufficient against adaptive attacks.
Scalability:
- Communication Efficiency: By distributing aggregation, ERIS removes the central bottleneck. For a 1.3B parameter model, ERIS reduces distribution time from ~5200s (FedAvg) to ~468s (without DSC) and down to ~236s (with DSC).
- Payload Reduction: With DSC, ERIS can reduce the upload size to ~1% of the full update (e.g., 46.8MB for a 1.3B model) while maintaining utility.

Significance and Claims

The paper claims that ERIS provides a foundational step for next-generation FL systems by demonstrating that privacy and scalability can be strengthened without sacrificing model utility.

Breaking the Trade-off: Unlike previous methods that force a choice between privacy/scalability and accuracy, ERIS maintains the exact optimization trajectory of centralized FL while distributing the load and limiting information exposure.
Independence from Heavy Cryptography: The framework achieves strong privacy guarantees through architectural design (sharding) and information-theoretic bounds, avoiding the computational overhead of secure multi-party computation or trusted execution environments.
Robustness: The system is shown to be robust to aggregator and link failures (degrading gracefully) and effective against both membership inference and data reconstruction attacks.
Generality: The modular nature of FSA allows it to support various centralized FL algorithms (e.g., FedAdam, FedYogi) and integrate with other privacy mechanisms, positioning it as a general-purpose layer for scalable, privacy-preserving learning.