💻 computer science

DIST-FL: Enhancing Security for TEE-based Aggregation in Federated Learning

This paper introduces DIST-FL, a distributed system leveraging multiple TEEs to form an append-only ledger that effectively thwarts server-side state rollback and I/O manipulation attacks in federated learning while achieving a 6x throughput boost over existing counterparts.

Original authors: Guanlong Wu, Ju Yang, Zhen Huang, Jianyu Niu, Guoxing Chen, Jianzong Wang, Yinqian Zhang

Published 2026-06-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Guanlong Wu, Ju Yang, Zhen Huang, Jianyu Niu, Guoxing Chen, Jianzong Wang, Yinqian Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a group of people trying to solve a giant puzzle together, but they don't want to show each other their individual puzzle pieces. This is Federated Learning. Instead of sending their private pieces to a central box, they keep them at home, do some work on them, and just send back a tiny note saying, "I think this piece goes here."

Usually, a central server (the "Box") collects these notes, mixes them together, and sends back a new, better version of the puzzle instructions. The problem? We have to trust the Box not to peek at the notes, steal them, or cheat by only picking notes from people who agree with the Box's bias.

To fix this, scientists put the Box inside a Trusted Execution Environment (TEE). Think of a TEE as a magical, unbreakable glass safe. Once the Box is inside, no one (not even the owner of the safe) can peek inside or change what's happening. It's supposed to be the ultimate trust.

The Problem: The "Magic Safe" Has a Glitch

The authors of this paper, DIST-FL, discovered that even these "magic safes" have two hidden backdoors that a sneaky server owner can exploit:

The "Rewind" Button (State Rollback): Imagine the Box is inside the safe. The owner can hit a "rewind" button on the safe's clock. If the Box picks a group of people to send notes, and the result isn't what the owner wants, the owner can rewind time, try again, and keep rewinding until the Box picks exactly the group the owner wants.
- The Result: The owner can cherry-pick only the notes they like, ignoring the rest, which ruins the quality of the final puzzle.
The "Mailman" Trick (I/O Manipulation): The safe is sealed, but the mailman bringing the notes to the safe is not. The owner can tell the mailman, "Don't let these specific notes in," or "Throw these away."
- The Result: The Box thinks it's getting notes from everyone, but it's actually only getting a biased, filtered selection.

The paper shows that with these two tricks, a bad server can secretly control the learning process and even figure out what specific people's private data looks like by watching how the results change when they rewind time.

The Solution: DIST-FL (The "Honest Jury" System)

To fix this, the authors built DIST-FL. Instead of relying on one single "magic safe," they created a distributed team of safes that act like an honest jury.

Here is how it works, using simple analogies:

The "Append-Only Ledger" (The Public Diary):
Imagine the team of safes keeps a shared, public diary that is append-only. Once a page is written in the diary, it cannot be erased, rewritten, or torn out. If a safe tries to "rewind" time to change a previous entry, the other safes will see that the diary doesn't match and reject the change. This stops the "Rewind" attack.
The "Proof-of-Input" (The Headcount Check):
To stop the "Mailman" trick, the system uses a clever check. Before the final mix happens, the "Leader" safe (the one in charge for that round) must prove it actually received notes from the other safes.
- The Leader sends out a "missing list" (a bitmap) saying, "I have notes from Alice and Bob, but I'm missing Charlie."
- The other safes check their own mail. If they have Charlie's note, they send it over.
- The Leader can only proceed if at least a majority of the other safes confirm they have the notes. If the Leader tries to hide a note, the other safes will catch it because they know they don't have a confirmation from the majority.

The Results: Faster and Safer

The authors built a prototype of this system and tested it in a real-world internet setting (like connecting computers across different cities).

Safety: It successfully stopped the bad actors from rewinding time or hiding notes. The model learned correctly without bias.
Speed: Surprisingly, this complex system of multiple safes was 6 times faster than other secure methods that try to solve the same problem using heavy cryptography. It performed almost as well as a single, unsecured server, but with the safety of the "Honest Jury."

In short: The paper proves that a single "magic safe" isn't enough to stop a clever cheater. But if you put a team of safes in a room with a public, unchangeable diary and a strict headcount check, you can stop the cheating, protect privacy, and still get the job done quickly.

Technical Summary of DIST-FL: Enhancing Security for TEE-based Aggregation in Federated Learning

Problem Statement

Federated Learning (FL) relies on a central server to coordinate client training and aggregate model updates. While Trusted Execution Environments (TEEs) have been proposed to secure the server-side against malicious actors by isolating code and data, this paper argues that existing TEE-based FL protocols are fundamentally vulnerable. The authors identify that server-side adversaries can exploit two inherent TEE limitations: I/O manipulation and state rollback.

Specifically, the paper demonstrates that:

Biased Client Selection: An adversary controlling the OS can manipulate I/O to exclude specific clients or use state rollback to replay the client selection phase until a desired (biased) subset is chosen. This degrades model robustness and performance.
Aggregation Replay: By rolling back the state of the TEE after an aggregation round, an attacker can replay the aggregation process with different subsets of clients. By comparing the results of repeated executions (e.g., removing one client at a time), the attacker can perform differential analysis to reconstruct and leak individual client updates. This vulnerability exists even in protocols designed to be robust against malicious clients (e.g., using Krum).

Existing solutions, such as those using append-only ledgers or distributed TEEs, often fail to address these specific FL requirements, leading to either high overhead (if appending raw gradients) or insufficient protection against I/O manipulation.

Methodology: DIST-FL

To address these vulnerabilities, the authors propose DIST-FL, a distributed system of servers guarded by multiple TEEs. The system is designed to ensure operation linearizability and mitigate I/O manipulation without incurring the prohibitive costs of direct consensus on raw client updates.

Core Architectural Components:

Distributed TEE Servers: The system employs $n = 2f + 1$ servers equipped with Intel SGX, tolerating up to $f$ malicious servers.
Modular Enclaves: The trusted logic is split into three enclaves:
- Attestation Proxy Enclave (APE): Manages keys and remote attestation to establish secure channels.
- Aggregator Enclave (AE): Handles random client selection, update synchronization, and aggregation.
- Ledger Enclave (LE): Integrates Nimble, a state-of-the-art append-only ledger, to maintain a shared history of state updates and prevent rollback.

Key Mechanisms:

Aggregate-Before-Append: To avoid the high overhead of appending large client gradients to the ledger, the leader server aggregates the gradients inside the TEE first. Only the encrypted aggregation result (the global model update) is appended to the Nimble ledger. This ensures rollback protection for the final state while maintaining efficiency.
Proof-of-Input (PoI): To counter I/O manipulation where a malicious leader might censor client updates, DIST-FL introduces a PoI mechanism. The leader cannot unilaterally decide the input set. Instead, it must collect a bitmap of received updates from other servers. The leader proceeds with aggregation only after receiving confirmations from at least $f + 1$ servers. This ensures that at least one honest server's view of the received updates is included, preventing the leader from arbitrarily excluding clients.
Linearizability: By committing the aggregation state to the append-only ledger before revealing the result, the system ensures that the state cannot be rolled back to a previous round, thwarting replay attacks.

Key Contributions

The paper claims the following contributions:

Security Analysis: It presents the first analysis revealing how TEE vulnerabilities (rollback and I/O manipulation) specifically compromise FL operations, demonstrating that they allow attackers to bias client selection and leak individual updates via aggregation replay.
System Design: The design of DIST-FL, which combines rollback-protected state continuity (via Nimble) with leader-resilient input establishment (via PoI). It introduces an "aggregate-before-append" strategy to scale TEE-based consensus to FL workloads.
Implementation and Evaluation: A prototype implementation on Intel SGX using PyTorch, evaluated in Wide Area Network (WAN) settings across 50 cloud instances with up to 3,400 clients.

Experimental Results

The authors evaluated DIST-FL against a Single-TEE baseline, a "Consensus-on-Updates" baseline (directly appending raw gradients), and a Multi-Party Computation (MPC) baseline.

Security Efficacy: Under attacks involving reduced participation (censoring 30 out of 50 clients) and biased participation (excluding specific data classes like "trucks" in CIFAR-10), the Single-TEE baseline suffered severe accuracy drops (e.g., from 86.6% to 12.5% on CIFAR-10 for biased attacks). In contrast, DIST-FL maintained high accuracy (84.0% on CIFAR-10), effectively neutralizing the attacks.
Performance:
- Throughput: DIST-FL achieved approximately 6x higher throughput compared to the Consensus-on-Updates and MPC baselines.
- Latency: While DIST-FL had slightly higher latency than a single vulnerable TEE (due to consensus overhead), it was significantly faster than the cryptographic alternatives. For a 13.1 MB model, DIST-FL's latency was roughly double that of Single-TEE, whereas the Consensus-on-Updates baseline was 14 times slower.
- Scalability: DIST-FL remained stable as the number of clients increased, whereas the other secure baselines saw latency and throughput degrade significantly.
- Failure Recovery: The system demonstrated resilience to non-leader server crashes with no visible impact on training. Leader crashes resulted in a short (~1s) recovery timeout, while exceeding the fault tolerance threshold (8 out of 9 servers crashing) correctly halted the system.

Significance and Claims

The paper positions DIST-FL as a practical solution that bridges the gap between the theoretical security of TEEs and the practical requirements of Federated Learning. The authors claim that while TEEs offer a straightforward path to server trust, they are insufficient without addressing state continuity and I/O integrity. DIST-FL demonstrates that it is possible to build a distributed TEE-based FL system that:

Thwarts server-side adversaries exploiting rollback and I/O manipulation.
Matches the performance of a single TEE server closely, avoiding the massive overhead of full cryptographic solutions (MPC) or naive distributed consensus.
Provides a scalable framework for robust and private aggregation in real-world WAN environments.

The authors conclude that their work highlights the necessity of moving beyond single-TEE assumptions in FL and provides a concrete, efficient architecture to secure the aggregation process against sophisticated server-side attacks.