Privacy Leakage in Federated Learning in Radiology Reports: A Comparative Evaluation of Tokenizer-Driven Privacy Risks
This study demonstrates that federated learning on radiology reports is vulnerable to significant gradient-based privacy leakage, where sensitive clinical text can be reconstructed with high fidelity regardless of the tokenizer used, indicating that domain-specific tokenizers alone are insufficient to protect patient privacy without additional safeguards like secure aggregation or differential privacy.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where hospitals could team up to build super-smart AI doctors without ever handing over their patients' private files. It sounds like a dream come true, doesn't it? This is the promise of Federated Learning. Think of it like a group of students working on a giant group project. Instead of everyone mailing their messy, personal notebooks to a central teacher (which risks getting them lost or stolen), each student keeps their notebook at home. They only send the teacher a single, tiny summary of what they learned—like a "cheat sheet" of the most important rules. The teacher combines these cheat sheets to make a master guide, then sends it back so everyone can learn from the new, smarter version.
But here's the twist: what if that "cheat sheet" accidentally contains the answers to the questions in the notebook? In the world of artificial intelligence, these cheat sheets are called gradients. They are mathematical updates that tell the AI how to get better. For a long time, scientists thought these updates were safe because they looked like gibberish. However, recent research has shown that with enough math magic, a sneaky observer could reverse-engineer those updates to reconstruct the original secret text. It's like looking at the dust on a bookshelf and being able to guess exactly which words were written on the pages inside. This paper dives into whether this "math magic" can steal sensitive medical stories from radiology reports, and it asks a surprising question: does using a specialized vocabulary make the secret safer or more dangerous?
The Great Medical Text Heist
In this study, researchers set up a digital heist to see how easily private radiology reports could be stolen from a Federated Learning system. They didn't use real patients or real hospitals; instead, they simulated a scenario with six "fake" hospitals (clients) and one "evil" central server. The goal? To see if the server could trick the system into spilling the beans on the patients' medical histories.
The researchers used a massive collection of real-world medical text—over 368,000 diagnostic reports and nearly 100,000 discharge summaries—to train their AI. They tested three different ways of breaking down the text into chunks, called tokenizers. Imagine these tokenizers as different types of scissors:
- GPT-2: A general-purpose pair of scissors that cuts text into small, generic pieces.
- LLaMA-2: Another general pair of scissors, slightly different in how it snips.
- RadBERT: A specialized, medical-trained pair of scissors designed specifically for doctor-speak. It knows that words like "pneumonia" or "fracture" are single, important units and tries to keep them whole.
The "evil server" in this story had a secret weapon: a tiny, invisible spy module hidden inside the AI model. This spy didn't just watch the math; it tweaked the model slightly before sending it out, allowing the server to capture the gradients and run a "gradient inversion" attack. This is a fancy way of saying the server tried to reverse the math to rebuild the original sentences from the cheat sheets.
The Shocking Results
The results were a bit of a wake-up call. The researchers found that the "evil server" was surprisingly good at stealing the text back.
- The Success Rate: Depending on how the data was grouped (called "batch size"), the server managed to perfectly reconstruct entire sentences 31% to 44% of the time. That means if you sent 100 medical reports through this system, the attacker could potentially read the full text of 31 to 44 of them without ever seeing the original files.
- The "Specialized" Trap: Here is the most interesting part. The team expected the specialized medical tokenizer (RadBERT) to be safer because it was built for doctors. Instead, it turned out to be the most vulnerable.
- With RadBERT, the server recovered 42.3% of sentences perfectly at the smallest batch size.
- With the general GPT-2, it was 42%.
- With LLaMA-2, it was 39.4%.
Why was the medical tokenizer worse? The researchers explain it with a simple analogy. If you use a general tokenizer, a complex medical term like "pneumothorax" gets chopped up into tiny, weird pieces like "pneu," "mo," "thor," and "ax." To steal the word back, the attacker has to perfectly reconstruct all four tiny pieces and put them in the right order. If they miss just one, the word is garbage.
But with the medical tokenizer (RadBERT), "pneumothorax" is treated as one single, solid block. If the attacker manages to steal that one block, they get the whole word instantly. The very thing that makes the AI better at understanding medicine (keeping terms whole) also makes it easier for a thief to steal those terms in one go. In fact, RadBERT recovered 18.1% of the unique clinical terms (like specific diagnoses and procedures), compared to only 12.5% for GPT-2 and 9.4% for LLaMA-2.
Does Bigger Help?
The researchers also tried to see if making the "groups" of data larger (increasing the batch size from 64 to 256) would hide the secrets better. It did help, but not enough to be safe.
- When they increased the batch size, the perfect reconstruction rate dropped. For example, with RadBERT, it went from 42.3% down to 37.2%.
- While this is an improvement, it means that even with large groups, the attacker could still steal nearly one-third of the sentences perfectly.
The Bottom Line
This paper doesn't claim that Federated Learning is broken forever, but it does sound a very loud alarm. It suggests that simply using a "smart" medical vocabulary or grouping data together isn't enough to protect patient privacy against a determined attacker.
The study highlights a tricky trade-off: the tools that make AI doctors smarter (specialized tokenizers) might also make it easier for bad actors to steal patient secrets. The authors conclude that to truly protect patients, hospitals can't just rely on the AI's design. They will likely need to add extra layers of security, like scrambling the data (differential privacy) or using secret codes (secure aggregation), to meet strict laws like HIPAA and GDPR. Until then, the "cheat sheets" sent between hospitals might be a little too easy to read.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.