Data Auditing and Quality Assurance in a Federated Learning Consortium; Getting the Best of Both Worlds from Cross-Institutional and In-House Data Quality Inspection
This paper demonstrates that combining in-house and federated learning dashboards creates an optimal data quality assurance model for cross-institutional research, effectively balancing the thoroughness of local validation with the scalable, ecosystem-wide pattern detection of collaborative oversight.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer
In the world of medical research, some of the most important questions concern rare diseases. Because these conditions affect so few people, a single hospital rarely sees enough cases to find a clear answer. To solve this, scientists need to combine records from many different hospitals across the globe. However, a strict barrier stands in the way: patient privacy. Laws and ethical rules forbid hospitals from sending detailed personal health files to a central location, as doing so could expose sensitive information. To get around this, researchers have developed a method called federated learning. Instead of moving the data, they send the computer program to the data. The program visits each hospital, learns from the local records, and brings back only the mathematical lessons, leaving the private files behind. This approach allows for powerful collaboration without breaking privacy, but it creates a new problem. If you cannot look at the raw files, how do you know if the data inside them is accurate? A program cannot learn correctly if the numbers it is fed are wrong, yet checking for errors usually requires seeing the individual records that the system is designed to hide.
A team of researchers set out to solve this specific dilemma within a large, evolving network called STRONG-AYA, which studies cancer in adolescents and young adults. They built two different tools, or dashboards, to inspect the quality of the data. One tool was designed for use inside a single hospital, where the researchers could see every individual record. The other tool was built for the federated network, where it could only see summaries and statistics sent back from the other hospitals. To test how well these tools worked, the researchers took a real dataset of breast cancer records from a cancer center in Lyon, France, and created two copies. They then deliberately introduced mistakes into the data, such as recording a patient's age as younger than their diagnosis date, or assigning a tumor type that does not exist for a specific gender. They also simulated a scenario where two different hospitals used different coding systems to describe the same illness, a common issue in real-world collaboration.
The results showed that the two tools offered very different, yet necessary, views of the same situation. The in-house dashboard, which had full access to the raw records, acted like a microscope. It could pinpoint exactly which patient had an error, such as a calculated body mass index that was physically impossible, and tell the hospital staff precisely which record needed fixing. It found that 9.2 percent of the data points for a specific cancer staging metric were missing, and it could trace every single one of those missing points to a specific person. In contrast, the federated dashboard acted like a wide-angle lens. Because it could not see individual names or records, it could not point to a specific patient. Instead, it looked at the patterns across the entire network. It successfully detected that one hospital was using a different coding system than the other, a discrepancy that would have been invisible if looking at only one site. It also spotted that the distribution of certain data points was different between the two simulated centers, revealing a systematic difference in how the data was recorded.
The study found that neither tool was perfect on its own, and that relying on just one would leave gaps in the research. The in-house tool provided the depth needed to clean the data, but it could not see the big picture of how different hospitals compared to one another. The federated tool could see the broad trends and differences between institutions, but it lacked the detail to fix specific errors. The researchers concluded that the best approach is to use both together. The federated system can alert the network to widespread issues or inconsistencies between sites, while the in-house system allows each hospital to dive deep and correct the specific mistakes. This combined strategy allows researchers to maintain the privacy of their patients while still ensuring that the data they use to make life-saving discoveries is as accurate and reliable as possible.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.