A distributed, privacy-preserving platform for linkage of epidemiological data with pathogen genome sequences

The authors present SecureEpiLink, a privacy-preserving distributed platform that uses cryptographic hashing to automatically and securely link epidemiological data with pathogen genome sequences across different organizations, enabling real-time outbreak detection without exposing personal identifying information.

The Gist

Imagine two groups of people trying to solve a mystery about how a virus is spreading, but they are holding different pieces of the puzzle in separate rooms.

The Problem: Two Rooms, One Puzzle
On one side, you have Public Health Units. They have a list of people who got sick, including details like their age, where they live, and when they were diagnosed. Let's call this the "Who and When" list.
On the other side, you have Diagnostic Labs. They have the genetic "fingerprint" (genome sequence) of the virus found in those people. Let's call this the "Virus DNA" list.

To understand how the virus is moving through the community, these two lists need to be matched up. But there's a catch: the "Who and When" list has private names and addresses. If you just send that list to the lab, or vice versa, you risk exposing people's identities. This is especially risky for diseases like HIV or Hepatitis C, where people might face stigma or legal trouble if their status is revealed.

Traditionally, to match these lists, a human had to sit in a middle room, look at both lists, and manually try to connect the dots. This is slow, like trying to find a specific needle in a haystack by hand, and it often takes months or years.

The Solution: A Secure, Invisible Handshake
The authors built a new tool called SecureEpiLink. Think of this as a secure, automated "invisible handshake" system that lets the two rooms talk to each other without ever opening the doors or showing their private lists.

Here is how it works, using a simple analogy:

1. The Secret Recipe: Instead of sending a person's full name (like "John Smith"), the system takes a few safe details (like the first two letters of the first and last name, plus the date of birth) and runs them through a special "digital blender." This creates a unique, scrambled code (a cryptographic hash). It's like turning a secret recipe into a single, unbreakable number.
2. The Salt: To make sure two different "John Smiths" don't get the same code, the system adds a random "salt" (a secret ingredient) to the mix before blending.
3. The Match: The Public Health Unit and the Lab both create these scrambled codes for their records. They send only the codes to each other.
   • If the codes match, the system knows, "Ah, this virus fingerprint belongs to this specific notification!"
   • Crucially, no names or addresses are ever shared. The system only knows that a match exists, not who the person is.
4. The Result: Once a match is found, the Lab can send the virus DNA to the Public Health Unit, and they can study the spread of the virus without ever knowing the patient's identity. The original private data stays locked safely in its own room.

Testing the System
The researchers tested this system using real data from New South Wales, Australia, involving HIV and Hepatitis C.

• Speed and Accuracy: They found that SecureEpiLink was just as good at finding matches as the slow, manual human method, and actually did a better job than some older computer methods.
• The "Glitch" Factor: The only time the system failed to match a record, it wasn't because the technology was broken. It was because of simple human errors in the original data (like a typo in a name or a wrong date). The system is only as good as the data fed into it.
• Scalability: They also tested the system with massive amounts of fake data (up to 12 million records). They found that while it works great for smaller groups (like a state), if the group gets huge, there's a small chance that two different people might accidentally get the same scrambled code (a "collision"). However, for the current size of HIV cases in the region, this risk is tiny.

The Big Picture
The paper concludes that SecureEpiLink is a working "proof-of-concept." It shows that we can link sensitive health data with virus genetics quickly and securely, without a central database that holds everyone's private information. It's like giving the two puzzle rooms a direct, secure phone line that only rings when a piece fits, keeping the rest of the puzzle hidden from prying eyes.

The authors have made the code for this system available online so others can try it out, but they emphasize that this is a new tool that needs careful setup and standard rules to work perfectly in the real world.

Technical Summary

Technical Summary: SecureEpiLink – A Distributed, Privacy-Preserving Platform for Epidemiological Data Linkage

Problem Statement
Efficient integration of epidemiological notification data (collected by public health units) with pathogen genome sequence data (generated by diagnostic laboratories) is critical for identifying transmission clusters, monitoring emerging mutations, and targeting public health responses. However, these data sources are often siloed across different organizations. Traditional linkage methods frequently rely on manual or semi-manual processes, causing significant delays in outbreak detection. Furthermore, centralized linkage approaches require transferring sensitive, personally identifiable information (PII) to a single repository, creating substantial privacy risks, particularly for stigma-associated pathogens like HIV, Hepatitis C (HCV), and Mpox, where transmission can lead to criminalization or social stigmatization.

Methodology
The authors developed SecureEpiLink, a proof-of-concept, distributed platform designed to automate the linkage of pathogen genome sequences and notification data without exposing PII.

• Cryptographic Protocol: The system utilizes a one-way cryptographic hash function to de-identify records. A linkage identifier is generated for each record by concatenating specific identifiable fields (e.g., name code, date of birth) and appending a randomly generated "salt." This process creates a non-decodable hash.
• Distributed Architecture: The platform operates on a peer-to-peer model using Node.js, Express, and Docker. It supports two distinct node types:
  • Laboratory Nodes: Manage pathogen sequencing data.
  • Public Health Unit Nodes: Manage notification metadata.
  • Linkage Transaction: Linkage occurs via RESTful APIs. During a transaction, nodes exchange only the hashed, anonymized identifiers. If a match is found, non-sensitive metadata and sequence files are shared. The salt is destroyed post-linkage to prevent re-identification.
• Evaluation Strategy:
  • Real-World Data: The system was benchmarked against manual linkage and the established Centre for Health Record Linkage (CHeReL) using empirical HIV and HCV datasets from New South Wales (NSW), Australia.
  • Simulated Data: To test scalability and collision rates, the authors generated large-scale synthetic datasets (up to 12 million records) using the randomNames R package, incorporating realistic ethnic diversity and simulated data entry errors (e.g., character distortions in name codes).
  • Field Analysis: Various combinations of linkage fields (name code, date of birth, sex, postcode, prison/clinic codes) were tested to determine optimal inputs for successful matching.
  • Pipeline Integration: The platform was integrated with a bioinformatics pipeline using SnakeMake, HIV-TRACE for cluster analysis, and Quarto for automated report generation.

Key Results

• Linkage Performance: SecureEpiLink demonstrated linkage success rates comparable to or exceeding established methods. In HIV datasets, SecureEpiLink successfully linked 87.3% of records, compared to 77.9% for the CHeReL manual approach. All linkage failures were attributed to data entry errors or inconsistencies in the underlying datasets rather than algorithmic flaws.
• Field Optimization: For both HIV and HCV datasets, the combination of name code (first two letters of surname and given name) and date of birth yielded the highest successful linkage percentages (97.63% for HCV and 88.77% for HIV) and the lowest rates of undetermined or unsuccessful matches.
• Scalability and Collisions: Simulated analysis revealed that linkage success is highly dependent on the uniqueness of the linkage fields. While small, state-level datasets showed negligible collision rates, scaling to 12 million records resulted in approximately 8% of records having duplicate identifiers, highlighting potential limitations for national or international deployment without additional fields or data standardization.
• Error Sensitivity: The deterministic nature of the protocol makes it susceptible to minor data entry errors. The study noted that introducing random character-level distortions to 5–16.67% of name codes significantly impacted linkage success, underscoring the need for rigorous data collection standards.
• System Integration: The platform successfully facilitated a full workflow, from data upload and linkage to bioinformatic analysis (pairwise distance matrices, network visualization) and the generation of dynamic HTML reports containing cluster networks and metadata distributions.

Significance and Claims
The paper positions SecureEpiLink as a viable alternative to centralized data repositories, offering a model where organizations retain control of their own data while enabling near real-time, privacy-preserving linkage. The authors claim that this approach:

1. Enhances Privacy: By using cryptographic hashing and a distributed architecture, the system allows for the identification of transmission clusters without exposing individual identities, addressing ethical and legal concerns regarding stigma and criminalization.
2. Improves Timeliness: It replaces irregular, manual linkage processes with an automated, transaction-based system, potentially accelerating the detection of emerging outbreaks.
3. Demonstrates Feasibility: The proof-of-concept validates that distributed linkage using hashed identifiers can achieve success rates comparable to established manual protocols using real-world empirical data.

The authors conclude that while the system shows promise, successful scaling to national levels will require addressing operational challenges, such as standardizing data entry to minimize errors, managing heterogeneous data sources, and implementing robust governance and security protocols across distributed nodes. The platform is released as open-source software to facilitate replication and further development.