🤖 AI

Technical Case Study of Privacy-Enhancing Technologies (PETs) for Public Health

This paper presents a technical case study demonstrating how Differential Privacy can generate realistic, privacy-preserving synthetic financial transaction data that, when combined with public health and mobility datasets, enables effective pandemic management tools for diagnostic nowcasting and predictive forecasting while ensuring data security.

Original authors: Avinash Laddha, Danil Mikhailov, Uyi Stewart

Published 2026-03-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Avinash Laddha, Danil Mikhailov, Uyi Stewart

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive, global puzzle. On one side, we have Public Health Officials trying to stop a virus from spreading. They need to know: Where are people gathering? Are they staying home? Who is meeting whom?

On the other side, we have Credit Card Companies (like Mastercard) holding a treasure trove of data: Where people are spending money, what they are buying, and when.

The Problem: The health officials need the credit card data to save lives, but the credit card companies can't just hand it over. That would be like giving a stranger your entire bank account history. It violates privacy and could expose people's secrets.

The Solution: This paper describes a brilliant experiment called the "Privacy-Enhancing Technologies (PETs) for Public Health Challenge." Think of it as a high-stakes cooking competition where the goal is to make a delicious soup (life-saving insights) without ever using the actual ingredients (private data).

Here is how they did it, broken down into simple concepts:

1. The "Fake" Data Recipe (Synthetic Data)

Instead of using real people's credit card statements, the team created "Synthetic Data."

The Analogy: Imagine a master chef who wants to teach students how to cook a specific dish but can't let them taste the real ingredients because they are too expensive or sensitive. So, the chef creates a perfectly realistic "fake" version of the ingredients using a recipe book. The fake carrots look, feel, and taste like real carrots, but they aren't actually from anyone's garden.
What they did: They built a computer program that generated millions of "fake" credit card transactions. These transactions looked exactly like real ones (people buying groceries, gas, or airline tickets in cities like Bogota or Santiago), but no real person ever made these purchases.

2. The "Privacy Shield" (Differential Privacy)

Even with fake data, there's a risk. If the fake data is too perfect, someone might guess, "Hey, this looks exactly like what my neighbor spends!"

The Analogy: Imagine you are trying to describe a crowd of people to a friend over the phone, but you have to whisper so no one else hears. To make sure you don't accidentally reveal who is standing next to whom, you add a little bit of static noise to your voice. The friend can still hear the general message ("There are 50 people in the room"), but they can't hear the specific whispers of individuals.
What they did: They used a mathematical technique called Differential Privacy. It adds a tiny bit of "mathematical noise" to the data. This ensures that even if you look at the results, you can never trace a specific transaction back to a specific person.

3. The Six Magic Tools (The Outcomes)

The challenge invited smart scientists from universities around the world to build tools using this "safe, fake data." Here is what they created, using everyday analogies:

🔍 The "Heat Map" Detector (Hotspot Detection)
- What it does: It spots crowded areas.
- The Analogy: Imagine a thermal camera that shows where the most "heat" (people) is. If a specific neighborhood shows a sudden spike in offline spending (like buying food at a market), the tool turns that area red.
- Why it helps: Health officials can send testing vans or vaccines to that red zone before the virus spreads there.
🚶 The "Commuter" Tracker (Mobility Analysis)
- What it does: It tracks how people are moving.
- The Analogy: It's like watching a school of fish. If the fish (people) suddenly stop swimming toward the "Restaurant" zone and start swimming toward the "Grocery" zone, the tool knows people are staying home and only buying essentials.
- Why it helps: It tells the government if their lockdown rules are actually working.
🛡️ The "Rule-Follower" Meter (Pandemic Adherence)
- What it does: It checks if people are following safety rules.
- The Analogy: If a city says, "Don't go to bars," and the data shows people are still buying expensive cocktails and hotel rooms, the meter turns red. If they are only buying medicine and toilet paper, it turns green.
- Why it helps: It helps officials know where to send more police or where to send more friendly reminders.
🤝 The "Who Meets Whom" Map (Contact Matrix)
- What it does: It figures out who is interacting with whom.
- The Analogy: Imagine a giant dance floor. This tool counts how many times teenagers dance with other teenagers, or how often grandparents dance with grandchildren.
- Why it helps: It helps scientists predict how fast a virus will jump from kids to adults, so they can protect the most vulnerable groups.
🚨 The "Early Warning" Siren (Outbreak Detection)
- What it does: It spots a virus outbreak before hospitals get overwhelmed.
- The Analogy: It's like a smoke detector. Usually, you smell smoke (sick people) after the fire starts. This tool smells the "smoke" in the spending habits (people buying medicine or staying home) before the fire gets big.
- Why it helps: It gives officials a head start to stop the fire.
🔮 The "Crystal Ball" (Forecasting)
- What it does: It predicts the future.
- The Analogy: By looking at how spending patterns change today, the tool predicts how many people will get sick next week.
- Why it helps: Hospitals can prepare extra beds and doctors before the patients even arrive.

The Big Lesson

The paper concludes that we don't have to choose between privacy and progress.

In the past, we thought, "We can't use this data because it's private," or "We must use this data even if it's risky." This project proved we can have both. By using "fake" data that looks real and adding a "privacy shield," we can solve big problems like pandemics without ever violating anyone's personal secrets.

In short: They built a safe, invisible bridge between private money data and public health needs, allowing scientists to see the invisible patterns of a virus without ever seeing the people behind the data.

1. Problem Statement

The core challenge addressed is the tension between the need for granular, timely data to manage pandemics and the strict privacy regulations (e.g., GDPR) and commercial sensitivities that prevent the sharing of private sector data.

The Gap: Traditional public health data collection is often too slow or lacks the granularity to track real-time human behavior (mobility, spending) which correlates with disease spread.
The Barrier: Financial transaction data held by companies like Mastercard is highly valuable for epidemiological modeling but cannot be shared due to privacy risks and commercial confidentiality.
The Objective: To assess the feasibility of using Privacy-Enhancing Technologies (PETs), specifically Differential Privacy (DP), to generate synthetic financial datasets that can be used for epidemiological decision-making without compromising individual privacy.

2. Methodology

The project followed a structured approach involving data synthesis, use-case definition, and tool development by a consortium of academic and private sector partners (Mastercard, data.org, Harvard/OpenDP, Javeriana University, and global awardees).

A. Synthetic Data Generation

Instead of sharing raw data, the team created a privacy-preserving synthetic dataset mimicking Mastercard transaction data.

Structure: The dataset included unique IDs, merchant categories, postal codes (covering Medellin, Bogota, Brasilia, Santiago), transaction types (online/offline), spend amounts, and transaction counts over weekly intervals (2019–2022).
Realism Calibration:
- Correlation: Financial trends were correlated with epidemiological metrics. The team initially used "New Cases" but switched to "New Deaths" as the primary metric because death counts are less volatile than case counts (which vary by testing availability) and better reflect policy-driven behavioral changes.
- Population Scaling: Transaction volumes were scaled to match the actual population proportions of the four cities.
- Bayesian Priors: Domain expertise was used to calibrate merchant category frequencies and apply "COVID-effect multipliers" to simulate spending shifts during lockdowns.
Privacy Layer: The OpenDP library was applied to the synthetic data generation process to ensure a baseline of differential privacy, preventing inference attacks on the original real-world data.

B. Evaluation Framework

Five specific policy use cases were defined to test the utility of the synthetic data:

Enhancement of Epidemiological Techniques: Integrating financial data into standard models.
Contact Pattern Inference: Deriving Who Acquires Infection From Whom (WAIFW) matrices.
Real-time $R_t$ Estimation: Improving the Effective Reproduction Number calculation.
Bias Mitigation: Correcting nowcasting biases caused by behavioral shifts.
Predictive Forecasting: Using transaction data as a feature for forecasting epidemic curves.

C. Tool Development

Six distinct tools/frameworks were developed by challenge awardees to address these use cases:

Hotspot Detection: Aggregates offline transactions by postal code with Gaussian noise to identify high-density interaction zones for resource allocation.
Mobility Analysis: Correlates transaction categories (retail, transit, grocery) with mobility trends, validated against Google Mobility Reports.
Pandemic Adherence Analysis: Monitors shifts between "essential" and "luxury" spending to gauge public compliance with health mandates.
Contact Matrix Estimation: Uses machine learning to infer age-group contact frequencies based on merchandise consumption patterns.
Outbreak Detection Framework: Uses ML (supervised and unsupervised) to detect anomalous patterns signaling early outbreaks under privacy constraints (DP-SGD).
Advanced Modeling Frameworks:
- DPEpiNN: A joint neural network and epidemic model integrating heterogeneous datasets.
- EpiDP: An open-access package for incorporating covariate time series (mobility/transactions) to improve $R_t$ estimates.

3. Key Results & Findings

The challenge demonstrated that synthetic transaction data, when processed through differential privacy, holds significant predictive power for public health.

Correlation Success: Cross-correlation analysis showed a strong, timely relationship (0–3 week lags) between epidemiological metrics (deaths) and transactional patterns.
Utility of Synthetic Data:
- Hotspot Detection: Successfully identified high-physical-interaction areas, enabling targeted resource deployment (testing, vaccines).
- Mobility Validation: Mobility trends derived from synthetic transactions aligned closely with external public datasets (e.g., Google Mobility Reports), validating the data's realism.
- Behavioral Monitoring: The tools successfully tracked shifts in spending behavior in response to lockdowns, providing a proxy for policy adherence.
Model Performance:
- Anomaly Detection: Supervised models achieved near 100% accuracy in detecting outbreaks with $\epsilon = 8.0$ , while unsupervised autoencoders showed promise for users without prior labeling knowledge.
- $R_t$ Estimation: While mobility data remained a strong covariate, the integration of financial data offered a new dimension for refining $R_t$ estimates, though the marginal gain over mobility data alone was noted as an area for further refinement.
Refinements: The project identified critical improvements for future datasets, such as switching from cases to deaths for correlation, adjusting city-specific sampling proportions, and refining spending multipliers to capture dynamic pandemic behaviors.

4. Significance and Contributions

This case study represents a paradigm shift in how private sector data can be leveraged for social impact.

Proof of Concept for PETs: It successfully demonstrated that Differential Privacy can be applied to financial data to create a "safe environment" for research without exposing sensitive customer information.
Bridging the Data Gap: It established a replicable framework for integrating cross-domain datasets (financial + epidemiological) to solve complex public health problems, overcoming the traditional silos between private data custodians and public health researchers.
Actionable Decision Support: The developed tools move beyond theoretical models to provide nowcasting (immediate hotspot detection) and forecasting (predicting infection rates) capabilities, directly aiding policymakers in resource allocation and intervention strategies.
Scalability and Openness: The release of open-source frameworks (like EpiDP and the Python packages from IIT Kanpur) and the detailed methodology provide a roadmap for other organizations to adopt similar privacy-preserving strategies.

Conclusion

The PETs for Public Health Challenge proved that it is possible to unlock the value of massive, sensitive financial datasets for pandemic management without violating privacy. By combining synthetic data generation, differential privacy, and advanced machine learning, the project delivered a suite of tools that enhance the speed, accuracy, and privacy of epidemiological decision-making. This approach offers a scalable model for future data-driven social impact initiatives across various domains.