Privacy Evaluation of Generative Models for Trajectory Generation

Technical Summary: Privacy Evaluation of Generative Models for Trajectory Generation

Problem Statement
Trajectory data is critical for urban intelligence applications but poses severe privacy risks, as a small number of spatiotemporal points can uniquely identify individuals. While deep generative models (Variational Autoencoders, Generative Adversarial Networks, and Diffusion Models) are increasingly used to synthesize trajectory data to avoid sharing real records, the assumption that these models inherently preserve privacy is unproven. Generative models are known to memorize training data, potentially allowing adversaries to reconstruct original records or infer sensitive information. Currently, there is a significant gap in the empirical evaluation of privacy within the trajectory generation literature; many studies focus solely on data utility and generation quality, often neglecting rigorous privacy assessments. Furthermore, formal privacy guarantees like Differential Privacy (DP) have limitations, including trade-offs with data utility, potential flaws in implementation proofs, and the inability to defend against all attack types (e.g., property inference).

Methodology
The authors propose a systematic investigation into the privacy of synthetic trajectory generation through three main steps:

1. Literature Review and Categorization: The paper first categorizes existing empirical privacy evaluation methods applicable to trajectories, including Trajectory User Linking (TUL), Linkage Attack Probability (LA), Reconstruction Attacks (RAoPT, iTracker), Tracking Attack Probability (TA), Mutual Information (MI), and Membership Inference Attacks (MIA).
2. Gap Analysis: The authors survey representative generative models (VAEs, GANs, and Diffusion Models) to map their usage of these evaluation methods. They find that VAE and Diffusion model studies generally lack privacy evaluation, while only a subset of GAN studies includes any form of evaluation. Crucially, no reviewed study had previously applied Membership Inference Attacks (MIAs) to generative trajectory models.
3. Empirical Evaluation via MIAs: To address this gap, the authors implement Membership Inference Attacks against four representative models:
   • GAN-based: LSTM-TrajGAN and MoveSim.
   • Diffusion-based: DiffTraj and Diff-RNTraj.
   • Attack Formulations:
     • For GANs, a discriminator-based MIA is used, leveraging the discriminator's confidence scores to distinguish between training (member) and non-training (non-member) trajectories.
     • For Diffusion Models, a loss-based MIA is employed, calculating the negative average noise prediction Mean Squared Error (MSE) over multiple timesteps. Lower loss indicates higher likelihood of membership.
   • Datasets: Experiments utilize the Foursquare NYC, GeoLife, and DiDi Chengdu taxi trajectory datasets. The evaluation is conducted in a white-box setting where the adversary has full access to model weights and architecture.

Key Contributions

1. Identification of Evaluation Gap: The paper identifies and categorizes empirical privacy evaluation methods for trajectory generation and demonstrates a significant lack of such evaluations in current literature, particularly regarding MIAs.
2. Systematic Mapping: The authors systematically map representative generative models (VAEs, GANs, Diffusion Models) to existing evaluation methods, highlighting that current practices often fail to assess privacy risks comprehensively.
3. Feasibility Demonstration: The work provides the first implementation of Membership Inference Attacks against generative trajectory models, proving that such empirical evaluations are practical and necessary.

Results
The empirical evaluation yielded mixed results across the four tested models, demonstrating that privacy risks are model-dependent:

• MoveSim (GAN): Exhibited an AUC-ROC of 0.7002, indicating clear evidence of membership leakage. This suggests the model memorized specific training trajectories to a degree exploitable by an adversary.
• LSTM-TrajGAN (GAN), DiffTraj (DM), and Diff-RNTraj (DM): Showed AUC-ROC values close to 0.5 (ranging from 0.4949 to 0.5029), consistent with random guessing. Under the specific attack formulations used, these models did not exhibit exploitable membership leakage.

Significance and Claims
The paper claims that the generative nature of these models does not eliminate privacy risks. The success of the attack on MoveSim confirms that privacy vulnerabilities are tangible and cannot be assumed absent simply because data is synthetic. The authors argue that:

• Formal guarantees (like DP) are insufficient on their own due to implementation flaws, inappropriate Unit of Privacy (UoP) choices, and scope limitations.
• Empirical evaluation is a necessary complement to formal guarantees, or a substitute where such guarantees are absent.
• The findings underscore the need for systematic, comprehensive empirical privacy evaluation across the full range of generative trajectory models to ensure the safety of synthetic data deployment.

The authors conclude modestly that while their work demonstrates the feasibility of MIAs, other attack formulations and categories (such as reconstruction or property inference attacks) may reveal additional vulnerabilities, necessitating further broad investigation.