Stable and Privacy-Preserving Synthetic Educational Data with Empirical Marginals: A Copula-Based Approach

1. Problem Statement

The field of Educational Data Mining (EDM) and Learning Analytics (LA) faces a critical tension between the need for high-quality, granular student data for research and strict privacy regulations that prevent data sharing. While Synthetic Data Generation (SDG) offers a solution by creating artificial datasets that mimic real statistical structures, existing methods suffer from two major technical flaws:

• Marginal Mismatch: Deep learning models (e.g., CTGAN, TVAE) and parametric approaches (e.g., Gaussian Copula) often fail to preserve the exact empirical marginal distributions of heterogeneous variables (continuous, integer, categorical). They tend to distort skewed, multi-modal, or discrete distributions, leading to systematic errors in downstream analysis.
• Regeneration Instability (Model Collapse): When synthetic data is used to train subsequent models (a "Synthetic Feedback Loop" or SFL), iterative regeneration causes progressive variance shrinkage and loss of distributional support. This leads to "model collapse," where rare subgroups and tail behaviors are erased, biasing subgroup analysis and policy decisions in education.
• Privacy Risks: Many existing generators lack formal privacy guarantees, risking re-identification of students through linkage or inference attacks.

2. Methodology: Non-Parametric Gaussian Copula (NPGC)

The authors propose NPGC, a plug-and-play synthesis method that replaces deep learning optimization with empirical statistical anchoring. The method is grounded in Sklar's Theorem and integrates Differential Privacy (DP) at two levels.

Core Workflow

The process is a three-stage generative pipeline:

1. Marginal Transformation ($X \to Z$):

   • Empirical Anchoring: Instead of assuming parametric families (e.g., Gaussian), NPGC uses the empirical Cumulative Distribution Function (CDF) for each variable.
   • Handling Heterogeneity:
     • Continuous/Integer: Maps values to the unit interval $(0, 1)$ using histogram-based CDFs or dequantization for discrete values.
     • Categorical: Maps to CDF intervals based on empirical category probabilities.
     • Missing Data: Treated as an explicit state. Missing values are mapped to a reserved interval in $(0, 1)$, preserving correlations between missingness patterns and observed variables.
   • Gaussian Projection: Transforms the unit interval values to a standard normal space ($Z$) using the inverse standard normal CDF ($\Phi^{-1}$).
   • Privacy Injection: Differential Privacy is applied here using the Laplace mechanism. Noise is added to the histogram bin counts or category counts to privatize the empirical CDFs before transformation.

2. Correlation and Gaussian Sampling:

   • Computes the empirical correlation matrix ($\hat{R}$) of the transformed latent data $Z$.
   • Privacy Injection: Adds symmetric Laplace noise to the correlation matrix entries.
   • PSD Correction: Ensures the noisy matrix remains Positive Semi-Definite (PSD) via eigen-decomposition and thresholding (replacing negative eigenvalues with a small $\delta$).
   • Sampling: Generates independent standard normal samples and applies the Cholesky decomposition of the noisy correlation matrix to induce the learned dependencies.

3. Marginal Reconstruction ($Z_{syn} \to X_{syn}$):

   • Maps the synthetic Gaussian samples back to the unit interval using the standard normal CDF.
   • Applies the inverse empirical CDF (stored during the fit phase) to map values back to the original feature space.
   • Key Feature: Because the marginals are anchored to the original empirical distribution (perturbed only by DP noise), the synthetic data perfectly preserves the original marginal shapes, regardless of the number of regeneration cycles.

Privacy Guarantees

NPGC provides formal $\epsilon$-Differential Privacy by splitting the privacy budget ($\epsilon = \epsilon_m + \epsilon_c$) between marginal estimation and correlation modeling.

3. Key Contributions

• Exact Empirical Marginal Preservation: Unlike deep learning or parametric copula methods, NPGC guarantees that synthetic data matches the observed marginal distributions of real data, including complex shapes (skewness, multi-modality) and discrete mass points.
• Stability in Synthetic Feedback Loops: The method is proven to be robust under iterative regeneration. It prevents the "model collapse" phenomenon where variance shrinks and distributional support is lost over multiple generations.
• Formal Privacy Integration: It incorporates Differential Privacy directly into the statistical estimation of both marginals and correlations, offering provable record-level privacy.
• Handling of Missing Data: It treats missingness as an informative state, preserving the correlation between missing values and other variables.
• Computational Efficiency: As a non-parametric statistical method, it is orders of magnitude faster than deep learning baselines (e.g., TVAE, CTGAN) while requiring no hyperparameter tuning.

4. Experimental Results

The authors evaluated NPGC against four baselines (CTGAN, TVAE, Copula-GAN, Parametric Gaussian Copula) across five UCI benchmark datasets (including two education-specific datasets: Student Dropout and Student Performance).

• Fidelity: NPGC achieved the highest Overall Score (0.962), outperforming all baselines in both univariate (Column Shapes) and bivariate (Pair Trends) agreement. Visualizations showed NPGC perfectly matching empirical densities where deep learning models collapsed modes or distorted shapes.
• Privacy: NPGC demonstrated the strongest privacy performance, with the lowest Discriminator AUC (0.801) and DCR Share (0.527), indicating the synthetic data is hardest to distinguish from real data and least likely to memorize records.
• Utility (TSTR): In "Train-on-Synthetic, Test-on-Real" tasks using XGBoost, NPGC achieved a competitive accuracy drop of 16.99%, outperforming CTGAN and Copula-GAN, though slightly behind TVAE (8.66%). The authors argue that for descriptive analytics and subgroup analysis, NPGC's superior fidelity is more valuable than the marginal gain in predictive accuracy.
• Regeneration Stability: In a 10-step iterative regeneration test, NPGC maintained an Overall Score of 95.27% (a drop of only 1.16%), whereas baselines like TVAE and Copula-GAN degraded significantly (dropping to ~65% and ~49%, respectively).
• Efficiency: NPGC had a training time of 0.65 seconds, compared to >260 seconds for TVAE and >790 seconds for CTGAN.

5. Significance and Real-World Application

• Educational Research Impact: The paper addresses a structural bottleneck in EDM where data cannot be shared due to privacy. NPGC enables the release of statistically valid, privacy-preserving datasets that allow for reproducible research and cross-institutional benchmarking.
• Protection of Rare Subgroups: By preserving exact marginal distributions, NPGC prevents the "erasure" of rare but critical educational signals (e.g., underrepresented student demographics or rare help-seeking behaviors) that often get smoothed out by deep learning models.
• Deployment Case Study: The authors successfully deployed NPGC on a large-scale online learning platform (7,506 interaction logs). The model successfully generated an augmented dataset (10,332 samples) that preserved the extreme class imbalance (e.g., "Hint" events <0.2%) without violating privacy policies, demonstrating practical viability.

Conclusion

The paper introduces NPGC as a lightweight, stable, and privacy-preserving alternative to deep learning for synthetic tabular data. By prioritizing empirical marginal anchoring over complex dependency modeling, it solves the critical issues of distributional drift and model collapse, making it particularly well-suited for the sensitive and heterogeneous nature of educational data.