Identifiable Deep Latent Variable Models for MNAR Data

1. Problem Statement

Missing data is a pervasive challenge in statistical analysis. Traditional imputation methods typically rely on the Missing-At-Random (MAR) assumption, where the probability of a value being missing depends only on observed data. However, in many real-world scenarios (e.g., medical records, surveys, recommendation systems), data is Missing-Not-At-Random (MNAR), meaning the missingness mechanism depends on the unobserved values themselves.

While deep generative models (e.g., VAEs, not-MIWAE) have been applied to MNAR data, they suffer from a critical theoretical flaw: non-identifiability. Without specific constraints, multiple distinct complete-data distributions can generate the same observed data distribution, leading to biased and unreliable imputations. Existing solutions often require auxiliary variables (which may not exist) or impose rigid parametric assumptions that risk model misspecification.

2. Methodology: IM-IWAE

The authors propose IM-IWAE (Identifiable Missing-Not-At-Random Importance-Weighted Autoencoder), a framework that combines deep latent variable models with a specific structural assumption to ensure nonparametric identifiability.

A. Theoretical Foundation: Conditional No Self-Censoring

The core innovation is a new assumption regarding the missingness mechanism, termed Conditional No Self-Censoring given Latent Variables:

• Assumption: For any variable $X_j$, its missingness indicator $R_j$ is conditionally independent of $X_j$ and other missingness indicators $R_{-j}$, given all other observed variables $X_{-j}$ and a latent variable $\tilde{Z}$.
• Mathematical Formulation: $R_j \perp\!\!\!\perp (X_j, R_{-j}) \mid X_{-j}, \tilde{Z}$.
• Significance: This assumption allows the model to capture correlations among missingness indicators via the latent variable $\tilde{Z}$ while breaking the direct dependency between a variable and its own missingness. The authors prove that under this assumption (along with a positivity condition), the true full-data distribution is nonparametrically identifiable.

B. Model Architecture

The model decomposes the joint distribution of data $X$ and missingness indicators $R$ into two latent variable components:

1. Data Generation: $X$ is generated from a latent variable $Z$ (standard VAE structure).
2. Missingness Mechanism: $R$ is generated from $X$ and a separate latent variable $\tilde{Z}$. Crucially, the decoder for $R_j$ depends on $X_{-j}$ and $\tilde{Z}$, but explicitly excludes $X_j$.

The joint distribution is modeled as:
$$p_{\theta, \psi}(x, r) = \int \int p_\theta(x|z) p_\psi(r|x, \tilde{z}) p(z) p(\tilde{z}) \, dz \, d\tilde{z}$$

C. Estimation Algorithm

To handle the intractable integrals over missing and latent variables, the authors employ Importance-Weighted Autoencoders (IWAE):

• Objective: Maximize the log-likelihood of observed data using a tighter lower bound (ELBO) than standard VAEs.
• Variational Inference: They introduce variational posteriors $q_\phi(z|x_{obs}, r)$ and $q_\lambda(\tilde{z}|x_{obs}, r)$ to approximate the true posteriors.
• Optimization: The objective function uses $K$ importance samples to estimate the likelihood, reducing bias as $K$ increases. The parameters are optimized via stochastic gradient descent with reparameterization tricks.

D. Imputation and Generation

• Imputation: Missing values are imputed by estimating the conditional mean $E[X_{miss} | X_{obs}, R]$ using self-normalized importance sampling.
• Generation: The model can generate synthetic complete data by sampling from the learned latent distributions, allowing for the recovery of the true underlying data distribution.

3. Key Contributions

1. Nonparametric Identifiability: The paper establishes the first theoretical proof of nonparametric identifiability for MNAR data within a deep latent variable framework, without requiring auxiliary variables or specific parametric forms for the ground truth.
2. Novel Assumption: The introduction of "Conditional No Self-Censoring given Latent Variables" bridges the gap between theoretical identifiability conditions (like no self-censoring) and the practical requirements of deep generative models.
3. IM-IWAE Algorithm: A practical, scalable algorithm that jointly learns the data distribution and the missingness mechanism, capable of recovering the ground-truth distribution under mild regularity conditions.
4. Robustness: The method demonstrates robustness even when the "no self-censoring" assumption is slightly violated (e.g., in the presence of some self-censoring), likely due to the correlations captured by the latent variables.

4. Experimental Results

The authors evaluated IM-IWAE against state-of-the-art methods (GINA, not-MIWAE) and classical baselines (MICE, missForest) across simulations and real-world datasets.

• Simulations:

  • Imputation Accuracy: IM-IWAE consistently achieved the lowest Root Mean Squared Error (RMSE) in imputing missing values across various MNAR scenarios (linear/nonlinear, with/without latent variables).
  • Distribution Recovery: In terms of Maximum Mean Discrepancy (MMD), IM-IWAE generated data distributions that were significantly closer to the ground truth than GINA or not-MIWAE, which often suffered from bias due to non-identifiability.
  • High Dimensions: The method remained stable in high-dimensional settings (up to 100 variables) with block-wise missingness, whereas other MNAR-specific models degraded significantly.
  • Parameter Estimation: In a Gaussian mixture setting, IM-IWAE accurately recovered the true mean of variables, whereas other methods produced biased estimates.

• Real-World Data:

  • UCI Datasets: On six UCI datasets with synthetic MNAR mechanisms, IM-IWAE outperformed all competitors, even when the missingness mechanism included self-censoring (a violation of the assumption), demonstrating practical robustness.
  • HIV Data (Botswana): Applied to a dataset of HIV-positive mothers, the model estimated the joint distribution of HAART usage, CD4 counts, and preterm delivery. The results were comparable to semi-parametric methods (AIPW) and distinct from Complete Case Analysis, validating its utility in epidemiological studies.
  • Yahoo! R3 Music Ratings: On a massive, sparse user-rating dataset (98% missing), IM-IWAE produced imputed distributions that closely matched the unbiased test set, outperforming GINA and not-MIWAE in recovering the true rating distribution.

5. Significance and Conclusion

This paper addresses a fundamental limitation in deep learning for missing data: the lack of identifiability in MNAR settings. By proving that a specific conditional independence assumption leads to identifiability, the authors provide a theoretical guarantee that their model can recover the true data distribution.

Significance:

• Theoretical: It moves MNAR modeling from heuristic approaches to a theoretically grounded framework.
• Practical: It offers a robust tool for domains where missingness is likely informative (e.g., healthcare, social sciences) without requiring auxiliary data.
• Generalizability: The method works effectively even when the exact missingness mechanism is unknown or slightly misspecified, making it a viable solution for real-world applications where assumptions are rarely perfect.

The authors conclude that IM-IWAE sets a new standard for MNAR imputation, balancing theoretical rigor with the flexibility of deep generative models.