Information-Preserving Domain Transfer with Unlabeled Data in Misspecified Simulation-Based Inference

Technical Summary: Information-Preserving Domain Transfer with Unlabeled Data in Misspecified Simulation-Based Inference

Problem Statement
Simulation-based inference (SBI) enables amortized Bayesian parameter inference from simulator-generated data without requiring explicit likelihood evaluation. However, the reliability of neural SBI methods, such as Neural Posterior Estimation (NPE), degrades under model misspecification—situations where real-world observations are not well represented by the simulator used for training. In such cases, direct inference on real-world data can yield biased or overconfident posteriors.

Existing approaches to mitigate misspecification often rely on unlabeled real-world data to align the marginal distributions of simulated and real observations (e.g., via Maximum Mean Discrepancy or domain-adversarial training). However, the authors argue that marginal alignment alone is insufficient. It does not guarantee the preservation of the conditional structure relating observations to task variables (parameters), potentially discarding information critical for posterior inference. Furthermore, many existing methods require a small calibration set of real-world data with known parameters, which is often unavailable.

Methodology: SPIN
The paper proposes SPIN (Simulation-based inference with Parameter-relevant Information-preserving domain transfer), a framework designed for NPE under model misspecification using only unlabeled, unpaired real-world observations.

• Core Mechanism: SPIN learns bidirectional transport maps between the simulator domain ($X_s$) and the real-world domain ($X_r$). Let $G_{sr}: X_s \to X_r$ map simulator observations to the real-world domain, and $G_{rs}: X_r \to X_s$ map real-world observations back to the simulator domain.
• Inference Protocol: At test time, a real-world observation $x_r$ is mapped to the simulator domain via $x_{rs} = G_{rs}(x_r)$. The posterior is then estimated using the simulator-trained NPE model: $\hat{p}(\theta | x_r) = q_\psi(\theta | x_{rs})$.
• Information-Preserving Objective: The key innovation is a training objective that ensures the transport preserves parameter-relevant information. Since real-world parameter labels are unavailable, SPIN utilizes the simulator labels during training.
  1. A labeled simulator pair $(\theta, x_s)$ is translated to the real-world domain ($x_{sr} = G_{sr}(x_s)$) and then mapped back to the simulator domain ($x_{srs} = G_{rs}(x_{sr})$).
  2. Because $x_{srs}$ originates from a labeled sample, the original parameter $\theta$ is known.
  3. SPIN optimizes a variational lower bound on the mutual information (MI) between the parameter $\theta$ and the transported observation $x_{srs}$. This is achieved by minimizing the negative log-likelihood of the original parameter given the transported observation:
     $$\mathcal{L}_{info}(q) = \mathbb{E}_{(\theta, x_s) \sim p_s} [-\log q(\theta | G_{rs}(G_{sr}(x_s)))]$$
  4. This loss encourages the transport maps to retain information about the parameter, rather than merely matching marginal distributions.
• Training Procedure: The framework jointly trains the transport networks ($G_{sr}, G_{rs}$), discriminators (for adversarial domain alignment), and the posterior estimator ($q_\psi$). The total generator loss combines adversarial translation, cycle-consistency, identity regularization, and the information-preservation term ($\mathcal{L}_{info}$).

Key Contributions

1. Information-Preserving Domain Transfer: Unlike prior domain alignment methods that focus on marginal distribution matching, SPIN explicitly constrains domain transfer to preserve the mutual information between the parameter of interest and the transported observations.
2. Unlabeled, Unpaired Adaptation: SPIN requires only simulator labels for training. It adapts NPE using unlabeled, unpaired real-world observations without needing a calibration set of real-world data with known parameters.
3. Theoretical Justification: The authors provide a population identity showing that if the learned transport satisfies a parameter-preserving joint-transport condition, the real-world test-time risk equals the simulator-labeled transported risk minimized by $\mathcal{L}_{info}$.

Experimental Results
The authors evaluated SPIN on four benchmarks:

• SIR: A weakly misspecified synthetic benchmark based on an epidemic model with delayed reporting.
• Pendulum: A synthetic benchmark introducing damping to a frictionless pendulum model.
• Wind Tunnel: A controlled physical benchmark using a wind-chamber system where the mechanistic model approximates real chamber responses.
• Light Tunnel: A controlled physical benchmark using an optical imaging system with simplified modeling of light-tunnel responses.

Findings:

• Performance: Across synthetic and physical benchmarks, SPIN consistently improved main inference-quality metrics (Root Mean Squared Error, Log Posterior Probability, and ACAUC) compared to standard NPE and marginal alignment baselines (NPE-MMD, NPE-DANN).
• Impact of Misspecification: The improvements of SPIN became more pronounced as the degree of model misspecification increased. For instance, in the Pendulum benchmark, as the damping coefficient (misspecification strength) increased, SPIN maintained superior posterior density and accuracy compared to baselines.
• Ablation: Removing the information-preservation term ($\mathcal{L}_{info}$) resulted in degraded performance, particularly in tasks with larger simulator-reality gaps. This confirms that the gains are driven by the preservation of parameter-relevant information, not just domain alignment.
• Data Efficiency: SPIN's performance depends on the amount of unlabeled real-world data ($N_r$), with larger datasets providing more stable transport learning, though gains were observed even with moderate data sizes.

Significance and Claims
The paper claims that for misspecified SBI, reducing marginal discrepancy between simulated and real data is necessary but insufficient. The authors argue that domain transport must explicitly preserve the information relevant to the parameter of interest. SPIN demonstrates that by leveraging simulator labels to enforce information preservation during the transport of unlabeled real-world data, one can achieve more robust and accurate posterior inference. The method is particularly effective when the gap between the simulator and reality is significant, offering a practical solution for settings where real-world parameter labels are unavailable.