Technical Summary: CoPFITi – Valid and Expressive Copulas for Irregular Multivariate Time Series

Problem Statement

Irregular Multivariate Time Series (IMTS) are prevalent in domains such as healthcare, climate science, and sensor networks, where variables are observed asynchronously at non-uniform timestamps. Forecasting in these settings requires full probabilistic predictions that capture the joint distribution over queried variables and time points. A critical challenge in IMTS forecasting is marginalization consistency: since queries may involve different subsets of variables and timestamps, a model must ensure that predictions over any subset are consistent with those derived from larger joint predictions.

Existing approaches face a trade-off. Joint models often fail to capture complex dependencies or lack marginalization consistency (e.g., ProFITi). Conversely, models that enforce consistency (e.g., MOSES, CircuITS) often struggle with expressivity or rely on restrictive assumptions. Furthermore, existing copula-based approaches for time series, such as TACTiS and TACTiS-2, fail to strictly preserve univariate marginals by architectural design, violating a fundamental property of copulas. Additionally, standard mixtures of Gaussian copulas are insufficient for modeling complex, multi-modal dependence structures because they share a common center in the latent space.

Methodology

The authors propose CoPFITi (Copulas for Probabilistic Forecasting of Irregular Time Series), a framework that decouples univariate marginal modeling from the joint dependency structure using a Gaussian Mixture Copula (GM-C).

1. Marginal Modeling: MargFlow

To ensure the validity of the copula approach, the authors introduce MargFlow, a dedicated model for univariate marginals.

• Architecture: MargFlow utilizes an encoder (adapted from CircuITS) that is query-separable, meaning the distribution at any query point depends only on the observation history and that specific query point, not on other query points. This property is crucial for marginalization consistency.
• Expressivity: Instead of restricting marginals to Gaussian distributions (as in Neural-ODE baselines), MargFlow employs Deep Sigmoidal Flows (DSF). These are strictly monotone scalar transformations parameterized by neural networks, capable of modeling complex, non-Gaussian marginal distributions.
• Training: MargFlow is trained independently to maximize marginal likelihood, ensuring high-quality univariate predictions before being coupled with the dependency model.

2. Dependency Modeling: Gaussian Mixture Copula (GM-C)

CoPFITi constructs the dependency structure in a latent space using a single Gaussian Mixture Model (GMM) rather than a mixture of separate Gaussian copulas.

• Latent Structure: The model defines a latent joint density $f_{GMM}(z) = \sum_{j=1}^K \pi_j \mathcal{N}(z; \mu_j, \Sigma_j)$ on $\mathbb{R}^N$.
• Parameter Prediction:
  • Mixture Weights ($\pi$): Predicted from a global summary of the observation history $X$ only. They are independent of the query set $Q$.
  • Means ($\mu_j$) and Standard Deviations ($\sigma_{j,n}$): Predicted per-query using the query embedding $e_n$. This allows the latent modes to be spatially separated, enabling the modeling of multi-modal dependencies.
  • Covariances ($\Sigma_j$): Constructed via a low-rank Gram matrix structure ($U_j U_j^\top + I$) derived from query embeddings, scaled by predicted standard deviations. This ensures symmetry and positive definiteness while allowing efficient computation.
• Copula Construction: The copula is induced by the GMM density $f_{GMM}$ via the probability integral transform. Unlike a mixture of Gaussian copulas (which mixes after transformation), CoPFITi mixes the latent densities before transformation. This allows the induced copula to inherit the multi-modal structure of the latent GMM, capturing complex, non-elliptical dependencies that standard mixtures cannot.

3. Marginalization Consistency

CoPFITi is marginalization consistent by construction.

• The mixture weights $\pi$ depend only on history $X$, not the query $Q$.
• The latent parameters $\mu_j$ and $\Sigma_j$ depend on $Q$ only through the specific entries corresponding to the queried points.
• Removing query points corresponds exactly to the standard marginalization rule for multivariate Gaussians.
• The transformation from latent space to target space is query-separable.
• The authors provide a formal proof (Appendix A) demonstrating that these properties guarantee that marginalizing the joint distribution yields the same result as predicting the marginal directly.

4. Training and Inference

• Two-Stage Training: First, MargFlow is trained independently. Then, its weights are frozen, and the copula parameters are trained to maximize the joint log-likelihood.
• Optimization: The objective is the normalized joint negative log-likelihood (njNLL). Since the inverse CDF of the GMM marginal has no closed form, the authors use a safeguarded Newton-bisection scheme for forward inversion and derive analytic gradients (via implicit differentiation) for backpropagation, avoiding the need to unroll the solver.

Key Contributions

1. CoPFITi Framework: The first copula-based framework for IMTS that is marginalization consistent by construction. It utilizes a Gaussian Mixture Copula to achieve high expressivity while maintaining structural validity.
2. MargFlow: A strong, query-separable marginal model using Deep Sigmoidal Flows that achieves state-of-the-art marginal likelihoods on benchmark datasets.
3. Theoretical and Empirical Validity: The paper establishes that decoupling marginals from the joint model yields better marginal performance than joint training. CoPFITi achieves joint likelihoods superior to non-copula baselines and competitive with (or better than) TACTiS-2, while strictly preserving marginal distributions—a property TACTiS-2 fails to guarantee.

Experimental Results

The authors evaluated CoPFITi on four standard IMTS benchmarks: USHCN (climate), PhysioNet-2012, MIMIC-III, and MIMIC-IV (clinical).

• Marginal Performance: MargFlow achieved the lowest Marginal Negative Log-Likelihood (mNLL) across all four datasets, significantly outperforming joint-trained baselines and other univariate models (GRU-ODE, NeuralFlows, CRU).
• Joint Performance: CoPFITi achieved the best or near-best Normalized Joint Negative Log-Likelihood (njNLL). It outperformed TACTiS-2 on PhysioNet, MIMIC-III, and MIMIC-IV.
• Expressivity of GM-C: An ablation study replacing the GM-C with a Mixture of Gaussian Copulas (Mix-GC) showed a significant performance drop, confirming that the ability to model multi-modal latent structures is critical.
• Validity Gap: Using Wasserstein-1 distance, the authors demonstrated that CoPFITi preserves univariate marginals with high fidelity (comparable to sampling noise), whereas TACTiS-2 exhibited orders of magnitude higher deviation, indicating a failure to preserve marginal distributions.
• Consistency Check: A toy experiment on a 3D distribution showed that TACTiS-2's direct bivariate predictions disagreed with its marginalized 3D joint predictions, while CoPFITi's predictions were consistent by construction.

Significance and Claims

The paper claims to establish a new state of the art in joint IMTS density modeling by resolving the tension between expressivity and validity.

• Structural Guarantee: Unlike previous attentional copulas that rely on convergence theory to learn marginal preservation, CoPFITi guarantees marginalization consistency through its architectural design.
• Decoupling Benefit: The results support the hypothesis that separating marginal modeling from dependency modeling leads to more accurate univariate predictions and, consequently, better joint density estimation.
• Applicability: While focused on forecasting, the authors note that the framework naturally extends to other conditional joint density estimation problems with variable target sets, such as missing-value imputation in tabular data.

The work is presented as a methodological contribution evaluated on public benchmarks, with a focus on providing reliable uncertainty estimates and ensuring that predictions at different resolutions do not contradict one another.