Technical Summary: Computational Identifiability

Problem Statement

Traditional causal identifiability, termed "theoretical identifiability," relies on mathematical derivations to prove the existence of a unique causal parameter given a set of assumptions (e.g., causal graphs, probability axioms). This approach generally assumes asymptotic properties, infinite data, or other idealized conditions. While sound and complete algorithms exist for specific model classes (e.g., do-calculus for recursive semi-Markovian SCMs), they often fail in practical scenarios involving:

• Finite sample sizes.
• Ambiguous graphical criteria where multiple adjustment sets exist but their optimality depends on unknown structural parameters.
• Complex mixtures of observational and interventional data.
• Settings requiring parametric assumptions (e.g., linearity, non-Gaussianity) that are difficult to verify analytically.

Consequently, theoretical guarantees often offer little guidance on whether a target parameter can be actually estimated in a specific, finite-data setting.

Methodology: Computational Identifiability

The authors propose computational identifiability, a framework that redefines identifiability as the success of a finite computational search procedure for an empirical estimator, rather than the existence of a unique analytical solution.

Core Definitions

1. Causal Mixture Distribution: The framework defines a weighted sum of empirical distributions (from observational, interventional, or counterfactual data) and theoretical distributions derived from a Structural Causal Model (SCM). This allows for the integration of mixed data sources.
2. Computational Identifiability (Definition 4): A causal query $\Theta[C, x]$ is considered $\epsilon$-$\delta$-identifiable conditional on a prior $\pi$ over SCMs and a hypothesis space $\mathcal{H}$ if there exists a function $\phi \in \mathcal{H}$ such that the probability of the estimator's error being within tolerance $\epsilon$ is at least $1-\delta$.
   • Unlike theoretical identifiability, this is conditional on the prior (distribution over SCMs), the hypothesis space (the class of estimators), the sample size, and the error tolerance.
3. Meta-Learning Approach: To operationalize this, the authors treat identifiability as a meta-learning problem. They define a joint distribution over causal mixture samples and causal query values. A meta-learner (specifically Conditional Neural Processes or Set Transformers) is trained to map observations and query points to causal query values.
4. Estimation of Identifiability: The probability of identifiability is estimated empirically by training the meta-model on a distribution of SCMs and evaluating its performance (e.g., RMSE or quantile regression accuracy) on a test set. The authors utilize quantile regression (Q-CNP and Q-TNP architectures) to handle skewed or multi-modal treatment effect distributions and to estimate the probability that the error falls within a specific tolerance.

Key Components

• Prior Conditioning: The distribution of the target parameter must be sufficiently concentrated around a unique value given the available information.
• Hypothesis Conditioning: An estimator must exist within the chosen hypothesis space that can predict the target parameter.
• Approximation: The goal is prediction within an acceptable error tolerance, not necessarily perfect point identification.

Key Contributions

1. Introduction of Computational Identifiability: A practical, computation-bound notion of identifiability that complements existing theoretical definitions. It frames identifiability as a search for a successful estimator within a finite computational budget.
2. Formalization of Meta-Learning for Causal Inference: The paper generalizes the connection between causal effect estimation and meta-learning, defining the necessary components (hypothesis space, error tolerance, confidence bounds, and mixture distributions) to compute and evaluate computational identifiability.
3. Empirical Demonstration: The framework is applied to complex, small finite-sample settings where theoretical guarantees are insufficient or ambiguous.

Experimental Results

The authors validate the framework across three distinct scenarios:

1. Optimal Adjustment Set Selection

In settings where multiple valid adjustment sets exist (e.g., $Z_0 = \emptyset$, $Z_1 = \{Z_1\}$, $Z_{12} = \{Z_1, Z_2\}$), theoretical criteria often cannot determine which is optimal without knowing structural parameters.

• Result: The meta-model successfully identified the "conditional optimal set" (O-set) based on the prior over structural equations. For linear models with uniform priors, $Z_{12}$ was optimal; for random MLP-based structural equations, $Z_0$ was optimal. This demonstrates the ability to distinguish between theoretically equivalent settings based on empirical performance.

2. Transportability with Mixed Data

The study examined identifying causal effects under hidden confounding using a mix of observational and interventional data where the treatment distributions differed.

• Result: Computational identifiability revealed a non-monotonic relationship between data composition and performance. While interventional data generally helped, a dataset consisting entirely of interventional data (with a mismatched treatment distribution) performed worse than having no interventional data. The framework quantified the optimal mix of data types to maximize identifiability probability.

3. Counterfactual Identification (CATE vs. ITE)

The authors compared the identifiability of Conditional Average Treatment Effects (CATE) and Individual Treatment Effects (ITE) in the presence of unobserved moderators and noise.

• Result:
  • CATE: Identifiable in all settings, requiring neither interventional nor counterfactual data.
  • ITE: Required counterfactual data for identification. Interventional data alone did not help and could degrade performance in noiseless settings.
  • Architecture Dependence: Identifiability was shown to be conditional on the hypothesis space. The Q-TNP (Set Transformer) architecture successfully learned to match individual datapoints for ITE recovery, whereas the Q-CNP (mean-pooled) failed, highlighting that "identifiability" is relative to the chosen estimator class.
  • Non-Monotonicity: Increasing dataset size improved CATE estimation but sometimes degraded ITE estimation, contradicting standard theoretical expectations and underscoring the conditional nature of the proposed framework.

Significance and Claims

The paper claims that computational identifiability provides an actionable and empirical alternative to theoretical identifiability. Its primary significance lies in:

• Practicality: It answers fine-grained questions about identifiability in settings with small samples, ambiguous graphs, and mixed data types where analytical solutions are intractable.
• Conditional Nature: It explicitly acknowledges that identifiability is not an absolute property of a model but is conditional on the prior, the hypothesis space, the sample size, and the error tolerance.
• Systematic Search: It offers a pathway to systematically search for identification algorithms and partial bounds in novel settings where theoretical results do not yet exist.

The authors maintain a modest stance, noting that this framework is a complement to, not a replacement for, theoretical identifiability. They suggest future work could extend these definitions to survival analysis, missing data, and high-dimensional settings, and further automate the search process.