Causal Identification from Counterfactual Data: Completeness and Bounding Results

This paper introduces the CTFIDU+ algorithm to establish the completeness of identifying counterfactual queries from physically realizable Layer 3 data, defines the fundamental limits of exact causal inference in this setting, and derives novel analytic bounds for non-identifiable quantities that are empirically shown to be tighter with counterfactual data.

The Gist

Imagine you are a detective trying to solve a crime, but you only have access to three types of clues:

1. The Scene (Observation): You see what happened naturally. (e.g., "The suspect was wearing a red hat and ran away.")
2. The Experiment (Intervention): You force a change and see what happens. (e.g., "We put a blue hat on the suspect and see if they still run away.")
3. The "What If" (Counterfactual): You imagine a different reality. (e.g., "If the suspect had worn a blue hat instead of the red one, would they have run away?")

For decades, scientists believed Type 3 clues were impossible to get in the real world. You can't go back in time and change the past. So, they built a "Causal Hierarchy" (a ladder of knowledge) where you could climb from Type 1 to Type 2, but Type 3 was considered a mythical peak you could never reach.

This paper is about breaking that ceiling.

The Big Breakthrough: "Time-Traveling Cameras"

The authors (Arvind Raghavan and Elias Bareinboim) start with a surprising discovery from their previous work: You actually can get Type 3 data in the real world.

They call this "Counterfactual Realizability."

The Analogy: Imagine a traffic camera filming a speeding car.

• Standard Experiment: You stop the car, paint it blue, and let it drive. This changes the car's actual color.
• Counterfactual Experiment: You use a special "digital filter" on the video feed. You tell the AI analyzing the video, "Pretend this car is blue," but you don't actually change the car's paint or the driver's behavior. The AI sees a blue car, but the real world remains unchanged.

This allows you to collect data about "what would have happened if the car were blue" without actually changing the car. It's like having a camera that can see parallel universes.

The New Detective Tool: CTFIDU+

Now that we have these "Time-Traveling Cameras," the big question is: What new mysteries can we solve?

Previously, algorithms could only solve mysteries using Type 1 and Type 2 data. The authors built a new super-algorithm called CTFIDU+.

• What it does: It takes a mix of regular data, experiment data, and this new "parallel universe" data to solve complex "What If" questions.
• The Guarantee: They proved that this algorithm is complete. This means if a "What If" question can be solved using the data you have, CTFIDU+ will find the answer. If it says "I can't solve this," then it is truly impossible to solve with that data.

The "Glass Ceiling" of Knowledge

Here is the most fascinating part. The authors asked: "If we have these time-travel cameras, can we solve every 'What If' question?"

The answer is No.

They discovered a fundamental limit. Even with these amazing cameras, there is a "Glass Ceiling" (which they call Layer 2.5).

• Below the ceiling: You can solve these problems by combining your data.
• Above the ceiling: There are some "What If" questions that are fundamentally unknowable, even with perfect experiments.

The Metaphor: Imagine trying to figure out the exact recipe of a cake by tasting it.

• If you can taste the cake (Observation) and bake a few test batches (Intervention), you can guess the recipe.
• If you can also magically taste the cake while imagining you added extra sugar (Counterfactual), you can guess even better.
• But: If the recipe depends on a secret ingredient that was never used in any version of the cake you've ever seen or imagined, you will never know it. No amount of time travel or magic tasting can reveal a secret that leaves no trace in reality.

The paper proves that some "What If" questions fall into this "secret ingredient" category. They are mathematically impossible to pin down exactly.

Why Does This Matter? (The "Tighter Bounds" Trick)

Even if we can't solve a mystery exactly, does this new data help? Yes!

The authors show that even for the "unsolvable" mysteries, using this new counterfactual data allows us to draw a much smaller circle around the answer.

The Analogy:

• Old Way (No Counterfactual Data): "The suspect's speed was somewhere between 0 and 100 mph." (Useless!)
• New Way (With Counterfactual Data): "The suspect's speed was somewhere between 45 and 55 mph." (Much more useful!)

They proved mathematically that adding this "parallel universe" data shrinks the range of possible answers, making our guesses much sharper, even if we can't get the single perfect number.

Summary

1. We can get "What If" data: We don't need to break physics; we just need to use clever experimental setups (like digital filters on video) to simulate alternate realities.
2. We have a perfect tool: The CTFIDU+ algorithm can tell us exactly which "What If" questions are solvable with our data and which are not.
3. There is a limit: Some questions are fundamentally impossible to answer exactly, no matter how much data we collect.
4. But we can still improve: Even for the impossible questions, this new data helps us narrow down the answer significantly, turning a wild guess into a precise estimate.

This work changes the game for AI, fairness, and medicine, giving us a roadmap for what we can know about the past and future, and where we must simply accept uncertainty.

Technical Summary

1. Problem Statement

The paper addresses a fundamental gap in causal inference regarding Counterfactual Identification (Layer 3 of Pearl's Causal Hierarchy).

• Context: Traditionally, counterfactual identification algorithms (e.g., IDC*, CTFID) assumed that input data was limited to observational (Layer 1) and interventional (Layer 2) distributions. It was widely presumed that Layer 3 data (counterfactual distributions) could not be directly observed, only inferred.
• The Shift: Recent work (Raghavan & Bareinboim, 2025) established that certain counterfactual distributions are physically realizable via a procedure called Counterfactual Randomization (ctf-rand). This allows experimenters to directly sample from specific Layer 3 distributions (e.g., observing an outcome under a hypothetical treatment while keeping the natural value of a confounder fixed).
• The Core Question: Given this new access to some Layer 3 data, what additional counterfactual quantities become identifiable? What are the theoretical limits of exact identification in this non-parametric setting? And for quantities that remain non-identifiable, can this new data tighten the bounds?

2. Methodology

The authors develop a comprehensive framework combining algorithmic identification, theoretical limits, and partial identification bounds.

A. The CTFIDU+ Algorithm (Complete Identification)

The paper introduces CTFIDU+, a new algorithm designed to identify counterfactual queries from an arbitrary set of input distributions, including realizable Layer 3 data.

• Input: A causal diagram $G$, a counterfactual query $P(Y^* = y)$, and a set of input distributions $\mathcal{A}$ (indexed by actions like observation, standard randomization rand(), or counterfactual randomization ctf-rand()).
• Mechanism:
  1. Un-nesting: Converts nested counterfactuals into un-nested terms using the Counterfactual Un-nesting Theorem.
  2. Ancestral Expansion: Expands the query into its set of counterfactual ancestors.
  3. Factorization: Decomposes the query into smaller counterfactual factors (ctf-factors) based on the graph's confounding structure (c-components).
  4. Sub-routine (IDENTIFY+): A novel sub-routine that attempts to identify a target ctf-factor from a given input ctf-factor. It generalizes the classic IDENTIFY algorithm (Tian & Pearl, 2003) to handle Layer 3 inputs.
  5. Hedge Detection: If identification fails, the algorithm detects a specific structure called a Counterfactual Hedge (a generalization of the "hedge" structure used in Layer 2 identification), which serves as a certificate of non-identifiability.
• Completeness: The authors prove that CTFIDU+ is complete: it returns a valid expression if and only if the query is identifiable from the given input data and graph.

B. Theoretical Limits: The Duality of Identifiability and Realizability

The paper establishes a foundational duality between Counterfactual Identifiability and Counterfactual Realizability.

• Layer 2.5 ($L_{2.5}$): The authors define a new layer in the Causal Hierarchy, $L_{2.5}$, which contains all counterfactual distributions that are physically realizable via ctf-rand() actions.
• The Limit Theorem: They prove that no purely Layer 3 quantity (outside $L_{2.5}$) is identifiable from $L_{2.5}$ data.
  • If a query belongs to $L_{2.5}$, it is identifiable from lower-layer data (or itself).
  • If a query belongs to $L_3 \setminus L_{2.5}$ (e.g., certain nested counterfactuals involving conflicting regimes on the same variable), it is fundamentally non-identifiable, even with access to all physically realizable counterfactual data.
• Duality: A counterfactual quantity is identifiable from experimental/observational data if and only if it is, in principle, realizable via counterfactual randomization actions.

C. Partial Identification and Bounding

For quantities that are non-identifiable (specifically those in $L_3 \setminus L_{2.5}$), the paper derives novel analytic bounds using realizable counterfactual data.

• Premise: While exact point identification is impossible for some quantities, adding $L_{2.5}$ data (via ctf-rand) constrains the space of compatible Structural Causal Models (SCMs).
• Result: The authors derive tighter bounds for the Natural Total Effect (NTE) and other probabilities of causation. They prove mathematically that bounds derived using $L_{2.5}$ data are strictly tighter (or equal) to those derived using only $L_2$ data.
• Simulation: Using Bayesian sampling over SCMs, they demonstrate that in practice, access to counterfactual data significantly narrows the credible intervals for non-identifiable quantities.

3. Key Contributions

1. CTFIDU+ Algorithm: A complete algorithm for identifying counterfactual queries from arbitrary sets of physically realizable data (including Layer 3). It subsumes previous algorithms (IDC*, PSIDC, CTFID) which were limited to Layer 2 inputs.
2. Completeness Proof: A rigorous proof that CTFIDU+ succeeds if and only if the query is identifiable, utilizing a new "Counterfactual Hedge" structure to certify non-identifiability.
3. Fundamental Limit Theorem: The proof that exact causal inference in non-parametric settings has a hard ceiling at Layer 2.5. Quantities beyond this layer cannot be identified even with perfect counterfactual randomization capabilities.
4. Identifiability-Realizability Duality: A theoretical result stating that a query is identifiable iff it is realizable. This bridges the gap between what can be computed from data and what can be physically measured.
5. Tighter Bounds for Non-Identifiable Quantities: Novel analytic bounds for the Natural Total Effect (NTE) and other metrics that leverage counterfactual data to reduce uncertainty, validated by simulations.

4. Results

• Algorithmic Performance: CTFIDU+ successfully retrieves known formulas (e.g., Front-Door adjustment) and solves complex queries where previous methods (limited to $L_2$) would return FAIL.
• Theoretical Boundary: The paper confirms that quantities like the Natural Total Effect (NTE) in certain graph structures (e.g., the "bow graph") are in $L_3 \setminus L_{2.5}$ and thus fundamentally non-identifiable.
• Bounding Efficacy:
  • In Example 2 (Traffic Camera), using $L_{2.5}$ data narrowed the 95% credible interval for the NTE significantly compared to using only $L_2$ data.
  • In Example 3 (Unit Selection), the counterfactual strategy allowed for sub-population specific decisions that strictly dominated standard interventional strategies, proving that even non-identifiable quantities can yield actionable insights when bounds are tightened.

5. Significance

• Paradigm Shift: The paper challenges the dogma that counterfactuals are purely theoretical constructs. It formalizes a pathway to empirically estimate counterfactual distributions through specific experimental designs (ctf-rand).
• Practical Application: For fields like Algorithmic Fairness, Explainable AI (XAI), and Personalized Medicine, where counterfactual reasoning is crucial (e.g., "Would this driver have received a ticket if their car color were different?"), this work provides the tools to determine exactly what can be inferred and how to design experiments to get the tightest possible answers.
• Experimental Design: It offers a blueprint for researchers to design experiments that maximize information gain, moving beyond standard Randomized Controlled Trials (RCTs) to counterfactual randomization when feasible.
• Theoretical Foundation: By defining the $L_{2.5}$ layer and the duality of identifiability, the paper sets a new standard for understanding the limits of causal inference in non-parametric settings, guiding future research on when stronger assumptions (parametric or structural) are necessary to overcome identification barriers.