🤖 machine learning

GoT-CD: Graph-of-Thoughts Causal Discovery and the Fragility of Post-hoc Path-Specific Fairness Audits

This paper introduces GoT-CD, a Graph-of-Thoughts framework that generates structurally competitive causal graphs while demonstrating that even high-fidelity discovery methods can fail to recover specific pathways necessary for accurate post-hoc path-specific fairness audits, thereby highlighting the critical need to evaluate causal discovery through the lens of downstream fairness requirements.

Original authors: Nitish Nagesh, Elahe Khatibi, Thomas Dean Hughes, Mahdi Bagheri, Pratik Gajane, Amir M. Rahmani

Published 2026-08-05

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Nitish Nagesh, Elahe Khatibi, Thomas Dean Hughes, Mahdi Bagheri, Pratik Gajane, Amir M. Rahmani

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery using only a blurry, black-and-white photo of a crime scene. You can't see the suspects clearly, but you have to figure out who did what to whom. In the world of data science, this is called causal discovery. Instead of running a controlled experiment where you can watch every move, scientists look at "observational data"—like a pile of medical records or traffic logs—and try to draw a map showing which factors cause others. For example, does eating too much sugar cause a headache, or are they just happening at the same time?

Once you have this map, you might want to check if it's fair. This is where path-specific fairness comes in. Imagine a hiring algorithm that rejects candidates. You want to know: did it reject them because of their skills (a legitimate reason), or because of their gender or race (an unfair reason)? To answer this, you need to trace the specific "path" on your map from the person's identity to the final decision. If the map is wrong, your fairness check is useless. You might think the algorithm is fair when it's actually biased, or vice versa. This is the tricky corner of science this paper explores: how do we build the best possible map, and how do we make sure that map doesn't trick us when we try to check for fairness?

The Paper: GoT-CD and the "Fake Clean" Trap

The authors of this paper, Nitish Nagesh and his team, are tackling a problem that sounds simple but is actually a bit of a trap. They noticed that when scientists use Artificial Intelligence (specifically Large Language Models, or LLMs) to draw these causal maps, they usually just check if the map looks "structurally correct." It's like grading a student's drawing of a house by counting how many windows and doors are in the right places. But what if the student drew a perfect house, but forgot to draw the front door? If you're trying to check if the house is accessible for a wheelchair user, that missing door is a disaster, even if the rest of the drawing is perfect.

The paper introduces a new method called GoT-CD (Graph-of-Thoughts Causal Discovery). To understand how it works, imagine a team of three detectives working on the same case, but instead of arguing back and forth, they each write down their entire theory of the crime on a piece of paper at the same time.

The Old Way (Pairwise or Traversal): Previous AI methods were like detectives who only looked at two clues at a time ("Does A cause B?") or who built the story one step at a time, committing to every guess immediately. If they made a mistake early on, the whole story was ruined, and they couldn't go back.
The GoT-CD Way: This method generates multiple complete maps at once. Then, a strict "referee" (a deterministic scoring function) checks them. The best parts of the best maps are merged together, but with a super-important rule: you cannot invent a new connection that no detective proposed. This prevents the AI from making up fake relationships just because they sound plausible. Finally, the team makes sure the map has no loops (you can't have A causing B, which causes C, which causes A), turning it into a clean, one-way street map called a DAG (Directed Acyclic Graph).

The team tested this new method against older, classic math algorithms and other AI methods using five different datasets, including one about Alzheimer's disease. They used a fixed amount of data (100 observations) and a specific AI model (gpt-4o-mini) to keep the playing field level.

The Big Discovery: The "False Clean" Certificate
The most surprising finding isn't just that GoT-CD draws better maps; it's what happens when you use those maps to check for fairness. The researchers focused on an Alzheimer's dataset where they knew for a fact there was an unfair path: Sex → Brain Volume → MOCA Score (a cognitive test). They knew that gender influenced brain volume, which in turn influenced the test score.

Here is the twist:

One of the older AI methods (LLM-BFS) drew a map that looked pretty good overall. It got a structural score (F1) of 0.649, which is decent.
However, when they checked that map for the unfair path, it was completely missing. The map showed no connection between Sex and the test score.
Because the path was missing, the fairness audit reported a result of 0. It looked like a "clean" certificate, saying, "No unfairness detected!"
But this was a lie. The real unfairness was huge (a value of 0.572). The AI didn't find the bias because it forgot to draw the road where the bias lived. It wasn't that the system was fair; it was that the map was broken.

In contrast, GoT-CD drew a map that not only had the best structural score (0.757) but also successfully found that specific unfair path. It reported the correct direction of the effect, showing that the bias was indeed there.

Another method, called GES, found the path but added so many extra fake paths that it made the unfairness look seven times bigger than it really was (a value of 4.034 instead of 0.572).

What This Means for You

The paper suggests that we can't just trust a "good score" on a map to tell us if a system is fair. If the map is missing the specific road that leads to a biased outcome, the fairness check will confidently tell you everything is fine, even when it's not. This is what the authors call a "false-clean certificate."

They argue that if you are using AI to build these maps for things like medical decisions or hiring, you shouldn't just look at the overall accuracy. You need to check specifically: "Did the map find the path we care about?"

The study suggests that GoT-CD is a strong contender because it uses a "team of thinkers" approach to build the map, ensuring it doesn't miss critical connections or invent fake ones. While the results are based on simulations and specific datasets (like the Alzheimer's example), the lesson is clear: in the race to build fair AI, a pretty map isn't enough; you need the right roads drawn on it, or you might accidentally certify a biased system as fair.

Technical Summary: GoT-CD and the Fragility of Post-hoc Path-Specific Fairness Audits

1. Problem Statement

Causal discovery is increasingly utilized in clinical settings to support mechanism reasoning and fairness audits of predictive models. However, a critical gap exists between structural discovery accuracy and the reliability of downstream fairness assessments. Path-specific counterfactual fairness estimands are defined relative to a supplied causal graph; they depend entirely on the existence and structure of directed paths from a protected attribute ( $S$ ) to an outcome ( $Y$ ).

Current evaluation practices for causal discovery rely on aggregate structural metrics (e.g., precision, recall, F1, Structural Hamming Distance) that weight all edges equally. These metrics fail to determine whether the specific sub-structure required for a fairness audit (the $S \to Y$ pathway) survives the discovery process. Consequently, a discovery method may achieve competitive structural F1 scores yet fail to recover the critical unfair pathway. When such a pathway is missing, the fairness audit reports a confident zero effect ( $PSE|_{\cdot}| = 0$ ), which is indistinguishable in the output from a genuinely fair model. This creates a "false-clean" certificate where structural omission masquerades as fairness.

2. Methodology: GoT-CD

The authors introduce GoT-CD (Graph-of-Thoughts Causal Discovery), a method that reframes causal discovery as a full-graph reasoning problem rather than a sequence of local queries. Unlike pairwise approaches (which query variable pairs) or traversal approaches (which expand nodes sequentially), GoT-CD treats the reasoning unit as a complete candidate edge set.

The GoT-CD pipeline executes a single pass of a Graph of Operations:

Generate: The Large Language Model (LLM) is prompted with variable names, descriptions, and a domain preamble to generate $k$ distinct candidate graphs (edge sets) in parallel.
Score: A deterministic, non-LLM validity function scores candidates. It penalizes undeclared variables and self-loops, rewards well-formed edges, applies a soft density prior, and adds a bonus for acyclicity.
KeepBestN: The top $N$ candidates (typically $N=2$ ) are retained.
Aggregate: The retained candidates are merged under a hard union constraint. The model is instructed to emit only edges that appeared in at least one input candidate. This prevents the model from synthesizing "plausible-sounding" edges that no reasoning branch proposed.
Improve: The merged graph is refined. The model may introduce new edges only if they do not create cycles with the already-accepted core.
Greedy Projection: A final deterministic step enforces acyclicity by traversing edges and retaining them only if they do not form a directed cycle with previously kept edges. This ensures the output is always a Directed Acyclic Graph (DAG).

The method uses a locked protocol with $n=100$ observations and the gpt-4o-mini backbone.

3. Key Contributions

The paper makes three primary contributions:

GoT-CD Algorithm: A causal discovery method where the reasoning unit is a complete candidate edge set, utilizing parallel generation, deterministic scoring, and a hard union constraint to prevent edge invention while enforcing acyclicity.
Comprehensive Benchmarking: A comparison of GoT-CD against four classical baselines (PC, GES, NOTEARS, DAGMA-linear) and three LLM baselines (Pairwise, LLM-BFS, GoT-CD-BFS) across five benchmarks (Asia, Child, Alzheimer's, COVID-Respiratory, Sweden-Traffic).
Post-hoc Path-Specific Fairness Audit: An evaluation framework applied to the Alzheimer's benchmark (where the ground-truth unfair path is $Sex \to Brain Volume \to MOCA Score$ ) to score methods not just on structural F1, but on the verbatim recovery of the specific $S \to Y$ pathway.

4. Results

Under a locked $n=100$ protocol:

Structural Performance: GoT-CD produces valid DAGs on all five benchmarks. It achieves the best DAG-valid F1 among LLM methods on Asia (0.750), Alzheimer's (0.757), and COVID-Respiratory (0.688), outperforming LLM-BFS on four of five datasets. It competes favorably with classical methods, though continuous-relaxation methods (NOTEARS, DAGMA) dominate on the linear-Gaussian Sweden-Traffic dataset.
DAG Validity: Unlike PC (cyclic on all datasets) and GES (cyclic on four), GoT-CD (and GoT-CD-BFS) returns DAGs consistently, avoiding the need for heuristic cycle-breaking prior to fairness analysis.
Fairness Fidelity (Alzheimer's Benchmark):
- GoT-CD: Recovers the ground-truth unfair path with the correct effect sign ( $TE_{lin} = -0.580$ vs. benchmark $-0.572$) and leads in structural F1.
- LLM-BFS: Despite a competitive structural F1 (0.649), it fails to recover any $S \to Y$ path. Consequently, it reports $PSE|_{\cdot}| = 0$ , producing a false-clean fairness certificate.
- GES: Recovers the unfair path but inflates the total path mass more than sevenfold ( $PSE|_{\cdot}| = 4.034$ ) due to eight additional spurious routes.
- Other Methods: PC, NOTEARS, DAGMA-linear, and GoT-CD-BFS all fail to recover the $S \to Y$ path, reporting zero effects.

5. Significance and Claims

The paper argues that structural fidelity does not imply fairness fidelity. A method can be structurally accurate (high F1) yet fail to recover the specific edges required for a fairness audit, leading to confident but incorrect "fair" conclusions.

The authors claim that pipelines chaining causal discovery to path-specific fairness must:

Report Pathway Recovery: Alongside the fairness estimate, pipelines must explicitly report whether the critical $S \to Y$ sub-structure was recovered.
Re-evaluate Discovery Metrics: Discovery methods intended for fairness pipelines should be evaluated on the edges the downstream analysis depends on, rather than solely on aggregate structural agreement.
Leverage Global Reasoning: The results suggest that global reasoning (GoT) is superior to local traversal (BFS) or pairwise querying for preserving fairness-relevant structures, as it avoids the "edge flooding" of pairwise methods and the "irrevocable error propagation" of traversal methods.

The paper concludes that while LLM priors are effective on knowledge-rich, compact graphs (like Alzheimer's), they complement rather than replace statistical methods (which excel on linear-Gaussian data), and that future work should explore score-guided variants and fairness-aware discovery objectives.