📊 statistics

Incorporating Expert Knowledge into Bayesian Causal Discovery of Mixtures of Directed Acyclic Graphs

This paper proposes a novel framework combining Bayesian experimental design and variational mixture structure learning to incorporate expert knowledge into the discovery of heterogeneous causal structures, enabling the inference of mixtures of causal Bayesian networks that outperform existing methods on both synthetic and real-world breast cancer data.

Original authors: Zachris Björkman, Jorge Loría, Sophie Wharrie, Samuel Kaski

Published 2026-04-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zachris Björkman, Jorge Loría, Sophie Wharrie, Samuel Kaski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out how a complex machine works by looking at a pile of mixed-up parts. Usually, scientists try to find one single blueprint that explains how every part connects to every other part. But what if the machine isn't one thing? What if it's actually two different machines mixed together in the same box, each with its own unique blueprint?

This is the problem the paper tackles. It's about Causal Discovery—figuring out cause-and-effect relationships (like "smoking causes cancer" or "rain causes wet grass").

Here is a simple breakdown of what the authors did, using everyday analogies:

1. The Problem: The "One Blueprint" Trap

Most computer programs that try to figure out how things work assume there is only one correct answer (one single graph).

The Analogy: Imagine you have a bag of LEGOs from two different sets: a castle and a spaceship. If you try to build a single model that explains how all the bricks fit together, you'll end up with a weird, broken mess. You can't have a castle turret that is also a spaceship engine.
The Reality: In real life (like in medicine or biology), data often comes from different "subgroups." Some patients might have one type of disease mechanism, while others have a different one. A single model tries to force a square peg into a round hole, resulting in a confusing, inaccurate map.

2. The Solution: The "Mixture" Approach

The authors created a new method called VaMSL (Variational Mixture Structure Learning).

The Analogy: Instead of trying to build one giant, confused model, their program says, "Okay, let's sort these LEGOs into two piles." It then builds two separate blueprints: one for the castle and one for the spaceship.
How it works: It automatically groups similar data points together and learns a different cause-and-effect map for each group.

3. The Secret Ingredient: The "Expert Consultant"

Usually, computers need massive amounts of data to figure out these blueprints. But what if you don't have enough data? That's where Expert Knowledge comes in.

The Analogy: Imagine you are a detective trying to solve a crime, but you only have a few clues. You call in a Consultant (a domain expert) who knows the neighborhood well.
The Old Way: You might ask the consultant, "Did the butler do it?" and they say "Yes." You just take their word for it.
The New Way (This Paper): The authors use a smart strategy called Bayesian Experimental Design. Instead of asking random questions, the computer calculates: "Which single question will teach me the most?"
- It asks the expert: "Between these two specific connections, which one is more likely to be true?"
- The expert answers (e.g., "I'm 80% sure A causes B").
- The computer uses that answer to instantly sharpen its blueprints, even with very little data.

4. Hard vs. Soft Constraints (The Rules of the Game)

The paper distinguishes between two ways an expert can help:

Hard Constraints (The "Hard No"): "This edge cannot exist." (e.g., "A patient's age cannot be caused by a tumor.") The computer treats this as an absolute law.
Soft Constraints (The "Gut Feeling"): "I think this edge is likely, but I'm not 100% sure." (e.g., "I think diet probably affects heart health.") The computer treats this as a strong hint, not a rule.
The Magic: The authors created a way to turn these "gut feelings" (probabilities) into a mathematical guide that helps the computer build the blueprints faster and more accurately.

5. What They Tested

The authors didn't just talk about it; they tested it:

Synthetic Data: They created fake data where they knew the "true" blueprints. They showed that their method, when guided by a simulated expert, could find the correct blueprints much better than methods that didn't ask for help or tried to use a single model.
Real-World Data: They tested it on a Breast Cancer dataset. They treated the different types of cancer (which have different causes) as the "mixed groups." Their method successfully identified the different subgroups and the specific cause-and-effect rules for each, outperforming standard methods.

Summary

Think of this paper as a new way to solve a puzzle.

Old way: Try to force all puzzle pieces into one picture.
New way: Realize there are two pictures mixed together, sort the pieces, and build two pictures.
The Boost: When you are stuck, you don't just guess; you ask a smart consultant the most helpful question possible to get unstuck.

The result is a system that can learn complex, mixed-up cause-and-effect relationships much faster and more accurately, especially when data is scarce but expert knowledge is available.

1. Problem Statement

Causal discovery aims to infer causal relationships (Directed Acyclic Graphs or DAGs) from observational data. While Bayesian methods are powerful for handling uncertainty and incorporating prior knowledge, existing approaches face two critical limitations:

Single-Graph Assumption: Most methods assume a single underlying causal graph explains all data. This fails in heterogeneous domains (e.g., biology, medicine) where data arises from multiple distinct subpopulations with different causal mechanisms (mixtures of DAGs).
Lack of Expert Integration: While domain experts possess valuable knowledge, current elicitation strategies are designed for single graphs and do not effectively handle mixtures. Furthermore, existing priors often rely on "hard" constraints (forcing edges to exist or not exist) rather than probabilistic "soft" constraints, which limits flexibility.

The paper addresses the challenge of learning mixtures of Causal Bayesian Networks (CBNs) while efficiently integrating expert knowledge through a principled querying strategy.

2. Methodology

The authors propose a unified framework combining Variational Mixture Structure Learning (VaMSL) with Bayesian Experimental Design (BED) for expert elicitation.

A. Variational Mixture Structure Learning (VaMSL)

The core inference engine extends the Differentiable Bayesian Structure Learning (DiBS) method to handle mixtures.

Latent Graph Embeddings: Instead of searching the discrete space of DAGs directly, the method uses a continuous latent space $Z = [U, V]$ to represent graph structures. A soft graph (matrix of edge probabilities) is derived from $Z$ using a logistic function.
Mixture Model: The data is modeled as a mixture of $K$ components, where each component $k$ has its own graph $G_k$ , parameters $\Theta_k$ , and mixing weight $\pi_k$ .
Variational Inference: The posterior is approximated using Stein Variational Gradient Descent (SVGD) for the latent embeddings and parameters, and Coordinate Ascent Variational Inference (CAVI) for the mixing weights and cluster assignments. This allows for gradient-based optimization in the graph space.
Differentiability: By relaxing the discrete graph space to a continuous one and using a cyclicity regularizer (penalizing cycles), the method enables efficient gradient descent to find DAGs.

B. Expert Elicitation via Bayesian Experimental Design (BED)

To incorporate expert knowledge without querying every possible edge (which is infeasible), the authors propose an active learning strategy:

Query Selection: The system selects the most "informative" edges to query using Expected Information Gain (EIG). The EIG measures how much a potential expert response would reduce uncertainty in the posterior distribution of the graphs.
Expert Simulator: A simulator models the expert's response based on the current latent graph posterior $p(Z|D)$ . The expert's belief is modeled as a Beta distribution, where the mode corresponds to the queried edge probability.
Informative Graph Prior:
- Hard Constraints: Edges explicitly forbidden or required by the expert are masked (0/1) in the latent space.
- Soft Constraints: For probabilistic beliefs, the authors use the "Imaginary Observations" device. They map the expert's probability response to a hypothetical set of trials (successes/failures) that would lead to that belief. This creates a likelihood term that biases the latent graph embeddings toward the expert's beliefs without rigidly fixing them.

C. The Iterative Loop

The method operates in an iterative loop:

Infer the current mixture of CBNs.
Use BED to select the top $M$ edges that maximize EIG.
Query the expert for beliefs on these edges.
Update the variational posterior using the new informative prior derived from the expert's responses.
Repeat until the query budget is exhausted.

3. Key Contributions

Novel Informative Graph Prior: A method to incorporate both hard and soft expert constraints into the latent graph space of a mixture model, using the concept of imaginary observations to translate probabilistic beliefs into a likelihood function.
BED-Based Elicitation Strategy: A principled approach to select optimal queries that maximize information gain, significantly reducing the number of queries needed compared to random or exhaustive querying.
VaMSL Algorithm: The first method to perform joint Bayesian inference on mixtures of both linear and non-linear CBNs while incorporating expert knowledge. It extends DiBS to handle heterogeneous data structures.
Handling Heterogeneity: Demonstrates that modeling data as a mixture of DAGs yields better structure learning and clustering than assuming a single graph, especially when expert knowledge is used to guide the separation of subpopulations.

4. Experimental Results

The authors evaluated their method on synthetic data and a real-world breast cancer dataset.

Synthetic Data (Linear & Non-linear):
- Structure Learning: VaMSL with expert information significantly outperformed baselines (standard DiBS, Gaussian Mixture Models, Causal k-means) in terms of Expected Structural Hamming Distance (ESHD).
- Clustering: The method achieved high classification accuracy in assigning data points to the correct causal subpopulation, matching the performance of an "oracle" that knew the true labels.
- Query Efficiency: The BED strategy outperformed random querying, achieving better results with fewer expert interactions.
- Robustness: The method remained effective even when the expert was simulated with imperfect reliability (e.g., 90% accuracy).
Real-World Application (Breast Cancer Dataset):
- Applied to the UCI Wisconsin Breast Cancer Diagnostic dataset.
- The method successfully identified distinct causal structures associated with different tumor classes (malignant vs. benign) in an unsupervised manner.
- VaMSL + Expert achieved the highest classification accuracy and the lowest ESHD compared to other methods, demonstrating the practical utility of incorporating domain knowledge into complex, heterogeneous medical data.

5. Significance and Impact

Bridging the Gap: This work bridges the gap between low-data/high-expertise scenarios and modern causal discovery. It provides a formal mechanism to leverage human intuition in complex, heterogeneous settings where data alone is insufficient.
Scalability to Heterogeneity: By moving beyond the single-graph assumption, the method addresses a major limitation in real-world causal modeling, where populations are rarely homogeneous.
Efficiency: The active learning component ensures that the cost of expert interaction is minimized, making the approach feasible for expensive domain experts.
Flexibility: The ability to handle both linear and non-linear relationships, as well as hard and soft constraints, makes the framework broadly applicable across various scientific domains.

In summary, the paper presents a robust, scalable, and expert-in-the-loop framework for discovering causal structures in heterogeneous data, significantly advancing the state of the art in Bayesian causal discovery.