Knowledge augmented causal discovery through large language models and knowledge graphs: application in chronic low back pain

This paper proposes a knowledge-augmented causal discovery framework that integrates Large Language Models with GraphRAG to overcome the limitations of data-only methods, demonstrating that this approach significantly improves the accuracy of inferring causal relationships in chronic low back pain compared to standalone or standard RAG-enhanced methods.

The Gist

Imagine you are trying to figure out why a specific car keeps breaking down. You have a pile of data: the car's mileage, the weather on the days it broke, the type of gas used, and the driver's habits.

The Old Way (Data-Driven Discovery)
If you just look at the numbers, a computer might say, "Hey, every time it rains, the car breaks!" But that's a trap. Maybe it's not the rain; maybe it's that the driver only takes the car to the beach when it rains, and the salt air is the real culprit. This is the problem with traditional data analysis: it sees correlations (things happening together) but struggles to find the true causes (what actually makes things happen). It's like trying to solve a mystery by only looking at footprints without knowing who made them.

The New Way (Adding a "Super-Expert" Brain)
This paper proposes a smarter approach. Instead of just looking at the data, we bring in a "Super-Expert" brain to help us. In this study, that brain is a Large Language Model (LLM)—a super-smart AI that has read almost everything written about chronic lower back pain.

However, just asking the AI "What causes back pain?" isn't enough. It might guess or hallucinate facts. So, the researchers gave the AI a Knowledge Graph. Think of this as a massive, organized library where every fact about back pain is connected to every other fact, like a giant web of truth.

The Three Tools Tested
The researchers tested four different ways to solve the "back pain mystery" to see which one worked best:

1. The Data Detective (Causal Discovery Alone): Just looking at the patient data.
   • Result: It got lost easily. (Score: 0.396)
2. The Smart Reader (LLM + RAG): Asking the AI to read a list of documents and answer.
   • Result: Better, but sometimes it missed the connections between facts. (Score: 0.714)
3. The Smart Reader with a Map (LLM + GraphRAG): This is the star of the show. Instead of just reading a list, the AI navigates the Knowledge Graph. It's like giving the detective not just a list of clues, but a treasure map that shows exactly how the clues connect to each other.
   • Result: This was the most accurate method. (Score: 0.745)
4. The AI Guessing Game (LLM alone): Just asking the AI without any specific documents.
   • Result: It was okay, but not as precise as using the map. (Score: 0.636)

The "Interview" Technique
The researchers also realized that how you ask the AI matters. Instead of just asking "Does A cause B?", they asked the AI specific questions, just like a real doctor would:

• "Is this relationship physically possible?"
• "Do these two things usually happen at the same time?"
• "Does one happen before the other?"

By asking these specific questions, the AI could filter out the nonsense and focus on the real causes.

The Big Picture
The main takeaway is simple: Data is great, but it needs a guide.

By combining raw data with a "map" of human knowledge (the Knowledge Graph) and a super-smart AI (the LLM), we can finally build a much clearer picture of what causes chronic lower back pain. It's like bridging the gap between the cold, hard numbers and the warm, experienced wisdom of doctors. This helps us move from guessing "maybe this causes that" to knowing "this definitely causes that," which is a huge step forward for treating patients.

Technical Summary

1. Problem Statement

Traditional data-driven causal discovery algorithms face significant limitations when attempting to infer causal relationships and reconstruct causal models solely from observational data. These limitations include:

• Structural Constraints: The algorithms are heavily dependent on the specific structural properties of the available dataset.
• Lack of Causal Logic: Purely statistical methods often struggle to distinguish between correlation and causation without underlying logical frameworks.
• Absence of External Knowledge: They fail to incorporate established domain expertise or prior scientific knowledge, leading to results that are merely suggestive rather than definitive.

Consequently, data-driven approaches alone often yield low accuracy in recovering true causal structures, particularly in complex medical domains like chronic low back pain (cLBP) where data may be sparse or noisy.

2. Methodology

The authors propose a framework that augments traditional causal discovery by integrating Large Language Models (LLMs) and Knowledge Systems. The methodology involves:

• Ground Truth Construction: An expert-defined causal graph for chronic low back pain was established to serve as the benchmark (ground truth).
• Augmentation Strategies: The study compares four distinct approaches to causal discovery:
  1. Data-Only: Standard causal discovery algorithms without external input.
  2. LLM-Augmented: Using an LLM to suggest causal relationships based on its internal training data.
  3. RAG-Augmented: Using Retrieval-Augmented Generation (RAG) to retrieve relevant external knowledge to guide the LLM.
  4. GraphRAG-Augmented: A novel approach utilizing Knowledge Graph-based RAG (GraphRAG). This system leverages the structured relationships within a knowledge graph to provide more precise, context-aware retrieval for the LLM, specifically tailored to the causal discovery task.
• Prompt Engineering: The study explores various prompting methodologies inspired by domain experts, specifically querying the models on:
  • The plausibility of causal relationships.
  • The presence of statistical associations.
  • The existence of temporal causal relationships.

3. Key Contributions

• Novel Framework: Introduction of GraphRAG as a superior mechanism for augmenting causal discovery compared to standard RAG or standalone LLMs.
• Benchmarking: A comprehensive performance evaluation comparing data-only methods against various AI-augmented approaches in a real-world medical context (cLBP).
• Prompting Analysis: Systematic investigation into how different query types (plausibility, association, temporality) influence the accuracy of LLM-generated causal hypotheses.
• Bridging the Gap: A demonstration of how to effectively merge domain-specific knowledge (via knowledge graphs) with data-driven statistical methods to create more robust causal models.

4. Results

The study presents quantitative results based on the F1 score, measuring the accuracy of the recovered causal graphs against the expert-defined ground truth:

Approach	F1 Score	Performance Context
Causal Discovery Alone	0.396	Baseline; lowest performance due to lack of external knowledge.
LLM Augmentation	0.636	Significant improvement over data-only, leveraging internal LLM knowledge.
RAG Augmentation	0.714	Improved over standalone LLM by retrieving specific external context.
GraphRAG Augmentation	0.745	Highest Performance. Demonstrates that structured knowledge graphs provide the most effective augmentation for causal inference.

The results indicate that while LLMs and RAG offer substantial improvements over traditional methods, the GraphRAG approach yields the most accurate causal models, suggesting that the structural integrity of knowledge graphs is critical for causal reasoning.

5. Significance

This research holds significant implications for medical informatics and causal inference:

• Accelerated Modeling: It offers a pathway to accelerate the modeling of complex conditions like chronic low back pain by reducing the reliance on massive, perfectly structured datasets.
• Integration of Expertise: It successfully bridges the gap between domain knowledge (encoded in knowledge graphs) and data-driven approaches, ensuring that statistical findings are grounded in biological and clinical plausibility.
• Future Directions: The findings suggest that future causal discovery systems should move beyond simple text retrieval (RAG) toward structured graph-based retrieval (GraphRAG) to achieve higher fidelity in scientific and medical applications.