💬 NLP

InsightFlow: LLM-Driven Synthesis of Patient Narratives for Mental Health into Causal Models

The paper introduces InsightFlow, an LLM-driven framework that automatically generates clinically meaningful 5P-aligned causal graphs from psychotherapy transcripts, demonstrating structural and semantic similarity to human expert formulations while offering a scalable tool to augment clinical case formulation workflows.

Original authors: Shreya Gupta, Prottay Kumar Adhikary, Bhavyaa Dave, Salam Michael Singh, Aniket Deroy, Tanmoy Chakraborty

Published 2026-04-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shreya Gupta, Prottay Kumar Adhikary, Bhavyaa Dave, Salam Michael Singh, Aniket Deroy, Tanmoy Chakraborty

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a complex mystery: Why is this person feeling so down or anxious?

In the world of mental health, therapists act as these detectives. They listen to a patient's story and try to draw a "map" of the problem. This map isn't just a list of symptoms; it's a causal web showing how one thing leads to another. For example: "Because my dad left (Event), I feel unloved (Feeling), so I stay up all night worrying (Behavior), which makes me tired (Symptom), which makes me snap at my boss (Trigger), which makes me feel unloved again (Loop)."

Therapists use a specific framework called the "5P Model" to build these maps:

Presenting: What's the problem right now?
Predisposing: What made them vulnerable in the past?
Precipitating: What was the recent trigger?
Perpetuating: What keeps the problem going?
Protective: What strengths do they have?

The Problem: Drawing these maps by hand is hard. It takes a long time, and two different therapists might draw two very different maps for the same person. It's like asking two artists to draw the same tree; one might focus on the roots, the other on the leaves.

The Solution: InsightFlow
The researchers in this paper built a digital assistant called InsightFlow. Think of it as a super-fast, tireless junior detective powered by a Large Language Model (an advanced AI like the one you might be talking to right now).

Here is how it works, using simple analogies:

1. The Interview (Input)

The AI listens to the recording of a therapy session. Imagine it's a scribe sitting in the corner, listening to every word the patient and therapist say.

2. The Sorting Hat (Extraction)

Instead of just reading the story, the AI has a magical sorting hat. It instantly separates the story into five buckets (the 5Ps):

"Oh, 'insomnia' goes in the Presenting bucket."
"Ah, 'losing a job last week' goes in the Precipitating bucket."
"And 'growing up in a chaotic home' goes in the Predisposing bucket."

3. The Web Weaver (Connecting the Dots)

This is the clever part. Once the AI has the buckets, it asks itself: "Does the job loss cause the insomnia? Does the chaotic home cause the job loss?"
It checks every possible pair of ideas. If the story supports a connection, it draws a line between them.

Human Therapists tend to draw straight lines (A leads to B, which leads to C). They are efficient and focus on the most obvious path.
The AI tends to draw a spiderweb. It connects A to B, but also A to C, and B to C. It's a bit more "chatty" and creates a denser, more interconnected map.

4. The Quality Check (Evaluation)

The researchers tested this AI junior detective against real human experts.

Did it get the facts right? Yes. The AI found the same key problems and triggers that the humans did.
Did it draw the map correctly? Mostly. The AI's maps looked a bit different (more like a tangled web vs. a straight chain), but the experts agreed that the AI's connections made sense.
The Verdict: The AI's work was about as good as a human therapist's work. It fell right within the "normal range" of how different humans might interpret the same story.

The Good, The Bad, and The "Weird"

The Good: The AI is fast. It never gets tired. It can spot connections humans might miss because it checks every possible link, not just the obvious ones. It also acts like a highlighter, showing exactly which sentence in the transcript led to a specific conclusion.
The Bad: Sometimes the AI gets a little too chatty. It might draw a line between two things that are related but not directly causing each other (like connecting "rain" to "umbrella" even if the umbrella was bought yesterday). It also sometimes repeats itself (saying "stress" and "feeling stressed" as two different things).
The Missing Piece: The AI is great at seeing the web, but it's not great at seeing the timeline. It doesn't always know that Event A happened before Event B, which is crucial for understanding cause and effect.

The Bottom Line

InsightFlow isn't here to replace the therapist. Think of it as a smart co-pilot.
Imagine a therapist is flying a plane (the therapy session). The AI is the autopilot system that instantly draws a map of the terrain (the patient's mind) and says, "Hey, look! Here are all the mountains, rivers, and storms we need to navigate. Here is a draft map. You, the expert pilot, can now look at it, say 'Yes, that's right,' or 'No, fix that line,' and then fly the plane."

It saves time, reduces the mental load, and ensures no important clue is left behind, but the human expert still holds the controls.

1. Problem Statement

Clinical case formulation is a critical process in psychotherapy where clinicians structure a patient's symptoms, history, and psychosocial factors into a causal model to guide treatment. Clinicians typically use the 5P framework (Presenting problems, Predisposing factors, Precipitating factors, Perpetuating factors, and Protective factors) to create directed causal graphs.

However, manually constructing these graphs from therapy transcripts is:

Time-consuming: Requires significant expert effort.
Subjective: Inter-rater agreement among clinicians is often low (Cohen's kappa ~0.33–0.45), leading to inconsistent formulations.
Scalability issues: Difficult to apply to large datasets for research or public health monitoring.

The paper addresses the challenge of automating this process using Large Language Models (LLMs) to extract causal factors and relationships from unstructured dialogue, aiming to produce graphs comparable to human expert formulations.

2. Methodology

The study proposes InsightFlow, an end-to-end system utilizing the Llama-3 8B model to generate 5P-aligned causal graphs. The methodology consists of four main stages:

A. Dataset Preparation & Ground Truth

Data Source: 46 psychotherapy intake session transcripts from the MentalCLOUDS corpus.
Annotation Protocol: Two trained clinical experts manually annotated the transcripts using a structured 5P protocol.
- Process: Identify Presenting Problems $\rightarrow$ Find Precipitating Factors $\rightarrow$ Trace Predisposing Factors $\rightarrow$ Add Perpetuating Factors $\rightarrow$ Connect via directed edges (causal chains).
- Output: Two sets of "ground truth" human-generated causal graphs for comparison.

B. Automated Graph Generation (InsightFlow Pipeline)

The system uses a two-stage iterative prompting strategy to minimize hallucinations and ensure logical consistency:

Node Extraction: The LLM is prompted (framed as a clinical psychologist) to extract specific factors (Presenting, Predisposing, Precipitating, Perpetuating) from the transcript and output them in a structured JSON format.
Edge Verification (Pairwise Causality): The system generates all potential directed pairs of nodes. For each pair, a secondary prompt asks the LLM to verify if a causal link exists based on the full conversation context, responding with "TRUE" or "FALSE" in JSON.
- Constraint: This feedback loop ensures edges are only added when supported by transcript evidence, filtering out spurious links.

C. Evaluation Framework

The generated graphs were evaluated against human graphs using three distinct approaches:

Automated Structural Metrics:
- NetSimile: Measures structural similarity based on degree distribution, clustering, and other graph features (0 = different, 1 = identical).
- Semantic Similarity: Used SBERT embeddings to calculate cosine similarity for node labels and edge pairs.
- Graph Theory Metrics: Density, centrality (degree, betweenness, closeness), clustering coefficients, and community detection (Leiden, Girvan-Newman, Infomap).
Human Expert Review:
- Five clinical experts rated the LLM graphs blindly on a 5-point scale across six dimensions: Completeness, Consistency, Specificity, Node Plausibility, Edge Plausibility, and Utility.
Statistical Analysis: Comparison of inter-annotator agreement (Fleiss' kappa) and distributional distances (KL divergence, Earth Mover's Distance).

3. Key Contributions

Benchmark Creation: Developed a novel benchmark dataset of 46 annotated mental health causal graphs based on the 5P framework, addressing the lack of gold-standard data in this domain.
Novel Prompting Architecture: Introduced an iterative, feedback-based prompting method (Extraction + Pairwise Verification) specifically designed for causal reasoning in clinical narratives.
Comprehensive Evaluation: Provided a multi-faceted validation combining computational graph metrics, semantic embedding analysis, and blinded clinical expert ratings.
Structural Analysis: Conducted a deep dive into topological differences, revealing that LLMs tend to create "web-like" networks while humans create "chain-like" structures.

4. Key Results

A. Similarity and Agreement

Structural Similarity: LLM-generated graphs achieved a NetSimile score of 0.38–0.46 against human graphs. This falls within the range of inter-human agreement (0.37–0.53), indicating the LLM performs comparably to one clinician relative to another.
Semantic Overlap: High semantic similarity was observed, with mean edge cosine similarity > 0.70 and node-set similarity ~0.75–0.80. The LLM identified the same core clinical factors as experts, even with different surface wording.
Expert Ratings: Experts rated LLM graphs moderately high (mean 3.2–3.6/5). Scores were comparable to human graphs, with "Consistency" being the highest-rated dimension and "Utility" the lowest.

B. Topological Differences

Density & Clustering: LLM graphs were denser (0.29 vs. 0.25 in human graphs) and had higher clustering coefficients (more triangular motifs).
- Interpretation: The LLM's exhaustive pairwise checking led to "shortcut" edges (e.g., $A \to C$ ) that humans often omit in favor of parsimonious chains ( $A \to B \to C$ ). Experts deemed these extra links plausible.
Centrality: LLM graphs showed higher betweenness and closeness centrality, indicating a more distributed influence across nodes. Human graphs tended to concentrate causal flow through specific "hub" nodes.
Community Structure: Both human and LLM graphs successfully grouped nodes by 5P categories, though LLM communities were slightly more numerous and logically grouped by type.

C. Limitations Observed

Temporal Reasoning: The LLM failed to explicitly order events chronologically, presenting factors as a flat network.
Redundancy: The model occasionally generated redundant nodes (e.g., "increased stress" vs. "stress increasing") or split concepts unnecessarily.
Subjectivity: Expert ratings showed low inter-rater agreement (Fleiss' $\kappa \approx 0$ ), confirming the inherent subjectivity of clinical formulation.

5. Significance and Conclusion

InsightFlow demonstrates that LLMs can effectively automate the synthesis of complex clinical case formulations, producing outputs that are structurally and semantically comparable to expert work.

Clinical Utility: The system serves as a promising decision-support tool that can reduce the time therapists spend on initial case conceptualization, highlight non-obvious relationships (due to higher density), and provide traceable evidence for each node (linking back to transcript quotes).
Research Impact: It enables the scalable analysis of large volumes of anonymized therapy sessions for public health research.
Future Directions: The authors suggest refining temporal reasoning, implementing post-processing to remove redundant nodes, and expanding the dataset to include diverse cultural and clinical populations.

In summary, while stylistic differences exist (LLM "webs" vs. Human "chains"), the automated outputs fall within the natural variability of human expert formulations, validating the potential of LLMs to augment clinical reasoning in mental health.