🤖 machine learning

A Neurosymbolic Approach for Constructing Planning Domain Models from Clinical Narratives

This paper introduces NSPIN, a neurosymbolic framework that combines large language models with empirical validation to automatically construct probabilistic planning domain models from unstructured clinical narratives, effectively addressing the challenges of implicit actions and data scarcity in complex surgical workflows.

Original authors: Ranveer Singh, Saurabh Mathur, Michael Skinner, Prasad Tadepalli, Kristian Kersting, Sriraam Natarajan

Published 2026-08-24

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ranveer Singh, Saurabh Mathur, Michael Skinner, Prasad Tadepalli, Kristian Kersting, Sriraam Natarajan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Hospitals are filled with stories written in the margins of patient care. Surgeons record their work in detailed notes, describing how they handled a specific patient's unique complications, from a perforated organ to an unexpected infection. For decades, this knowledge has remained trapped inside individual charts, useful only for the person who wrote it or the team caring for that specific patient. It has never been gathered into a single, clear picture of how these procedures actually work across thousands of cases. Scientists have long wanted to turn these messy, handwritten stories into a formal set of rules—a logical map that shows exactly what must happen before a surgeon can take the next step, and what might happen as a result. This kind of map is essential for building artificial intelligence that can help train new doctors, review surgical notes automatically, or even guide future robotic systems. The challenge is that human language is full of shortcuts and assumptions. A surgeon might write that they "placed a port" without mentioning that they first had to make an incision, because to them, that step is so obvious it doesn't need to be said. Computers, however, cannot guess what is obvious; they need every single step explicitly stated to understand the logic of the operation.

A team of researchers has developed a new method to solve this problem, turning thousands of unstructured surgical notes into a precise, logical model of a common pediatric surgery called a laparoscopic appendectomy. This procedure, which involves removing an inflamed appendix through small incisions, is performed frequently in children's hospitals, providing a rich dataset of over 2,600 real-world notes written by nine different surgeons. The researchers created a system that acts as a bridge between the fluid nature of human language and the rigid requirements of computer logic. They call this system NSPIN. It works by first using a powerful language model, a type of artificial intelligence trained on vast amounts of text, to read the raw notes and pull out the specific actions and observations mentioned. Because the notes are often incomplete, the system then uses the same language model to fill in the missing gaps, inferring the hidden steps that a surgeon would have taken but didn't write down. Finally, it uses a different kind of mathematical logic to organize these steps into a coherent plan, checking the work against the data to ensure the rules make sense.

The process begins with the system reading the raw text of a surgical note. It identifies the key events, such as making an incision or placing a surgical instrument, and arranges them in the order they occurred. However, the researchers found that simply reading the text was not enough. Surgeons often skip writing down universal protocols or obvious intermediate steps, leaving the computer with a broken chain of events. To fix this, the system asks the language model to act as a clinical expert, suggesting the missing steps that logically connect the recorded events. For instance, if a note mentions that an appendix was removed, the system infers that the base of the appendix must have been secured first, even if the surgeon did not explicitly write that down. This step is crucial because it recovers the silent knowledge that experts carry in their heads but rarely commit to paper. The system generates multiple possible versions of these missing steps and then combines them, keeping only the ones that the majority of the generated versions agree on, ensuring the final sequence is robust and reliable.

Once the system has a complete, ordered sequence of events for each surgery, it builds a formal model that describes the rules of the operation. This model defines what conditions must be true before a specific action can be taken and what the likely outcomes of that action are. For example, it learns that a surgeon cannot place a surgical port until an incision has been made, and that placing the port might lead to the discovery of adhesions, or scar tissue, which then requires a specific procedure to remove. The researchers tested this approach by comparing it against methods that relied solely on the language model without the logical checking step. They found that the combined approach was far superior. The system that used the logical rules to refine the language model's suggestions produced a model that was much better at predicting what would happen next in a surgery and at identifying which actions were actually possible at any given moment.

The results showed that the system could learn from the notes of nine different surgeons and create a general set of rules that worked for all of them, even when tested on notes from a surgeon it had never seen before. When the researchers asked medical experts to review the steps the system had inferred, the experts confirmed that the missing steps were clinically correct and necessary. In one instance, the system correctly recovered the information that a surgeon had closed the abdominal wall with a specific type of suture, even though the information extractor initially failed to pull it out from a note that explicitly stated, "The umbilical fascia was closed with 0 Vicryl suture." In another case, it deduced that securing the base of the appendix was a prerequisite for cutting it, a logical necessity that was not explicitly stated in the text. These findings demonstrate that the system can successfully translate the implicit, unwritten knowledge of human experts into a formal, usable format.

The study also revealed that relying only on the language model, without the logical refinement, led to errors. The language model alone tended to create rules that were too loose or too restrictive, often missing the true cause-and-effect relationships that govern a surgical procedure. By combining the language model's ability to understand text with a symbolic system's ability to enforce logical consistency, the researchers created a model that was both flexible enough to handle the variety of human writing and strict enough to be trusted by machines. This approach suggests a path forward for building artificial intelligence in high-stakes fields like medicine, where understanding the "why" behind an action is just as important as knowing the "what." The work does not claim to have solved all the problems of surgical automation, but it provides a proven method for turning the vast, unstructured history of patient care into a structured foundation for future tools that could improve patient safety and surgical training.

Problem Statement
Formalizing complex surgical workflows, such as laparoscopic appendectomy, into interpretable planning domain models remains a significant challenge. While such models are essential for AI-in-the-loop systems, surgical training simulators, and quality improvement, existing approaches struggle with the nature of clinical data. Clinical narratives are unstructured, linguistically diverse, and frequently omit "obvious" intermediary steps that are implicit to surgeons but essential for logical consistency. Purely empirical symbolic methods fail because they require structured state-action trajectories, which are rarely available. Conversely, Large Language Models (LLMs), while capable of linguistic extraction and imputation, struggle with inductive logical reasoning and often produce overly simplistic or logically unsound models, particularly when lacking external support like executable environments or clean domain descriptions. Furthermore, clinical domains require probabilistic modeling to capture stochastic outcomes, a capability often missing in deterministic LLM-based formalizers.

Methodology: The NSPIN Framework
The authors introduce NSPIN, a neurosymbolic framework designed to induce probabilistic planning domain models (specifically in Probabilistic Planning Domain Definition Language, or PPDDL) from unstructured clinical narratives. The framework bridges the gap between the linguistic flexibility of LLMs and the logical rigor of symbolic induction through four distinct stages:

Schema Creation: A pretrained LLM generates a formal domain schema ( $P_0$ ) based on a natural language description of the domain. This schema defines typed object categories (e.g., instruments, anatomical structures), observation predicates (state properties), and action predicates (surgical steps). At this stage, action schemas are syntactically valid but contain empty preconditions and effects, serving as a fixed vocabulary to constrain the subsequent search space.
Sequence Generation: This stage converts raw surgical notes into structured, grounded predicate sequences through two sub-phases:
- Information Extraction: The LLM extracts a sequence of grounded predicates from the text. To handle stochasticity, $K$ candidate sequences are generated and aggregated using majority filtering for predicate retention, consensus voting for argument grounding, and weighted Kemeny rank aggregation for temporal ordering.
- Sequence Imputation: Recognizing that surgeons omit implicit steps, the framework uses the LLM's internal world model to infer missing intermediary actions or observations necessary for logical continuity. Similar to extraction, $K$ candidate imputations are generated and aggregated to produce a final, enriched sequence ( $S$ ) that represents the complete workflow.
Planning Model Construction:
- Data-Driven Induction: A greedy, coverage-based inducer constructs initial PPDDL action schemas from the enriched sequences. It estimates preconditions based on observation contexts and probabilistic effects based on maximum-likelihood outcome frequencies, filtering out low-probability effects.
- LLM Refinement: To address spurious correlations where data-driven induction might include non-essential preconditions, an LLM refines the induced preconditions. The LLM proposes revisions based on satisfied and failed contexts, accepting changes only if they do not degrade validation performance. This ensures the preconditions are semantically grounded rather than merely associative.
Evaluation: The resulting model is evaluated on a held-out set of surgical notes to measure its ability to predict observation sets and subsequent actions.

Key Contributions
The paper makes three primary technical contributions:

NSPIN Framework: A novel neurosymbolic approach that transforms unstructured surgical notes into formal, probabilistic PPDDL domains by combining LLM-based extraction/imputation with symbolic induction and refinement.
Implicit Knowledge Imputation: The exploitation of pretrained LLMs to fill gaps in surgical notes, inferring latent clinical actions and preconditions that are essential for logical plan consistency but often omitted by human writers.
Empirical Validation: A comprehensive evaluation on a dataset of 2,660 real-world laparoscopic appendectomy notes from nine surgeons, demonstrating that the neurosymbolic approach outperforms purely LLM-based baselines in generalizing to unseen workflows.

Results
The authors evaluated NSPIN against two LLM-only baselines (one zero-shot, one with in-context examples) and a variant of NSPIN without the final LLM refinement step. Key findings include:

Imputation Efficacy: Models trained on imputed data significantly outperformed those trained on raw extractions across all metrics (Top-1/Top-3 accuracy and Negative Log-Likelihood), confirming that recovered implicit information is critical for accurate modeling.
Refinement Impact: The LLM refinement step substantially improved next-action prediction metrics. Specifically, it reduced the True Action Disallowed Rate (TADR), indicating that the refined preconditions were less restrictive and more compatible with observed workflows than the purely data-driven versions.
Neurosymbolic Superiority: NSPIN (with refinement) outperformed all LLM-only baselines in observation prediction and next-action prediction. While unrefined symbolic models initially underperformed neural baselines, the addition of semantic refinement allowed NSPIN to surpass them, demonstrating that symbolic validation is necessary to ground LLM procedural priors in empirical data.
Qualitative Alignment: Expert review confirmed that imputed steps (e.g., fascia closure, securing the appendix base) were clinically correct and that refined preconditions accurately reflected genuine clinical necessities rather than descriptive associations.

Significance
The paper claims that NSPIN provides a pathway to creating interpretable, generalizable models of surgical procedures that can support trustworthy, deployable clinical AI. By successfully formalizing unstructured narratives into probabilistic planning domains, the approach enables downstream applications such as:

Improving surgical simulators by capturing rare complications and procedural variations.
Enabling automated review of operative notes.
Informing quality improvement initiatives by tracking systematic differences among surgeons and institutions.
Laying the groundwork for future autonomous surgical systems.

The authors emphasize that their approach addresses the specific limitations of current methods by handling the lack of structured data and the prevalence of implicit actions in clinical settings, offering a robust alternative to purely symbolic or purely neural approaches.

More like this