🤖 machine learning

A Neurosymbolic Approach for Explainable Early Diagnosis of Alzheimer's Disease

This paper presents a neurosymbolic pipeline that leverages pretrained foundation models to extract linguistic markers from verbal fluency audio and constructs a Bayesian Network for explainable, automated early diagnosis of Alzheimer's disease, successfully recovering known clinical knowledge while identifying novel progression indicators.

Original authors: Ranveer Singh, Pranuthi Tenali, Saurabh Mathur, Ameet Soni, Vaishali Phatak, Karla Lynch, Daniel Murman, Matthew Rizzo, Sriraam Natarajan

Published 2026-08-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ranveer Singh, Pranuthi Tenali, Saurabh Mathur, Ameet Soni, Vaishali Phatak, Karla Lynch, Daniel Murman, Matthew Rizzo, Sriraam Natarajan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a mystery, but the clues are hidden inside a chaotic, noisy room. In the world of medicine, doctors often need to listen to how people speak to spot early signs of Alzheimer's disease. They know that as the disease creeps in, a person's ability to find words, group ideas, or speak smoothly starts to change. However, listening to hours of audio recordings and manually writing down every word, pause, and hesitation is like trying to find a specific needle in a haystack while wearing thick gloves. It takes forever, it's exhausting, and it's hard to do for thousands of people at once. This is where the field of "neurosymbolic" AI comes in. Think of it as a super-team where one part is a "neural" brain—a computer that is amazing at hearing sounds and understanding messy patterns—and the other part is a "symbolic" brain—a logical detective that follows strict rules to make sure the conclusions make sense. The big question scientists are asking is: Can we build a robot that doesn't just listen, but actually understands the story behind the speech to help doctors catch Alzheimer's earlier?

This paper introduces a new team called NeSyQuaKE, which stands for Neurosymbolic Qualitative Knowledge Extraction. It's a clever pipeline designed to take raw audio recordings of patients doing word games (like naming as many animals as they can in one minute) and turn them into clear, logical rules about how their speech relates to cognitive decline. Instead of just guessing, NeSyQuaKE uses a two-step dance. First, it uses powerful "foundation models"—super-smart AI tools trained on massive amounts of data—to act as a translator. These tools listen to the audio, write it down, and pull out specific details like how many words were said or how long the pauses were. But here's the magic: the system doesn't stop there. It passes these details to a "symbolic" layer that acts like a strict accountant. This layer uses math to calculate exact features (like speech rate) and then builds a "Bayesian Network," which is essentially a map showing how different speech habits influence each other and the disease.

The researchers tested this system on 162 audio recordings from patients who were either healthy or showing signs of mild cognitive impairment. They wanted to see if the robot could figure out the same things human experts know, and if it could find new clues humans might miss. The results were quite promising. NeSyQuaKE successfully recreated the known "rules" that experts use. For instance, it confirmed that people with cognitive impairment tend to speak fewer words, switch between topics less often, and have smaller groups of related words compared to healthy people. In fact, when the researchers compared the robot's notes to human-written notes, the robot was much more accurate, making far fewer mistakes in counting words or identifying clusters.

But the robot didn't just copy the experts; it found new patterns, too. It discovered subtle trends, such as how cognitive impairment might negatively affect the size of word groups in specific types of word games. The authors suggest that while the system is very good at spotting these trends, it's not perfect yet. They even tried a "feedback loop" where the robot could ask the AI translator to double-check words it thought were wrong, which made it even more accurate. The paper concludes that NeSyQuaKE is a strong, explainable tool that bridges the gap between messy audio and clear medical knowledge. It suggests that by combining the listening power of modern AI with the logical rigor of symbolic reasoning, we can build a scalable way to track Alzheimer's progression, potentially helping doctors catch the disease earlier and more reliably than ever before.

Technical Summary: A Neurosymbolic Approach for Explainable Early Diagnosis of Alzheimer's Disease

Problem Statement

The early diagnosis of Alzheimer's Disease (AD) relies heavily on identifying subtle linguistic and semantic deficits, such as reduced verbal fluency and word-finding difficulties. Currently, the analysis of verbal fluency tests (e.g., naming animals or words starting with specific letters) is a manual, labor-intensive process requiring expert transcription and qualitative analysis. This bottleneck limits the scalability of longitudinal tracking.

The core technical challenge lies in the representational gap between sensory input and clinical knowledge:

Input: Cognitive impairment indicators are embedded in unstructured, continuous acoustic signals (raw audio).
Knowledge: Clinical domain knowledge consists of structured, qualitative relationships between discrete variables (e.g., "cognitive impairment negatively influences speech rate").

Pure neural architectures excel at pattern recognition in raw audio but lack mechanisms for logically consistent, verifiable, and interpretable reasoning. Conversely, symbolic systems cannot process high-dimensional raw audio. Bridging this gap requires a framework capable of high-fidelity symbol grounding and rigorous reasoning.

Methodology: NeSyQuaKE

The authors introduce NeSyQuaKE (Neurosymbolic Qualitative Knowledge Extraction), a framework following a Neuro | Symbolic (Type 3) architecture. The pipeline decouples the task into three distinct stages:

1. Information Extraction (Symbol Grounding)

This stage maps raw audio recordings ( $a^{(i)}$ ) to tabular feature vectors ( $x^{(i)}$ ) using a sequential combination of foundation models and deterministic symbolic programs:

Speech-to-Text (S2T): A pre-trained transformer-based model (WhisperX) transcribes raw audio into timestamped utterances.
Large Language Models (LLMs): A domain-specific LLM (MedGemma-27b) processes the transcripts to extract structured responses (e.g., identifying valid words within semantic categories).
Deterministic Symbolic Programs: To ensure mathematical consistency and verifiability, the LLM's structured output is processed by symbolic programs. These programs compute clinical features (e.g., Total Words, Speech Rate, Cluster Size) using explicit logic (e.g., total_words = LEN(FILTER(VALID, words))). This step grounds the neural outputs into precise, interpretable variables.

2. Bayesian Network Construction (BNC)

The framework constructs a Bayesian Network (BN) to represent the joint probability distribution over the variables of interest (Lexical, Semantic, and Acoustic features) and the diagnostic label (Cognitive Impairment, $Y$ ).

Structure Learning: Due to the scarcity and noise of clinical data, traditional structure learning is infeasible. Instead, the BN structure is elicited by prompting an LLM multiple times with feature descriptions. The resulting graphs are pooled, and edges are added based on occurrence frequency while maintaining acyclicity.
Parameter Learning: A Bayesian Estimator with a strong prior (BDeu) is used to learn the conditional probability distributions. This approach mitigates issues with small data regimes where Maximum Likelihood Estimation might assign zero probability to unobserved configurations.

3. Qualitative Knowledge Extraction (QuaKE)

Using the QuaKE algorithm, the system reasons over the learned joint distribution to extract Monotonic Influence (MI) statements.

The algorithm checks for the presence of monotonic influence of the diagnostic label $Y$ on descendant features $X_d$ .
It computes a score ( $\delta_d$ ) quantifying the strength of the influence based on the cumulative difference in conditional probabilities.
The output is a set of qualitative rules (e.g., "Cognitive Impairment has a negative monotonic influence on Speech Rate"), which are easily interpretable and comparable to medical literature.

Key Contributions

NeSyQuaKE Framework: A novel neurosymbolic architecture that extracts qualitative knowledge from clinical audio by combining foundation models (for grounding) with Bayesian Networks (for reasoning).
Deterministic Symbolic Layer: The introduction of a layer of symbolic programs to map neural outputs to clinical features, ensuring that metrics like speech rate and semantic switching are mathematically consistent and verifiable.
Empirical Validation: Evaluation on a real-world dataset of 162 verbal fluency tests (Healthy Controls vs. Mild Cognitive Impairment) demonstrating the system's ability to recover established clinical knowledge and generate new hypotheses.

Experimental Results

The system was evaluated on semantic (animals/vegetables) and phonemic (F/L letters) fluency tasks.

Symbol Grounding Accuracy (Q1): NeSyQuaKE significantly outperformed a rule-based baseline. It achieved a 94.3% improvement in the semantic fluency task and 77.2% in the phonemic task regarding the Pairwise Conditional Estimation Error (PCEE) compared to manual annotations.
Alignment with Expert Knowledge (Q2):
- NeSyQuaKE matched 100% of the monotonic influences initially elicited from domain experts for both tasks.
- The LLM-generated BN structure recovered 80% of the edges from the expert-defined network, significantly outperforming the data-driven Peter-Clark (PC) algorithm, which matched only 1 of 15 edges.
Hypothesis Generation (Q3): The framework discovered 9 additional influences beyond expert expectations.
- Validated: Experts confirmed negative monotonic influences of cognitive impairment on cluster size, speech rate, and word length in semantic tasks, and similar negative trends in phonemic tasks (though less pronounced).
- Novel/Speculative: The system inferred a positive monotonic influence of cognitive impairment on word length in phonemic tasks. The authors speculate this may be due to participants using shorter words to maximize count in a time-constrained task, a strategy less applicable to semantic categories.
Feedback Loop (Q4): Incorporating a feedback mechanism where rejected words are sent back to the LLM for phonetically similar replacements improved PCEE by 13.7% (semantic) and 14.5% (phonemic), though the monotonicities discovered remained unchanged.

Significance and Claims

The paper claims that NeSyQuaKE offers a scalable and explainable approach to tracking Alzheimer's disease progression. Its primary significance lies in:

Bridging the Gap: Successfully connecting unstructured acoustic signals to structured qualitative medical knowledge without requiring massive labeled datasets for end-to-end training.
Explainability: Unlike black-box deep learning models, NeSyQuaKE produces Qualitative Influence Statements that align with how clinicians communicate trends (e.g., "as impairment increases, speech rate decreases").
Knowledge Recovery and Discovery: The system not only recovers known clinical markers but also identifies subtle, less prominent trends (such as specific monotonic relationships in phonemic tasks) that might be missed by manual analysis or standard statistical methods.

The authors maintain a modest stance, acknowledging that while the feedback loop showed promise, it requires further validation as corrections are currently based on transcript context rather than direct audio grounding. They also note that future work should explore larger foundation models and specialized speech-language models to handle pathological speech more effectively.