A Cognitive Explainer for Fetal ultrasound images classifier Based on Medical Concepts

This paper proposes an interpretable framework that leverages a concept-based graph convolutional neural network to incorporate medical prior knowledge, thereby providing clinicians with transparent, cognition-aligned explanations for fetal ultrasound scan plane detection.

The Gist

🏥 The Problem: The "Black Box" Doctor

Imagine you are a pregnant woman, and your doctor uses an ultrasound machine to check on your baby. To get a clear picture, the doctor has to find very specific angles (like a perfect side profile or a perfect cross-section of the belly). This is incredibly hard; it takes years of training to learn how to hold the probe just right.

Recently, computers (AI) have gotten really good at finding these perfect angles automatically. But there's a catch: The AI is a "Black Box."

Think of the AI like a genius wizard who can guess the right answer 99% of the time, but when you ask, "How did you know that?" the wizard just shrugs and says, "I just know." Doctors can't trust a wizard they can't understand, especially when it involves a baby's health. They need to know why the computer thinks it found the right picture.

💡 The Solution: The "Medical Detective"

The authors of this paper built a new kind of AI that doesn't just guess; it explains its reasoning using the same language and logic that human doctors use.

Instead of looking at the ultrasound image as a blurry mess of pixels (like a computer usually does), this new AI looks for specific medical "clues" (concepts) that a human sonographer would look for.

The Analogy: Finding a Needle in a Haystack

Imagine you are trying to find a specific type of needle in a haystack.

• Old AI: Looks at the whole haystack and says, "I found the needle!" but can't tell you where or why.
• New AI (This Paper): Says, "I found the needle because I saw three specific things:
  1. A shiny silver tip (the spine).
  2. A round, dark hole nearby (the stomach bubble).
  3. A curved line connecting them (the umbilical vein)."

The AI doesn't just see pixels; it sees anatomy.

🕸️ How It Works: The "Concept Web"

The researchers taught the computer to think like a doctor by building a Concept Graph.

1. Spotting the Clues: First, the computer scans the ultrasound image and finds the important body parts (like the baby's spine, stomach, or thigh bone). It uses a special "highlighter" to find these spots.
2. Drawing the Map: Then, it draws a map connecting these clues. It asks: "Is the stomach next to the spine? Is the thigh bone in the right shape?"
3. The Decision: The computer uses this map to make a decision. It's like a detective connecting the dots on a corkboard. If the dots (clues) are connected in the right way, it confirms, "Yes, this is the correct view!"

🧠 Why This is a Big Deal

The paper tested this new AI against real doctors and found two amazing things:

1. It Speaks "Doctor": When the AI makes a mistake, it can explain why. For example, it might say, "I thought this was the stomach view, but the spine was in the wrong spot." This is a language doctors understand instantly.
2. It Builds Trust: In a test, real doctors were shown the AI's "reasoning map." They said, "Oh, I see what it's looking at. I trust this result." The AI didn't just give an answer; it gave a rationale.

🚀 The Bottom Line

This paper introduces a "Cognitive Explainer." It's like giving the AI a voice and a brain that thinks like a human. Instead of being a mysterious black box, the AI becomes a junior assistant that can show its work, point to the evidence, and explain its logic.

This is a huge step forward because it bridges the gap between "smart computers" and "trusting doctors," potentially making prenatal care safer and more accessible for everyone.

Technical Summary

1. Problem Statement

The automatic detection of fetal standard scan planes (e.g., abdominal, femoral, thalamic) during 2D mid-pregnancy ultrasound examinations is a critical but highly complex task. While Deep Neural Networks (DNNs) have achieved state-of-the-art performance in this domain, their application in clinical settings is hindered by two main factors:

• Lack of Interpretability: Most existing models operate as "black boxes," providing predictions without explaining the rationale.
• Misalignment with Clinical Cognition: Current explainability methods (e.g., saliency maps, Grad-CAM) typically focus on pixel-level features. These often produce blurry, hard-to-identify heatmaps that fail to capture the relationships between anatomical structures. They do not align with how sonographers actually reason, which involves identifying specific anatomical landmarks (concepts) and their spatial relationships.

2. Methodology

The authors propose an interpretable framework that mimics the cognitive reasoning process of a sonographer. Instead of classifying raw pixels directly, the system extracts "medical concepts" and models their relationships using a Graph Convolutional Network (GCN).

The pipeline consists of four main stages:

A. Medical Concept Identification (with Prior Knowledge)

• Goal: To extract key anatomical structures relevant to specific standard planes (Fetal Abdominal Standard Plane - FASP, Fetal Thalamus Standard Plane - FTSP, Fetal Femur Standard Plane - FFSP).
• Process:
  1. Superpixel Segmentation: A Simple Linear Iterative Clustering (SLIC) algorithm is used to generate candidate regions.
  2. Medical Prior Filtering: To overcome the issue of medical images having small foregrounds (where multi-resolution segmentation often picks up background noise), the authors integrate medical prior knowledge.
     • Example (FASP): The stomach bubble (SB), umbilical vein (UV), and spine (SP) are located relative to the fetal abdomen's elliptical shape. The SB is oval and low-brightness; the SP has coarse texture.
     • Example (FTSP): The septum pellucidum cavity (CSP) and thalami (LT/RT) are located around the major axis of the head ellipse.
  3. Attention Constraint: Grad-CAM is used to generate attention heatmaps to constrain the search area to the foreground, filtering out irrelevant background tissues.

B. Concept Graph Construction

• Graph Definition: The image is transformed into a graph $G = (V, E)$.
  • Nodes ($V$): Represent the extracted medical concepts. Node attributes are high-order features encoded by a pre-trained CNN (e.g., ResNet, MobileNet).
  • Edges ($E$): Represent the relationships between concepts. These are defined by:
    1. Spatial Relationships: Relative positions of concepts in the image.
    2. Medical Correlation: Pre-defined correlations based on medical prior knowledge (e.g., the spine is always posterior to the stomach).

C. Concept Graph Learning (GCN)

• A Graph Convolutional Network (GCN) is employed to process the graph-structured data.
• Mechanism: The GCN performs message aggregation and updates node features based on neighbors. It utilizes a predefined prior coefficient ($\alpha$) to measure the interdependence between concepts.
• Output: An MLP processes the final graph features to generate a probability distribution for the standard plane class.

D. Post-hoc Graph Explainer

To interpret the GCN's decision, the authors evaluate three graph-based explainability techniques:

1. Graph Sensitivity Analysis (SA): Uses gradient norms to determine node/edge contribution.
2. Graph Integrated Gradients (IG): Calculates feature importance by scaling inputs from a baseline.
3. Graph Grad-CAM: Visualizes saliency at intermediate layers by combining activations and gradients.

• Result: These methods produce "concept importance maps," highlighting which anatomical structures and relationships were most critical for the classification decision.

3. Key Contributions

1. Cognitive Framework: Proposes the first interpretable framework for fetal US plane classification that operates from the perspective of sonographer cognition rather than pixel-level features.
2. Concept-Based GCN: Introduces a method to construct a graph where nodes are medically significant concepts and edges encode spatial and semantic relationships, allowing the model to "reason" like a doctor.
3. Expert Validation: Conducted extensive qualitative and quantitative assessments, validating the explanations with expert sonographers. The study demonstrates that concept-based explanations are more clinically useful and trustworthy than traditional pixel-level heatmaps.

4. Experimental Results

• Dataset: Retrospective data from two hospitals (Hospital A: 1070 patients; Hospital B: external validation). Classes included FASP, FFSP, FTSP, and "Other."
• Performance:
  • CNN Baselines: Standard CNNs (ResNet, VGG, MobileNet) achieved high accuracy (e.g., ResNet50 reached ~90% on validation data).
  • GCN Models: The proposed GCN approach maintained high performance (e.g., DenseNet121-based GCN achieved ~96.74% Accuracy on the test set).
  • Generalization: The model showed robust performance on the external Hospital B dataset, though with slightly lower metrics than the internal test set, indicating the need for further multi-center validation.
• Qualitative Assessment:
  • Expert sonographers reviewed 100 random cases.
  • Findings: The proposed method successfully identified the correct anatomical landmarks (e.g., focusing on the spine and stomach for FASP) and their relationships.
  • Comparison: Unlike Grad-CAM or CAMERAS which often highlighted vague regions, the graph-based explainers provided clear, structured reasoning.
  • Trust: All five participating doctors agreed that the concept-based explanations enhanced trust in the model and were more clinically useful for identifying failure cases.

5. Significance and Clinical Implications

• Bridging the Gap: This work bridges the gap between deep learning performance and clinical trust. By explaining decisions in terms of anatomical structures (which doctors understand) rather than pixels (which are often ambiguous), the model becomes a viable tool for clinical decision support.
• Error Analysis: The framework allows clinicians to understand why a model failed (e.g., missing a specific landmark or misinterpreting a spatial relationship), facilitating better debugging and training.
• Future Potential: The approach sets a precedent for "Medical AI" that is not just accurate but also transparent and aligned with human medical reasoning, potentially accelerating the adoption of AI in obstetrics and reducing the shortage of experienced operators.

Limitations Noted

• Data Size: The external validation set (Hospital B) was relatively small.
• Real-time Processing: The current method focuses on static images; real-time video processing was not implemented.
• Computation: The step involving the localization of anatomical structures using medical priors is computationally intensive.