Automated neuroradiological support systems for multiple cerebrovascular disease markers -- A systematic review and meta-analysis

Technical Summary: Automated Neuroradiological Support Systems for Multiple Cerebrovascular Disease Markers

Problem Statement
Cerebrovascular diseases (CVD) are a leading cause of stroke and dementia globally. The presence of specific CVD markers on brain imaging—namely white matter hyperintensities (WMH), acute and chronic ischaemic stroke lesions (ISL), lacunes, enlarged perivascular spaces (PVS), haemorrhagic lesions, cerebral microbleeds (CMB), and brain atrophy—indicates elevated risks for future adverse events. While the simultaneous identification of these markers is crucial for preventative management and patient triage, there is a worldwide shortage of radiologists, and manual interpretation suffers from inter-observer variability. Although automated systems exist, previous reviews have largely focused on single pathologies (e.g., acute stroke or WMH alone) or specific modalities. There is a critical gap in understanding the landscape of systems capable of assessing multiple CVD markers simultaneously, particularly regarding their commercial availability, technical methodologies, and validation rigor.

Methodology
The authors conducted a systematic review and meta-analysis of both commercially available software and peer-reviewed research publications.

• Selection Criteria: The primary inclusion criterion required systems to fully automatically assess at least two distinct CVD markers from radiological brain images. Semi-automated methods, single-marker systems, and non-peer-reviewed abstracts were excluded.
• Search Strategy:
  • Commercial Systems: Identified via industry reviews, AI radiology registries (e.g., Grand Challenge), and conference exhibitions (RSNA) between October 2022 and October 2024. Validation was cross-referenced with company websites, white papers, and FDA 510(k) documents.
  • Research Publications: A literature search on Web of Science (2008–2024) targeted papers mentioning at least two CVD features alongside automation or machine learning terms.
• Data Extraction: Information was extracted regarding imaging modalities, targeted pathologies, system features (segmentation, quantification), development/validation data characteristics, deep learning architectures, and regulatory approvals.
• Bias Assessment: The QUADAS-2 tool was adapted to assess risks of bias and applicability, categorizing entries as "good," "medium," or "bad" to account for the inverse relationship between bias risk and applicability.

Key Contributions and Results

• Commercial Landscape:

  • Volume: 29 commercial products and 13 research publications met the inclusion criteria.
  • Categorization: Commercial systems fall into two distinct groups:
    1. Acute Stroke Triage (19 systems): Primarily focus on Non-Contrast CT (NCCT) to detect intracranial haemorrhages (ICH) and ischaemic stroke lesions (ISL) or large vessel occlusions (LVO). Most utilize Deep Learning (DL), specifically Convolutional Neural Networks (CNNs).
    2. Neurodegenerative Support (16 systems): Focus on MRI (T1/FLAIR) to quantify brain atrophy and WMH, often for dementia management.
  • Gap in Comprehensive Analysis: No single commercial system performs a comprehensive joint analysis of all CVD markers (WMH, ISL, lacunes, PVS, haemorrhagic lesions, CMB, and atrophy). Most systems targeting chronic markers (e.g., lacunes, PVS, CMB) are absent from the commercial market.
  • Validation: While 19 systems have FDA approval and 26 have CE marking, detailed validation data (e.g., sensitivity, specificity, Dice scores) is often limited or proprietary.

• Research Landscape:

  • Focus: Research publications heavily favor WMH (11/13) and ISL (9/13). Lacunes and PVS were targeted only twice each, and CMB only once.
  • Methodology Shift: Early studies (pre-2018) utilized rule-based or classical machine learning methods. Recent studies (2018–2024) have shifted almost exclusively to Deep Learning, predominantly using U-Net architectures (2D or 3D).
  • Performance:
    • WMH: Reported Dice Similarity Coefficients (DSC) average 0.76.
    • ISL: Reported DSC averages 0.59, with a wide range (0.32–0.79), indicating inconsistent performance, likely due to task complexity (e.g., distinguishing ISL from WMH).
    • Small Lesions: Detection of lacunes and PVS remains challenging, with few studies reporting robust metrics.
  • Data Limitations: Most research relies on non-open datasets. Only a few studies used public challenges (e.g., WMH Segmentation Challenge, ISLES). There is a notable lack of publicly available datasets with expert annotations for multiple markers simultaneously.

• Bias and Applicability:

  • Demographics: Research datasets often show a bias toward male subjects.
  • Reproducibility: A significant number of studies were rated "bad" regarding reproducibility due to insufficient methodological detail (e.g., preprocessing steps, data splits) and a lack of standardized benchmark datasets.
  • Ground Truth: While many studies use manual expert annotation, some rely on semi-automated ground truth, introducing potential bias.

Significance and Claims
The paper concludes that while significant progress has been made in automating the detection of acute stroke (via CT) and neurodegenerative markers (via MRI), no comprehensive, fully automated, and well-validated neuroradiological support system currently exists that jointly assesses all major CVD markers.

The authors claim that such a system would offer substantial societal benefits by enabling more efficient patient management and facilitating large-scale research into CVD as a "whole-brain disease." They emphasize that the development of a robust, multi-marker system is currently hindered by:

1. The difficulty of differentiating markers with similar appearances (e.g., WMH vs. chronic ISL, PVS vs. lacunes).
2. A lack of standardized, publicly available datasets with multi-marker annotations.
3. The need for methodological innovations in data-efficient learning and small lesion segmentation.

The paper posits that future advancements in Deep Learning, combined with community-agreed benchmark datasets and metrics, are prerequisites for realizing a complete CVD support system.