⚡ electrical engineering

GlaBoost: A Multimodal Structured Framework for Glaucoma Risk Stratification

GlaBoost is a novel, interpretable multimodal framework that integrates fundus image embeddings, free-text clinical assessments, and structured biomarkers via an enhanced XGBoost model to achieve superior and clinically consistent glaucoma risk stratification compared to existing unimodal and generic multimodal approaches.

Original authors: Cheng Huang, Zeyu Han, Weizheng Xie, Karanjit Kooner, Tsengdar Lee, Jui-Kai Wang, Jia Zhang

Published 2026-07-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Cheng Huang, Zeyu Han, Weizheng Xie, Karanjit Kooner, Tsengdar Lee, Jui-Kai Wang, Jia Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to diagnose a patient with glaucoma, a serious eye disease that can steal vision if not caught early. Traditionally, a doctor acts like a detective, gathering clues from three different sources:

The Photo: A picture of the back of the eye (the fundus).
The Measurements: Hard numbers like the size of the optic nerve cup or the thickness of the nerve layer.
The Notes: The doctor's handwritten or typed notes describing what they see, like "the rim looks pale" or "there is a notch."

Most computer programs (AI) built to help with this have been like one-eyed detectives. They are great at looking at the photo, but they often ignore the measurements and the notes. Or, if they try to look at everything, they get confused and make mistakes.

Enter GlaBoost.

The authors of this paper built a new AI system called GlaBoost. Think of GlaBoost not as a single detective, but as a super-efficient team meeting where three experts sit around a table, share their specific findings, and then vote on the final diagnosis.

Here is how the team works, using simple analogies:

1. The Three Experts (The Inputs)

GlaBoost takes information from three distinct "experts":

The Visual Expert (The Camera): This part looks at the eye photo. It uses a powerful "eye" (a pre-trained computer vision model called ResNet-152) to spot visual patterns, like the shape of the optic nerve, without getting distracted by the details.
The Data Analyst (The Spreadsheet): This part reads the structured numbers and categories. It looks at specific facts like "Cup-to-Disc Ratio: 0.8" or "Rim Thinning: True." It treats these like a checklist of hard facts.
The Translator (The Reader): This part reads the free-text notes doctors write. It uses a language model (mBERT) to understand the meaning behind words like "bayoneting" or "pallor," turning human descriptions into data the computer can understand.

2. The Team Leader (The Decision Maker)

Once these three experts gather their clues, they don't just mash them together into a messy pile. Instead, they pass their findings to a Team Leader called XGBoost.

Think of XGBoost as a very experienced coach who knows how to weigh different pieces of evidence.

If the Visual Expert says "It looks suspicious" but the Data Analyst says "The numbers are normal," the Coach knows how to balance those conflicting signals.
The Coach builds a "decision tree" (a flowchart of logic) to determine the final risk level.

3. The Results: Why It Won

The researchers tested this team on two different groups of real patient data (one public, one private). They compared GlaBoost against:

AI that only looks at photos.
AI that only reads text.
AI that tries to do both but uses different, more complex methods.
Even some of the newest, massive "Large Language Models" (like GPT-4 or Llama).

The Outcome:
GlaBoost won. It was the most accurate at predicting glaucoma risk.

The "Icing on the Cake": The study found that while the photo is the most important clue, adding the text notes and the structured numbers made the diagnosis even sharper.
The "Coach's Logic": Unlike some AI that acts like a "black box" (where you don't know why it made a decision), GlaBoost is transparent. The researchers asked the Coach, "Why did you say this patient is at risk?" The Coach pointed to specific, real-world medical facts: The Cup-to-Disc Ratio, Rim Thinning, and whether the eye followed the ISNT rule (a standard pattern for healthy nerves).

The Bottom Line

The paper claims that by combining a photo, a spreadsheet of numbers, and a doctor's notes into one organized system, GlaBoost creates a more accurate and trustworthy tool for spotting glaucoma. It proves that you don't need a giant, complex AI to solve this; sometimes, a smart, structured team that listens to all three types of clues works best.

What the paper does not claim:

It does not say this tool is currently being used in hospitals to treat patients.
It does not claim it can replace human doctors entirely.
It does not promise it will work on every single type of eye disease, only glaucoma risk stratification based on the data they provided.

In short, GlaBoost is a new, highly effective way to organize the clues a doctor already has, making the final diagnosis clearer and more reliable.

Technical Summary: GlaBoost – A Multimodal Structured Framework for Glaucoma Risk Stratification

1. Problem Statement

Glaucoma is a leading cause of irreversible blindness worldwide, requiring early and accurate diagnosis to prevent vision loss. However, current diagnostic processes are complex, relying on heterogeneous evidence including fundus images, anatomical measurements, and expert clinical descriptions. Existing AI approaches, particularly those based on Convolutional Neural Networks (CNNs), typically operate on a single visual modality. These unimodal systems fail to incorporate structured biomarkers (e.g., intraocular pressure, cup-to-disc ratio) and free-text clinical narratives that ophthalmologists routinely use. This "unimodal bias" limits both diagnostic accuracy and clinical interpretability, creating a mismatch between AI outputs and the multimodal nature of clinical reasoning.

2. Methodology: The GlaBoost Framework

The authors propose GlaBoost, a multimodal gradient boosting framework designed to unify three complementary signals for glaucoma risk prediction. The architecture consists of four distinct stages:

A. Input Decomposition

The input feature vector for each instance is decomposed into four modalities:

Structured Clinical Features ( $x_{struct}$ ): Extracted from machine-readable fields in clinical reports (e.g., optic disc size, cup-to-disc ratio). Categorical data is one-hot encoded, and missing values are explicitly handled.
Human-Evaluated Indicators ( $x_{human}$ ): Optional subjective inputs such as preliminary risk assessments and confidence levels provided by annotators. These are treated as auxiliary inputs rather than ground-truth labels.
Visual Features ( $x_{img}$ ): High-level embeddings extracted from fundus images using a pretrained ResNet-152 (frozen convolutional layers, fine-tuned projection layers).
Textual Features ( $x_{text}$ ): Unstructured clinical narratives (e.g., neuroretinal rim descriptions) encoded using a pretrained mBERT (multilingual BERT) transformer model with mean-pooling.

B. Fusion and Classification

All modalities are concatenated into a single high-dimensional representation ( $z_i$ ). This fused vector is fed into an enhanced XGBoost classifier. The model is optimized using logistic loss with regularization to control complexity. This approach leverages the tree-based boosting method's ability to handle heterogeneous feature types and provide feature importance metrics, contrasting with the "naïve feature concatenation" often seen in deep multimodal encoders.

3. Datasets and Experimental Setup

The framework was evaluated on two real-world datasets:

Public Dataset: JSON-formatted clinical reports paired with fundus images, containing structured biomarkers and free-text descriptions.
Private Dataset (UTSW): Derived from clinical OCT reports, containing 16 biomarkers across Retinal Nerve Fiber Layer (RNFL), Optic Nerve Head (ONH), and Ganglion Cell Complex (GCC) categories. This dataset includes a distinct "Borderline" category for biomarkers, adding complexity to the classification task.

Baselines: The study compared GlaBoost against a wide range of baselines, including:

Unimodal Models: CNNs (ResNet, ViT, ConViT), NLP models (BERT, Transformer, Mamba, RoBERTa), and traditional ML models (Logistic Regression, Random Forest).
Multimodal Baselines: Dual-stream architectures (e.g., CNN-LSTM, CNN-BERT) and fine-tuned Large Language Models (LLMs) such as GPT-4O, LLaMA, and DeepSeek.

4. Key Results

Performance

GlaBoost consistently outperformed all baselines on both datasets:

Public Dataset: Achieved 98.73% Accuracy, 99.01% Precision, and 98.83% F1-score.
Private Dataset: Achieved 99.21% Accuracy, 98.89% Precision, and 99.10% F1-score.
Comparison: While vision-only models (e.g., ViT, ConViT) achieved high accuracy (up to 100% on the public set), they lacked interpretability. Dual-stream multimodal encoders generally underperformed compared to GlaBoost, suggesting that tree-based fusion is more effective than deep concatenation for this specific task. LLMs, even when fine-tuned, trailed behind GlaBoost, indicating that the diagnostic signal is dominated by structured biomarkers rather than long-form textual reasoning.

Ablation Studies

Modality Impact: Structured biomarkers were the primary driver of performance. Image-only inputs performed well but lacked clinical interpretability. Text-only inputs performed poorly (~46-52% accuracy) but improved significantly when combined with structured data.
Human Factors: Including clinician-derived risk assessments and confidence levels acted as a performance booster ("icing on the cake"), particularly in high-precision scenarios.
Component Sensitivity: Deeper CNN backbones (ResNet-101/152) yielded saturated accuracy, while mBERT hyperparameters and XGBoost depth tuning provided consistent marginal gains.

Feature Importance

Analysis of the XGBoost model highlighted clinically consistent predictors:

Public Dataset: Cup-to-disc ratio, ISNT rule adherence, and rim thinning were the top features.
Private Dataset: Superior RNFL thickness, Cup-to-Disc Horizontal Ratio, and Average GCC thickness were dominant.
These findings align with established ophthalmologic practices, confirming the model's reliance on biologically plausible markers.

5. Significance and Contributions

The paper claims the following contributions:

Multimodal Integration: GlaBoost successfully unifies fundus images, structured biomarkers, and free-text clinical narratives within a single interpretable framework, addressing the gap between image-driven AI and clinical reasoning.
Interpretability: By utilizing XGBoost, the framework provides transparent feature importance analysis, identifying clinically established predictors (e.g., cup-to-disc ratio) as dominant factors, thereby enhancing clinical trust.
Performance Superiority: The framework demonstrates that a structured multimodal approach with gradient boosting outperforms both unimodal deep learning models and complex dual-stream architectures in glaucoma risk stratification.
Scalability: The approach offers a scalable foundation for decision support in ophthalmology, capable of handling heterogeneous data sources without requiring massive parameter counts typical of large foundation models.

The authors conclude that while GlaBoost offers a robust foundation, future work should explore ophthalmology-specific foundation models, biomedical language encoders, and validation across multi-site external cohorts.