💻 computer science

Braided Vision Transformer for Stroke Detection in Multi-view Retinal Fundus Imaging

This paper introduces the Braided Vision Transformer (BViT), a novel deep learning model that leverages multi-view retinal fundus images from both eyes to detect stroke and transient ischemic attacks by simultaneously extracting features and modeling inter-view relationships, achieving an AUC of 0.75 on a newly collected dataset.

Original authors: Aysen Degerli, Mika Hilvo

Published 2026-08-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Aysen Degerli, Mika Hilvo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Stroke remains one of the world's most devastating health challenges, a leading cause of death and long-term disability that strikes without warning. For decades, doctors have relied on heavy, expensive machines like CT scanners and MRIs to see inside the brain and confirm a stroke. These tools are essential for treatment, but they are not always available for quick screening, especially in remote areas or for early warning signs. A growing body of research suggests that the key to spotting these dangers might lie not in the brain itself, but in the eye. The retina, the light-sensitive tissue at the back of the eye, is the only place in the body where doctors can directly observe blood vessels without surgery. Because the vessels in the eye and the brain are so similar, changes in the tiny blood vessels of the retina often mirror the health of the brain's own circulatory system. This connection has sparked a search for a faster, cheaper, and non-invasive way to assess stroke risk using nothing more than a photograph of the eye.

Researchers at the VTT Technical Research Centre of Finland have taken this concept a step further by developing a new computer system designed to read these retinal photographs with unprecedented depth. In a recent study, they introduced a method called the Braided Vision Transformer, a type of artificial intelligence that looks at the eye in a way no previous system has. Instead of analyzing a single image in isolation, this system examines four distinct views of the retina at once: the macula and the optic nerve head from both the left and right eyes. The macula is the central part of the retina responsible for sharp vision, while the optic nerve head is where the nerve fibers exit the eye to travel to the brain. By capturing these specific areas from both eyes, the system gathers a much richer picture of a person's vascular health than a single snapshot ever could.

The core innovation of this work lies in how the computer processes these four images. Traditional artificial intelligence models often treat each image as a separate puzzle piece, solving them one by one before combining the answers. The new Braided Vision Transformer, however, weaves the information together from the very beginning. It creates a structure where the computer constantly compares the blood vessel patterns in the left eye against those in the right eye, and the central view against the side view, all at the same time. This approach mimics how a skilled clinician might look at a patient, constantly shifting focus between different parts of the eye and both eyes to spot subtle inconsistencies that signal trouble. The researchers trained this system on a dataset of over 800 retinal images collected from more than 200 participants, including people who had suffered a stroke, those who had experienced a transient ischemic attack—a temporary blockage often called a mini-stroke—and healthy individuals.

The results of this training were promising. When tested on its ability to distinguish between healthy eyes and those showing signs of stroke or mini-stroke, the new system achieved a performance score of 0.75, a measure of accuracy that was higher than other standard computer vision models tested in the same study. The system proved particularly effective at recognizing the complex patterns associated with cerebrovascular events, outperforming models that looked at images individually or used older methods of combining data. The study also highlighted a crucial finding: looking at multiple views of the eye is significantly better than looking at just one. When the researchers tested a version of the system that only looked at a single image, its ability to detect stroke dropped dramatically, suggesting that the subtle signs of vascular disease are distributed across different parts of the retina and both eyes, and are easily missed if the view is too narrow.

While the system is not yet ready to replace the CT scanner in a hospital emergency room, it represents a significant leap forward in the potential for early screening. The researchers note that their approach offers a portable and cost-effective alternative that could one day be used in community clinics or even remote locations to flag individuals who need immediate, more detailed medical attention. The study confirms that the retina holds valuable clues about brain health and that modern artificial intelligence, when designed to look at the whole picture rather than just a part, can unlock those clues more effectively than before. By successfully weaving together multiple views of the eye, this new method suggests a future where the first line of defense against stroke might be as simple as a quick, painless photograph.

Technical Summary: Braided Vision Transformer for Stroke Detection in Multi-view Retinal Fundus Imaging

Problem Statement
Stroke and Transient Ischemic Attack (TIA) remain leading causes of global mortality and morbidity. While Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) are the gold standards for stroke evaluation, their clinical utility is limited by high costs, long acquisition times, and radiation exposure (in the case of CT). Retinal fundus imaging offers a non-invasive, portable, and cost-effective alternative, as retinal microvasculature alterations reflect cerebrovascular health. However, existing deep learning approaches for stroke assessment using retinal images face two primary limitations: (1) most studies utilize only single-view images from one eye, failing to capture inter-eye vascular variations; and (2) existing multi-view methods rely on Convolutional Neural Networks (CNNs), while the potential of Vision Transformers (ViTs) for this specific task remains unexplored. Furthermore, TIA detection has been largely overlooked in favor of ischemic stroke.

Methodology
The authors propose the Braided Vision Transformer (BViT), a novel architecture designed to analyze multi-view retinal fundus images from both eyes.

Input Data: The model processes a set of four images per patient: macula-centric ( $v_2$ ) and optic nerve head-centric ( $v_1$ ) views from both the left ( $l$ ) and right ( $r$ ) eyes.
Architecture:
- Embedding: Each of the four input images is divided into patches, flattened, and augmented with positional embeddings.
- Braided Transformer Modules: Unlike standard ViTs where channels are processed independently or mutual transformers that connect only pairs of modalities, BViT employs a "braided" structure. This involves two sequential stages of transformer blocks:
  1. First Stage: The four image embeddings ( $\dot{a}, \dot{b}, \dot{c}, \dot{d}$ ) are processed via mutual attention mechanisms where queries from one embedding attend to keys and values from another, establishing initial cross-channel interactions.
  2. Second Stage: The outputs of the first stage are further processed in a second braided block, creating a complex intersectional connectivity among all input channels. This design mimics clinical practices by simultaneously modeling intra-eye (different views) and inter-eye (left vs. right) relationships.
- Classification: The final feature vectors are flattened, concatenated, and passed through fully connected layers with ReLU and Softmax activations for binary (Stroke/TIA vs. Healthy) or multi-class (Stroke, TIA, Healthy) classification.
Dataset and Training: The model was trained on the Stroke-Data dataset (220 participants: 26 TIA, 73 stroke, 121 healthy controls), comprising 802 retinal images. Due to data scarcity, extensive data augmentation (rotation, flipping, shear, zoom) was applied to reach 12,000 training samples. The model utilized a stratified 5-fold cross-validation scheme with an Adam optimizer and categorical cross-entropy loss.

Key Contributions

First ViT for Retinal Stroke Assessment: To the best of the authors' knowledge, this is the first study to apply Vision Transformers to retinal fundus imaging for stroke and TIA detection.
Novel Braided Architecture: The introduction of the BViT model, which utilizes a specific braided structural form to capture inter-view relationships across both eyes simultaneously, distinguishing it from standard ViTs and mutual transformer modules.
Multi-view Learning Paradigm: The study demonstrates the superiority of multi-view learning (utilizing both eyes and two focal views) over single-view approaches in capturing the heterogeneous retinal biomarkers associated with cerebrovascular events.

Experimental Results
Experiments were conducted on the Stroke-Data dataset comparing BViT against baseline models (Vanilla ViT, Mutual ViT, and CNN-based MVS-Net variants).

Binary Classification (Stroke/TIA vs. Healthy): BViT achieved the highest performance with an AUC of 0.752, outperforming the Vanilla ViT (0.729), Mutual ViT (0.717), and the best CNN baseline, MVS-Net-v1 (0.707). BViT also achieved a sensitivity of 0.647 and specificity of 0.727.
Multi-class Classification: In distinguishing between Stroke, TIA, and Healthy controls, BViT achieved the highest AUC for Healthy controls (0.716) and TIA patients (0.618). For the Stroke class, it achieved an AUC of 0.715, slightly behind the Mutual ViT (0.719).
Single-view vs. Multi-view: A critical ablation study compared the multi-view BViT against a single-view ViT baseline. The single-view model achieved a poor AUC of 0.49, whereas the multi-view approach reached 0.71, highlighting the necessity of aggregating information across views and eyes.
Model Efficiency: Despite being a transformer-based model, BViT (approx. 8 million parameters) performed competitively against larger CNN architectures like MVS-Net-v2 (approx. 35 million parameters).

Significance and Claims
The paper claims that the BViT model successfully demonstrates the feasibility of using retinal fundus imaging for rapid, non-invasive stroke risk assessment. The authors emphasize that the braided architecture effectively mimics clinical evaluation practices by leveraging complementary vascular and structural information from multiple views and both eyes. While the results show promise, the authors maintain a modest tone, noting that neuroimaging (CT/MRI) remains essential for therapeutic decision-making. The study establishes a foundation for future transformer-based frameworks in ischemic stroke prediction but acknowledges the clinical difficulty in discriminating TIA from stroke during early diagnostic phases, evidenced by lower sensitivity in multi-class TIA detection.

Technical Summary: Braided Vision Transformer for Stroke Detection in Multi-view Retinal Fundus Imaging

More like this