📄 medicine

UWF-FM: A Foundation Model for Comprehensive Ultra-Widefield Retinal Assessment

UWF-FM is a novel self-supervised foundation model pretrained on over 630,000 unlabeled ultra-widefield retinal images that outperforms existing non-UWF models across diverse clinical tasks and devices, demonstrating robust generalizability and successful real-world deployment to enhance clinician diagnostic sensitivity without disrupting workflow.

Original authors: Yalin Zheng, Jianyang Xie, Xiuju Chen, Bingjie YAN, Mostafa Daho, Yanda Meng, Sarah Coupland, Heinrich Heimann, Rumana Hussain, Mohammed Lazouni, Mohammed Brixi-Nigassa, Quanyong Yi, Yitian Zhao, Danl

Published 2026-08-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yalin Zheng, Jianyang Xie, Xiuju Chen, Bingjie YAN, Mostafa Daho, Yanda Meng, Sarah Coupland, Heinrich Heimann, Rumana Hussain, Mohammed Lazouni, Mohammed Brixi-Nigassa, Quanyong Yi, Yitian Zhao, Danli Shi, Xiaoxin Li

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your eye is like a camera, but instead of just snapping a photo of the center of a scene, it can see the whole landscape, from the horizon to the edges. For decades, doctors have used a special kind of eye scan called "fundus photography" to check for diseases, but it was like taking a selfie: it only showed the middle of the retina, missing the important stuff happening at the edges. Then came "Ultra-Widefield" (UWF) imaging, which is like upgrading to a panoramic lens that captures the entire backyard of the eye in one shot. This is a game-changer because many eye problems, like tears or tumors, often hide in those far corners. However, teaching a computer to understand these giant, complex panoramic photos is tricky. It's like trying to teach a dog to recognize a whole forest instead of just a single tree; the computer gets confused by all the new details. This is where "Foundation Models" come in. Think of these as super-smart AI students that read millions of books (images) before they ever take a test. Usually, these students only read books about the "center of the forest," so when they are handed a panoramic photo, they struggle. The big question scientists have been asking is: Can we train a super-smart AI specifically on these giant panoramic eye photos so it becomes a true expert at spotting trouble anywhere in the eye?

Enter UWF-FM, a new kind of AI foundation model designed specifically to master these ultra-widefield retinal images. The researchers, led by a team from the University of Liverpool and partners around the world, decided to stop trying to force old AI models to work on these new, wide photos. Instead, they built a brand-new student from scratch. They fed this AI a massive library of 632,627 unlabeled panoramic eye images. The AI didn't just look at them; it studied them using a technique called "self-supervised learning," which is like a student trying to solve a puzzle by looking at the pieces and guessing how they fit together without a teacher telling them the answer. By doing this, the AI learned the deep, hidden patterns of the entire retina, from the center to the very edges.

Once this AI was "trained," the researchers put it to the test to see if it was actually smarter than the old models. They didn't just ask it to find one thing; they asked it to do a whole job description. They tested it on 27,641 labeled image-task pairs to see if it could:

Diagnose diseases: Spot 12 different eye conditions, including tumors.
Grade severity: Check how bad diabetic retinopathy was on a scale of 1 to 5.
Find specific clues: Identify 28 different types of retinal findings, like tiny bleeds or tears, whether they were in the center or the far corners.

The results were impressive. The new UWF-FM model consistently beat the older AI models that were trained on standard, non-widefield photos. It was like bringing in a local guide who knows the whole forest versus a tourist who only knows the main path. The new model achieved high scores (AUCs around 0.90 for tumor detection and 0.94 for general disease diagnosis), proving it understood the whole picture, not just the middle.

But the researchers didn't stop there. They wanted to know if this AI was a "one-trick pony" that only worked on the specific cameras it learned from. They tested it on images from different hospitals and even different camera brands (switching from Optos cameras to Zeiss cameras). Even without any extra training or "tuning" for these new cameras, the AI still performed better than the old models. This suggests that the AI learned the language of the retina itself, rather than just memorizing the look of a specific camera.

To see if this AI could actually help real doctors, they ran a special study with four ophthalmologists (eye doctors). They showed them 600 panoramic eye images twice: once alone, and once with the AI's help. The AI didn't make the decisions; it just whispered its "guesses" to the doctors. When the doctors used the AI's help, they got better at spotting diseases (higher sensitivity) without making many more mistakes (only a tiny drop in specificity). It was like having a super-observant assistant pointing out things the doctors might have missed, especially the tricky stuff in the corners. The doctors, particularly the less experienced ones, became better at their jobs with this help.

Finally, the team tested if this AI could actually work in a real hospital without causing chaos. They did a "silent deployment" at the Xiamen Eye Center. For 37,665 consecutive eye exams over two months, the AI ran in the background, analyzing every single image. It didn't interrupt the doctors, didn't change the workflow, and didn't even show its results to anyone during the test. It processed every single image with a 100% success rate, taking only about 1.1 seconds per exam. It proved that this powerful AI could be integrated into a busy hospital without breaking anything.

In short, this paper suggests that by training an AI specifically on the vast, wide-angle views of the retina, we can create a robust tool that understands the whole eye, works across different hospitals and cameras, helps doctors catch more diseases, and fits seamlessly into real-world medical workflows. It's a step toward a future where AI acts as a reliable, panoramic-sight partner for eye doctors, ensuring that no part of the retina is left in the dark.

Technical Summary: UWF-FM: A Foundation Model for Comprehensive Ultra-Widefield Retinal Assessment

Problem Statement
Ultra-widefield (UWF) retinal imaging offers comprehensive visualization of both posterior-pole and peripheral retinal pathology, addressing the limitations of conventional fundus photography which often misses peripheral abnormalities critical for conditions like diabetic retinopathy, retinal tears, and tumors. However, the heterogeneity of UWF data across diseases, clinical tasks, and imaging devices has limited the applicability of existing retinal foundation models. Most current models are pretrained on conventional, posterior-pole-centric fundus images, raising uncertainty about whether their learned representations transfer effectively to the broader anatomical coverage and wider spectrum of pathology found in UWF imaging. Furthermore, existing UWF AI studies typically focus on single clinical objectives (e.g., specific disease detection), lacking a unified framework capable of supporting the multi-level assessments required in routine practice, including disease diagnosis, severity grading, and fine-grained finding characterization.

Methodology
The authors developed UWF-FM, a foundation model specifically designed for UWF retinal imaging.

Pretraining: The model utilizes a Vision Transformer (ViT-L) backbone based on the DINOv3 framework. It was pretrained using self-supervised learning on 632,627 unlabeled UWF retinal images acquired from routine clinical practice at Xiamen Eye Center. The input was adapted to accept two-channel retinal images (choroidal and vascular contrast) rather than standard RGB.
Downstream Evaluation Framework: The model was evaluated within a clinically structured framework spanning three complementary levels of assessment:
1. Disease Diagnosis: Classification of 12 retinal diseases and tumors.
2. Diabetic Retinopathy (DR) Assessment: Five-scale severity grading and image quality/adequacy evaluation.
3. Retinal Finding Characterization: Detection of 28 specific retinal findings, covering both posterior and peripheral pathologies.
Adaptation Strategies: Performance was assessed under two regimes: full fine-tuning (FT) and frozen-feature (Probe) evaluation to determine the transferability of representations.
Generalization Testing: The model was tested on 27,641 labeled image–task pairs (internal) and 10,686 external images from independent institutions (Algeria, UK, China) and different imaging vendors (Optos and Zeiss CLARUS 500). Crucially, cross-vendor evaluation was performed without additional retraining or site-specific adaptation.
Clinical Utility Studies:
- Multi-Reader Study: A prospectively registered randomized crossover study (NCT07651943) involving four ophthalmologists evaluating 600 images to assess AI-assisted interpretation.
- Silent Deployment: A retrospective (45 days) and prospective (25 days) silent deployment on a hospital server processing 37,665 consecutive examinations to evaluate operational feasibility without influencing clinical workflows.

Key Results

Superior Representation Transfer: UWF-FM consistently outperformed foundation models pretrained on non-UWF data (including MAE-In1K, RETFound, and DINOV3). Under full fine-tuning, it achieved AUCs of 0.901 for tumor classification, 0.947 for multi-disease diagnosis, and 0.878 for retinal finding characterization on internal cohorts.
Robustness in Frozen-Feature Settings: The performance gap widened under frozen-feature evaluation, with UWF-FM showing up to 15% higher AUC than baselines. This indicates that the benefits arise from transferable retinal representations rather than task-specific optimization.
Cross-Vendor and Cross-Cohort Generalization: Without any site-specific adaptation, UWF-FM maintained its performance advantage on external Optos cohorts and when applied directly to Zeiss CLARUS 500 images. For instance, on Zeiss images for retinal finding characterization, the zero-adaptation model achieved an AUC of 0.801, only 0.042 lower than a model trained directly on Zeiss data.
Clinically Relevant Operating Points: Under clinically constrained thresholds (e.g., 80% fixed specificity for screening, 90% fixed sensitivity for diagnosis), UWF-FM demonstrated higher sensitivity and specificity compared to baselines across diverse disease categories, including referral-critical peripheral findings like retinal detachment and breaks.
Human-AI Collaboration: In the multi-reader study, AI assistance increased clinician sensitivity (mean $\Delta$ = 0.052) with only a modest reduction in specificity (mean $\Delta$ = -0.006), resulting in improved balanced accuracy and a reduced distance to the ideal ROC operating point.
Operational Feasibility: During silent deployment, UWF-FM achieved a 100% processing success rate across 37,665 examinations with a median inference time of approximately 1.1 seconds per exam, integrating seamlessly into existing hospital infrastructure without workflow disruption.

Significance and Claims
The paper establishes UWF-FM as the first foundation model specifically pretrained for ultra-widefield retinal imaging. The authors claim that UWF-specific pretraining yields representations that are inherently more generalizable across clinical tasks, patient populations, and imaging vendors than models trained on conventional fundus data.

The study demonstrates that a foundation model tailored to the expanded anatomical coverage of UWF imaging can support a scalable framework for end-to-end retinal care. By validating performance across multiple levels of assessment, external institutions, and real-world silent deployment, the work argues that UWF-FM provides a robust, generalizable, and clinically deployable platform for retinal decision support. The findings suggest that leveraging the broader anatomical context of UWF imaging through self-supervised learning is a critical step toward developing AI systems capable of handling the complexity of routine ophthalmic practice, particularly for identifying peripheral pathology that is often overlooked by standard imaging.

More like this