UWF-RI2FA: Generating Multi-frame Ultrawide-field Fluorescein Angiography from Ultrawide-field Retinal Imaging Improves Diabetic Retinopathy Stratification

This study presents UWF-RI2FA, a generative AI model that synthesizes realistic, multi-frame, dye-free ultrawide-field fluorescein angiography images from noninvasive retinal imaging, significantly improving diabetic retinopathy stratification accuracy compared to using retinal imaging alone.

The Gist

The Big Idea: A "Magic Filter" for Eye Scans

Imagine you have a camera that can take a picture of your entire eye, including the very edges (the periphery), which is usually hard to see. This is called Ultrawide-field Retinal Imaging (UWF-RI). It's like taking a panoramic photo of a landscape.

Now, to really understand if a patient has Diabetic Retinopathy (a serious eye disease caused by diabetes), doctors usually need a second type of picture called Fluorescein Angiography (UWF-FA). This shows the blood vessels lighting up like a neon map.

The Problem: To get this "neon map," a doctor has to inject a yellow dye into the patient's arm. It's invasive, can make people feel sick (nausea, shock), and is expensive. Because of this, many people skip this crucial test, and doctors might miss dangerous spots on the edge of the retina.

The Solution: The researchers in this paper built an AI "magic filter." They taught a computer to look at the normal, safe panoramic photo (UWF-RI) and generate the "neon map" (UWF-FA) automatically, without ever injecting a single drop of dye.

---

How They Did It: The "Master Chef" Analogy

Think of the AI model as a Master Chef who has never tasted a specific dish but has seen millions of photos of the ingredients and the final plated meal.

1. The Training (Learning the Recipe):
   The researchers gathered a massive library of 18,321 pairs of images. In every pair, they had the "safe photo" (the ingredients) and the "dye photo" (the finished dish) from the same patient.
   They fed these pairs into a Generative Adversarial Network (GAN). You can think of this as a game between two chefs:

   • Chef A (The Generator): Tries to paint a fake "dye photo" based on the "safe photo."
   • Chef B (The Discriminator): Tries to spot the fake.
     They played this game over and over. Chef A got better at painting, and Chef B got better at spotting fakes, until Chef A could paint a "dye photo" so realistic that even the experts couldn't tell the difference.

2. The Dynamic Movie (Not Just a Still):
   Real dye tests happen in phases: early (arteries light up), mid (capillaries fill), and late (veins fill). Most AI only makes one static image. This team taught their AI to make three different frames (Early, Mid, and Late), effectively creating a short "movie" of how the blood flows, all from a single static photo.

3. The Alignment (The Puzzle Piece):
   Taking a wide-angle photo of an eye is tricky; the edges get distorted (like a fisheye lens). To make the AI work, the researchers had to perfectly align the "safe photo" with the "dye photo" pixel-by-pixel, like matching two jigsaw puzzles that were slightly warped. They used the blood vessels as the guide to stitch them together perfectly.

---

Did It Work? The "Turing Test" for Eyes

The team put their AI-generated images to the ultimate test: The Turing Test.

They showed 50 pairs of images to two expert eye doctors. One set was real dye photos, and the other was the AI's fake ones. The doctors had to guess which was which.

• The Result: The doctors were fooled 56% to 76% of the time. In other words, the AI's fake images looked so real that the experts often thought they were real.
• The Quality: On a scale of 1 to 5 (where 1 is "perfectly real"), the doctors rated the AI images around 1.6 to 1.9. That is incredibly close to a perfect 1.

---

Why Does This Matter? The "Superpower" for Diagnosis

The most important part of the study wasn't just making pretty pictures; it was about saving sight.

The researchers took their AI-generated "dye photos" and added them to a computer program designed to diagnose diabetic retinopathy.

• Without the AI: The computer looked at the safe photos and got a score of 0.869 (a measure of how good it was at spotting disease).
• With the AI: When the computer was allowed to "see" the AI-generated dye maps, its score jumped to 0.904.

The Analogy: Imagine a detective trying to solve a crime.

• Scenario A: The detective only has a black-and-white photo of the scene. They can guess who did it, but they might miss clues.
• Scenario B: The detective has the black-and-white photo plus a thermal imaging overlay that shows exactly where the suspect was standing.
• Result: The detective solves the case much faster and more accurately.

By adding the AI-generated "thermal overlay" (the dye map), the system became significantly better at spotting severe diabetic retinopathy.

The Bottom Line

This paper introduces a safe, non-invasive, and cheaper way to get the detailed blood vessel maps that doctors need to save eyes.

Instead of sticking a needle in a patient's arm to inject dye, the doctor can just take a quick, safe photo of the eye, and the AI will instantly generate the "dye map" for them. It's like having a time machine that lets you see the future flow of blood without ever having to wait for the dye to circulate.

In short: They taught a computer to "hallucinate" the truth, and that hallucination is so accurate it helps doctors catch eye disease earlier and safer than ever before.

Technical Summary

1. Problem Statement

• Clinical Need: Ultrawide-field fluorescein angiography (UWF-FA) is the gold standard for detecting peripheral retinal lesions and stratifying Diabetic Retinopathy (DR) severity. However, it requires intravenous dye injection, which carries risks (allergic reactions, shock, vomiting) and limits its use as a routine screening tool.
• Technological Gap: While non-invasive Ultrawide-field Retinal Imaging (UWF-RI) exists, it lacks the vascular contrast provided by FA. Existing Generative AI (GenAI) solutions for cross-modal translation (e.g., fundus to FA) are limited to standard 55° fields of view. They fail to address the complexities of UWF imaging (200° field of view), such as peripheral distortion, eyelid artifacts, and the need to generate multi-frame (dynamic) images that mimic the temporal phases of retinal circulation.

2. Methodology

The study proposes a UWF-RI2FA framework using Generative Adversarial Networks (GANs) to synthesize dye-free, multi-phase UWF-FA images from non-invasive UWF-RI images.

A. Data Collection & Preprocessing

• Dataset: 1,352 patients from The Second People's Hospital of Foshan provided 3,885 UWF-RI and 23,332 UWF-FA images (Optos California, 200° FOV, 4000×4000 resolution).
• Phases: UWF-FA images were categorized into Early (25–60s), Mid (1–5 min), and Late (>5 min) phases post-injection.
• Cross-Modal Registration:
  • Low-quality peripheral regions (eyelids, iris vignetting) were cropped.
  • Vessel-based Alignment: The Retina-based Microvascular Health Assessment System (RMHAS) segmented retinal vessels.
  • Feature Matching: AKAZE detectors identified key points on vessel maps, and RANSAC generated homography matrices to align UWF-RI with UWF-FA.
  • Quality Control: Pairs with a Dice coefficient < 0.5 or excessive rotation/scaling were excluded.
  • Final Dataset: 18,321 registered pairs (1,863 early, 10,275 mid, 6,183 late).

B. Model Architecture

• Base Model: A modified pix2pixHD (a high-resolution GAN).
• Transfer Learning: The model was initialized with weights pre-trained on 55° standard-field RI-FA pairs to leverage existing knowledge of lesion-to-angiography mapping.
• Loss Function Modification: To enhance high-frequency details (retinal structures and lesions), Gradient Variance Loss was added to the standard adversarial and pixel-reconstruction losses.
• Training Strategy: Three independent models were trained for the three distinct phases (Early, Mid, Late). Data augmentation included random resizing (0.3–3.5x), flipping, and rotation (0–45°).

C. Evaluation Framework

1. Quantitative Metrics: Mean Absolute Error (MAE), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Multi-scale SSIM (MS-SSIM).
2. Human Evaluation:
   • Visual Quality: Two ophthalmologists rated images on a 5-point scale (1 = Real quality, 5 = Poor).
   • Turing Test: Experts attempted to distinguish real vs. generated images.
3. Clinical Utility (Downstream Task):
   • External Dataset: The DeepDRiD dataset (256 UWF-RI images) was used.
   • Task: DR severity classification (Grades 0–4).
   • Comparison: Four groups were tested using a Swin Transformer classifier:
     1. Real UWF-RI only (Baseline).
     2. Real UWF-RI + Generated Early FA.
     3. Real UWF-RI + Generated Early + Mid FA.
     4. Real UWF-RI + Generated Early + Mid + Late FA.

3. Key Results

Image Generation Quality

• Quantitative: The generated images showed high fidelity.
  • MS-SSIM: Ranged from 0.70 to 0.91 across phases.
  • PSNR: Ranged from 27.49 to 29.09.
• Human Evaluation:
  • Visual Scores: Mean scores were close to 1 (indicating real quality), ranging from 1.64 to 1.98. Inter-rater agreement (Kappa) was high (0.79–0.84).
  • Turing Test: In 56% to 76% of cases, generated images were mistaken for real images by expert ophthalmologists, demonstrating high authenticity.

Clinical Impact (DR Stratification)

Using the DeepDRiD external dataset, integrating generated FA images significantly improved DR classification performance compared to the baseline (UWF-RI only):

• Baseline (UWF-RI only): AUROC = 0.869.
• Full Integration (All 3 phases): AUROC increased to 0.904 ($P < .001$).
• Other Metrics: Significant improvements were also observed in AUPR (0.717 $\to$ 0.785) and F1-score (0.599 $\to$ 0.625).
• Conclusion: Adding multi-phase synthetic FA data provided critical information that enhanced the model's ability to detect and grade DR severity.

4. Key Contributions

1. First Multi-Frame UWF Generation: This is the first study to successfully generate dynamic, multi-phase (early, mid, late) UWF-FA images from non-invasive UWF-RI, capturing the temporal evolution of retinal circulation.
2. Large-Scale Cross-Modal Registration: Developed a robust pipeline to register 200° UWF images, overcoming challenges like peripheral distortion and artifacts that previously hindered high-quality translation.
3. Enhanced Loss Function: The integration of Gradient Variance Loss into the GAN architecture proved effective in preserving high-frequency details (lesions and fine vessels) in the generated images.
4. Clinical Validation: Demonstrated that synthetic dye-free FA images are not just visually realistic but functionally valuable, significantly improving the accuracy of automated DR screening systems.

5. Significance and Limitations

• Significance: The study offers a safe, cost-effective, and non-invasive alternative to traditional UWF-FA. By eliminating the need for intravenous dye, it enables wider adoption of high-precision DR screening, particularly in primary care or telemedicine settings where dye injection is impractical or risky.
• Limitations:
  • Peripheral Artifacts: Despite registration efforts, peripheral distortion and artifacts (e.g., from eyelashes) still occasionally affect the quality of generated peripheral lesions.
  • Dataset Scope: The model was trained on a specific hospital dataset; real-world validation across diverse populations and devices is needed.
  • Application Scope: Currently validated only for DR; further research is required to extend applicability to other retinal diseases (e.g., vein occlusions, macular degeneration).

In conclusion, UWF-RI2FA represents a paradigm shift in ophthalmic imaging, leveraging GenAI to bridge the gap between non-invasive screening and invasive diagnostic gold standards, thereby improving the management of diabetic retinopathy.