💻 computer science

FunPiQ: A New Benchmark for Pixel-Level Quality Assessment in Fundus Images

This paper introduces FunPiQ, the first benchmark for pixel-level fundus image quality assessment based on anatomical visibility, and proposes the explainable-by-design EFIQA-CP method, which demonstrates superior performance in evaluating localized degradations compared to existing image-level approaches.

Original authors: Pengwei Wang, José Morano, Virginia Mares, Hrvoje Bogunović

Published 2026-06-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pengwei Wang, José Morano, Virginia Mares, Hrvoje Bogunović

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor trying to look at the back of a patient's eye (the retina) through a camera. This is called a fundus photo. Sometimes, the photo comes out blurry, dark, or has weird shadows. If the photo is bad, the doctor can't see the details needed to diagnose diseases.

For a long time, computers have been trying to automatically decide: "Is this photo good enough to use?" But there's a problem with how they've been doing it.

The Old Way: Judging the Whole Picture

Previously, computers looked at the entire photo and gave it a single grade, like a teacher giving a student a "C" for the whole test.

The Flaw: A photo might have a perfectly clear center but a blurry edge. If the computer says "Bad" because of the edge, the doctor might throw away a photo that was actually useful for the center.
The Confusion: Different doctors have different ideas of what "good" means. One doctor might need to see blood vessels clearly, while another just needs to see the optic nerve. This made it hard to create a universal rule for what a "good" photo looks like.

The New Solution: FunPiQ (The "Pixel-by-Pixel" Report Card)

The authors of this paper created a new tool called FunPiQ. Instead of grading the whole photo, FunPiQ acts like a high-resolution heat map. It looks at every single tiny dot (pixel) in the image and decides:

Good: "I can see the details clearly here."
Usable: "It's a bit fuzzy, but I can still make out the shape."
Bad: "This spot is too dark, blurry, or blocked to see anything."

This is like giving a student a report card that doesn't just say "Math: F," but instead highlights exactly which numbers on the page were written in pencil and which were written in ink. It removes the guesswork about where the problem is.

The New Method: EFIQA-CP (The Smart Detective)

To use this new "pixel-by-pixel" system, the authors built a new AI detective called EFIQA-CP.

The Problem it Solves: The AI needs to learn what "bad" looks like, but it doesn't have a teacher to show it every single bad spot. Instead, it has to guess based on clues (like: "If I can't see the blood vessels here, this spot is probably bad"). These guesses are often noisy and unreliable, like a detective working with blurry surveillance footage.
The Trick: The authors gave the AI two special tools:
1. A Wide-Angle Lens: Instead of looking at one tiny dot at a time, the AI looks at a broader neighborhood around each dot. This helps it understand context (e.g., "The center of the eye naturally has fewer vessels, so don't call that 'bad' just because it's empty").
2. A "Don't Panic" Filter: Because the AI's initial guesses are messy, they taught it a special learning rule (called nnPU) that prevents it from overreacting to its own mistakes. It learns to be confident only when it's sure, and to ignore the confusing parts.

The Results: Who Won the Race?

The authors tested their new detective (EFIQA-CP) against other existing AI methods using their new "pixel-by-pixel" test (FunPiQ).

The Winner: EFIQA-CP was the clear champion. It was better at pinpointing exactly where the photo was blurry or dark.
The Runner-up: The previous version of this detective (EFIQA) did well but sometimes got confused in areas that naturally look empty (like the very center of the eye).
The Losers: Other methods that just looked at the whole image or tried to find "anomalies" without specific training were often too noisy or missed large blurry areas.

Why This Matters (According to the Paper)

The paper argues that this new way of thinking is better because:

It's Explainable: Instead of a black box saying "Reject this image," it shows a map of why (e.g., "The top left is too dark").
It's Flexible: Because it looks at the anatomy (the actual eye parts) rather than just a generic "blurry" label, it can adapt to different medical needs.
It Helps Non-Experts: If a nurse takes a photo, the system can tell them, "The center is fine, but the edge is too dark; please take another one," rather than just saying "Bad photo."

In short, the paper introduces a new, more precise way to grade eye photos by looking at the details rather than the whole picture, and they built a smarter AI to do the grading.

Technical Summary: FunPiQ and EFIQA-CP

Problem Statement
Color fundus photography (CFP) is the primary modality for large-scale ophthalmic screening but is highly susceptible to local degradations (e.g., inconsistent illumination, poor contrast, defocus) that often affect specific regions rather than the entire image. Current Fundus Image Quality Assessment (FIQA) methods rely on image-level labels, which vary significantly across datasets and clinical tasks. A definition of "high quality" that is suitable for one disease (e.g., glaucoma) may be insufficient for another. This task-dependency renders manual image-level assessment subjective and time-consuming. Furthermore, existing image-level labels preclude the quantitative evaluation of localized degradations, limiting the development of trustworthy, explainable FIQA systems. While some methods attempt to localize degradations, they often rely on noisy pseudo-labels or lack global context, leading to imprecise localization (e.g., misclassifying naturally low-vessel-density areas like the fovea as degraded).

Methodology
The paper introduces FunPiQ, a new benchmark dataset, and EFIQA-CP, a novel method for pixel-level quality assessment.

FunPiQ Dataset:
- Composition: The dataset comprises 300 images sourced from three existing benchmarks (EyeQ from the US; BRSET and mBRSET from Brazil) to ensure diversity in ethnicity, acquisition devices (tabletop to mobile), and degradation types.
- Annotation Strategy: Unlike previous datasets with global labels, FunPiQ provides pixel-level quality annotations. Images were annotated by expert readers under the supervision of a board-certified ophthalmologist.
- Classes: Three ordinal categories are defined based on anatomical visibility:
  - Good: Sharply delineated retinal structures with well-defined borders.
  - Usable: Identifiable structures with mild reduction in sharpness/contrast.
  - Bad: Non-gradable regions due to severe artifacts, signal dropout, or media opacity.
- Selection: The dataset specifically samples images previously labeled as "bad" in source datasets to ensure a mix of good, usable, and bad regions within each image, testing a model's ability to differentiate quality locally.
EFIQA-CP Method:
- Architecture: The method builds upon the EFIQA pipeline, utilizing a frozen DINOv3 backbone for feature extraction. It replaces the standard linear adapter with a shallow, wide ConvNeXt-style network featuring 7×7 depthwise convolutions. This design integrates spatial information through a larger receptive field to incorporate global context, addressing the issue of misclassifying low-vessel-density regions.
- Learning Strategy: To handle the inherent noise in pseudo-labels (generated by a Vessel Unsupervised Anomaly Detection model based solely on vessel absence), the authors employ Non-Negative Positive-Unlabeled (nnPU) learning.
  - A strict threshold isolates reliable low-quality pixels (positive class), treating the remainder as unlabeled.
  - The nnPU framework prevents the estimated risk of hidden negatives from dropping below zero, stabilizing training and mitigating pseudo-label overfitting.
  - The objective function uses a softplus surrogate loss with a positive class prior ( $\pi = 0.05$ ).

Key Contributions

FunPiQ Benchmark: The first public benchmark dedicated to pixel-level quality assessment for CFP, providing ground-truth annotations for localized degradation.
EFIQA-CP: A novel method that improves upon existing explainable-by-design (EBD) approaches by combining a wide-receptive-field adapter with nnPU learning to handle noisy pseudo-labels and improve spatial precision.
Comprehensive Evaluation: A thorough benchmarking of existing FIQA methods (including end-to-end, EBD, and anomaly detection approaches) on the new dataset, establishing a baseline for pixel-level performance.

Results
Extensive evaluations were conducted on the FunPiQ dataset using Quadratic Weighted Kappa (QWK) and mean Sørensen–Dice coefficient (mDice), alongside binary "Reject" task metrics (AUROC, AUPRC, S95).

Performance: EFIQA-CP achieved state-of-the-art performance across all metrics and datasets (BRSET, EyeQ, mBRSET, and the merged set). It demonstrated statistically significant improvements over the original EFIQA (e.g., $p < 0.0001$ for QWK on the merged dataset).
Comparison: EBD methods (EFIQA-CP, EFIQA, EyeQual) consistently outperformed end-to-end supervised methods (MCF-Net, FGR-Net) and general anomaly detection methods (PatchCore, UniVAD).
Qualitative Analysis: EFIQA-CP produced the most precise quality maps with smooth regions and better utilization of the score range. While EyeQual showed high precision for "good" classes, it struggled to distinguish "usable" from "bad." General anomaly detection methods tended to highlight degraded regions but produced noisy maps with high scores across patches, suggesting they are biased toward compact anomalies rather than large-scale quality degradations.
Ablation: Replacing the nnPU loss with standard Binary Cross-Entropy (BCE) led to overfitting and performance degradation, confirming the necessity of the nnPU framework for this noisy-label setup.

Significance and Claims
The authors argue that pixel-level FIQA based on anatomical visibility represents a more task-agnostic and explainable approach than image-level assessment. By providing a benchmark with pixel-level ground truth, FunPiQ enables the quantitative evaluation of localization precision, which was previously impossible.

The paper claims that explainable-by-design (EBD) approaches hold a distinct advantage for local quality prediction. Specifically, EFIQA-CP demonstrates that combining architectural improvements (global context) with robust learning strategies (nnPU) can effectively mitigate the noise of pseudo-labels. The authors posit that pixel-level maps offer direct clinical value by supporting quality assurance for non-expert operators (identifying where an image is insufficient) and guiding targeted image reacquisition. Furthermore, these maps could serve as priors for image enhancement models to distinguish anatomy from artifacts. The work aims to encourage further research into pixel-level CFP quality detection and the development of more robust, flexible FIQA methods with stronger clinical applicability.