Through BrokenEyes: How Eye Disorders Impact Face Detection?

This paper presents a computational framework using the BrokenEyes system to simulate five common eye disorders and analyze their specific impacts on deep learning feature representations, revealing critical disruptions in models trained under degraded visual conditions.

The Gist

Imagine your brain is a super-smart security guard at a club, and its job is to look at people walking by and decide, "That's a human!" or "That's a statue!" This security guard is very good at its job when the lights are bright and the view is clear.

But what happens if the security guard puts on different kinds of bad glasses? Or if the camera lens they are looking through gets foggy, scratched, or covered in spots?

This paper, titled "Through BrokenEyes," is like a scientific experiment where researchers built a computer program to simulate exactly that. They wanted to see how five common eye problems mess up a computer's ability to recognize faces.

Here is the breakdown of their experiment in simple terms:

1. The "BrokenEyes" Filter (The Bad Glasses)

The researchers created a special digital tool called BrokenEyes. Think of this as a set of five different "bad glasses" they put over every photo in their database. Each pair of glasses mimics a specific real-world eye disease:

• The Cataract Glasses (The Foggy Window): Imagine looking through a window covered in thick fog. You can see shapes, but everything is blurry and washed out. This simulates cataracts, where the eye's lens gets cloudy.
• The Glaucoma Glasses (The Tunnel Vision): Imagine looking through a long, narrow pipe. You can see what's right in front of you, but everything on the sides is pitch black. This simulates glaucoma, which eats away at your side vision.
• The AMD Glasses (The Black Hole in the Middle): Imagine a dark, blurry spot right in the center of your vision, like a smudge on a camera lens that you can't wipe off. This simulates Age-related Macular Degeneration, which ruins your ability to see details in the center.
• The Refractive Error Glasses (The Out-of-Focus Camera): Imagine your camera lens is just slightly out of focus. The whole picture is a bit fuzzy, but you can still make out shapes. This simulates nearsightedness or farsightedness.
• The Retinopathy Glasses (The Spotted Lens): Imagine looking through a window with random black spots or dirt scattered all over it. This simulates Diabetic Retinopathy, where damage to the retina creates blind spots.

2. The Experiment (The Security Guard Training)

The researchers took thousands of photos of people and non-people (like cars or trees) and ran them through these "BrokenEyes" filters.

Then, they trained a computer brain (a neural network called ResNet18) to play a game: "Is this a human or not?"

• First, they trained it on clear photos (Normal Vision). The computer got 100% right. It was a perfect security guard.
• Next, they trained separate versions of the computer on each of the five blurry/distorted sets of photos.

3. The Findings (How the Computer Got Confused)

The big question was: How did the computer's "brain" change when the input was bad?

They didn't just look at whether the computer got the answer right or wrong. They looked inside the computer's brain to see how it was "thinking" about the images. They used two main tools to measure this:

• Cosine Similarity (The "Vibe Check"): This measures how much the computer's internal thoughts about a blurry face look like its thoughts about a clear face.

  • High Score: The computer is thinking almost the same way as usual.
  • Low Score: The computer is completely confused; its internal map of the face is totally different.

• Activation Energy (The "Effort Meter"): This measures how hard the computer is working.

  • High Energy: The computer is screaming, "I'm trying so hard to find the face!" (It's working overtime).
  • Low Energy: The computer is giving up or not seeing enough to get excited.

The Results:

• The Worst Offenders: Cataracts and Glaucoma caused the biggest chaos.
  • The computer's "vibe check" score dropped massively. It was like the security guard was looking at a completely different person.
  • The "Foggy Window" (Cataracts) made the computer work the hardest (highest energy) because it was trying to find edges that were gone.
  • The "Tunnel Vision" (Glaucoma) confused the computer the most because it lost the context of the whole picture.
• The Survivors: Refractive Errors (fuzzy vision) and Retinopathy (spotted vision) didn't mess things up as much. The computer could still figure out the face, even if the picture wasn't perfect. It was like the security guard squinting a bit but still recognizing the person.

4. Why Does This Matter?

This isn't just about computers; it's about understanding us.

The computer's brain is designed to work somewhat like the human brain. When the computer struggled with the "Foggy Window" (Cataracts) or "Tunnel Vision" (Glaucoma), it mirrored what happens to humans with those conditions. It showed us that when we lose specific parts of our vision, our brains (and computers) have to completely reorganize how they process information.

The Takeaway:
This study gives us a "digital twin" to test how eye diseases affect our ability to see the world. By understanding exactly how these disorders break the system, we can build better AI assistants for the future. Imagine an app for someone with glaucoma that knows exactly what parts of the image are missing and helps fill in the gaps, making the world easier to navigate.

In short: If you want to build a robot that can see like a human with bad eyes, you first have to teach it what "bad eyes" actually feel like. This paper taught the robot how to see through broken eyes.

Technical Summary

1. Problem Statement

Face detection and recognition are fundamental computer vision tasks that rely heavily on the integrity of visual input. However, millions of people suffer from visual impairments (e.g., cataracts, glaucoma, AMD) that alter how visual signals are processed. While prior research has explored human perception under these conditions, there is a lack of computational frameworks that explicitly simulate these disorders to analyze their specific impact on hierarchical feature representations in deep learning models.

The core problem addressed is: How do specific eye disorders degrade the internal feature maps of a neural network, and how does this degradation affect the model's ability to distinguish between human and non-human faces?

2. Methodology

The study proposes a controlled computational pipeline to simulate visual impairments and analyze their effects on a ResNet18 backbone.

A. The "BrokenEyes" Framework

The authors developed a novel filter generation system called BrokenEyes to simulate five common eye disorders by applying specific degradations to raw images:

1. Glaucoma: Simulated via a vignette effect (circular transparent center fading to black edges) to mimic tunnel vision and peripheral loss.
2. Refractive Errors: Simulated using randomized Gaussian blur with varying kernel sizes to mimic defocus (myopia, hyperopia, astigmatism).
3. Age-related Macular Degeneration (AMD): Simulated by introducing a central darkened region with blurred edges (central scotoma) to mimic central vision loss.
4. Diabetic Retinopathy: Simulated by adding random black elliptical shapes (floaters/patches) to mimic localized retinal damage.
5. Cataract: Simulated via desaturation (reduced color contrast), haze, and heavy Gaussian blur to mimic clouding of the lens.

B. Dataset and Experimental Setup

• Datasets: A combination of LFW (Labelled Faces in the Wild) for the "Human" class and MS-COCO 2017 for the "Non-Human" class.
• Preprocessing: Images were resized, normalized, and filtered to ensure balanced class distribution across six conditions (Normal + 5 disorders).
  • Total Samples: ~8,484 Human and ~7,854 Non-Human images.
• Model Architecture: ResNet18 was used as the backbone.
• Training Protocol:
  • Transfer Learning: Models were initialized with ImageNet weights.
  • Fine-tuning: A binary classifier (Human vs. Non-Human) was trained on each disorder-specific dataset.
  • Strategy: The backbone was frozen initially, then Layer 4 (the final convolutional block) was unfrozen for fine-tuning to adapt to specific distortions.
• Evaluation Metrics:
  • Cosine Similarity: Measures the angular alignment between feature maps of the disorder-specific model and the normal (baseline) model.
  • Activation Energy: The sum of absolute activations ($E = \sum |A_{i,j,k}|$) to measure the magnitude of network response.

3. Key Contributions

1. BrokenEyes Framework: A novel, clinically inspired image degradation system that realistically simulates five distinct eye disorders while preserving semantic content.
2. Disorder-Aware Training Pipeline: A controlled experimental setup using matched human/non-human datasets to train and evaluate ResNet18 models under specific visual impairments.
3. Feature-Level Analysis: A quantitative approach using activation energy and cosine similarity to map how visual degradation causes "representation drift" in deep neural networks, linking computational outputs to known neuroscientific deficits (e.g., V1 and FFA processing).

4. Key Results

The study evaluated the models on a held-out test set and compared feature maps from the final convolutional layer (Layer 4).

• Classification Confidence:

  • The normal model achieved 100% accuracy.
  • Under disorder conditions, confidence scores dropped significantly. AMD (0.6795) and Glaucoma (0.7060) showed the largest drops, indicating that central and peripheral vision loss severely impacts the model's certainty in identifying faces.
  • Refractive errors and Retinopathy maintained higher confidence (>0.85), suggesting the model can compensate for these distortions better.

• Feature Representation Drift (Cosine Similarity):

  • Glaucoma: Lowest similarity (0.4551), indicating severe misalignment of feature geometry due to peripheral vision loss.
  • Cataract: Low similarity (0.6350) and the highest activation energy (30,180.73). This suggests the network struggles with edge detection and contrast, forcing it to over-activate to find salient features.
  • AMD: High similarity (0.9344). Despite central vision loss, the model retained feature geometry, likely because peripheral inputs (which AMD spares) were sufficient to maintain the structural representation.
  • Refractive Error & Retinopathy: High similarity (>0.83), indicating minimal disruption to the learned feature space.

• Visual Evidence: Heatmaps of feature map differences confirmed that Cataract and Glaucoma caused the most widespread spatial deviations, particularly in high-frequency regions, whereas AMD and refractive errors showed more localized or minor deviations.

5. Significance and Implications

• Neuroscientific Alignment: The computational results mirror known human neural processing challenges. For instance, the low similarity in Glaucoma models aligns with the loss of peripheral input affecting spatial encoding in the visual cortex (V1). The high energy in Cataract models reflects the brain's compensatory mechanisms for reduced contrast sensitivity.
• Accessibility AI: The findings highlight the need for assistive AI systems that are robust to specific types of visual degradation. Current models trained on "clean" data may fail or become uncertain when inputs are degraded by specific disorders.
• Future Directions: The paper suggests that integrating neuroscientific validation (e.g., fMRI or eye-tracking) with these computational models could further bridge the gap between artificial and biological vision, leading to better diagnostic tools and adaptive interfaces for the visually impaired.

In conclusion, this work demonstrates that visual impairments do not just lower image quality; they fundamentally reshape the internal feature representations of deep learning models. The severity of this shift is disorder-dependent, with Cataract and Glaucoma posing the most significant challenges to feature alignment and recognition confidence.