🤖 AI

Image Quality Dependent Degradation for AI Systems

This paper proposes a fail-degraded strategy for AI-based automated driving systems that utilizes a novel normalizing flow method to estimate image quality and dynamically lower confidence thresholds, thereby reducing critical errors in poor-quality conditions without resorting to fallback solutions.

Original authors: Yannick Kees, Elena Hoemann, Frank Köster, Sven Hallerbach

Published 2026-07-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yannick Kees, Elena Hoemann, Frank Köster, Sven Hallerbach

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are driving a car that has a brain made of math instead of meat. This "brain" is a type of artificial intelligence called a neural network, and its job is to look at the world through a camera and spot things like pedestrians, stop signs, or other cars. It's amazing at this when the weather is sunny and the lens is clean, but it has a secret weakness: it gets confused when the picture is blurry, dark, or covered in dirt. Think of it like a super-smart student who aces every test in a quiet library but starts making wild guesses the moment a siren goes off outside. The big question scientists are asking is: how do we teach this AI to know when it's seeing a "bad picture" and to change its behavior accordingly, rather than just guessing blindly or shutting down completely? This paper dives into that problem, exploring a strategy where the AI admits, "Hey, this view is messy," and decides to be extra cautious instead of trying to be perfectly precise.

The researchers behind this study, working at the German Aerospace Center, propose a clever solution they call a "fail-degraded" system. Usually, when a system gets bad data, it might try to fix the image first or switch to a different sensor, like a laser scanner. But this team suggests a different approach: let the AI keep using the camera, but change how it makes decisions based on how "dirty" the image looks. They built a special quality-checker that acts like a judge, looking at the incoming photo and comparing it to the thousands of photos the AI learned from. If the photo looks familiar and clear, the AI stays confident and sharp. But if the photo looks weird, dark, or noisy, the AI gets nervous and lowers its guard. Instead of saying, "I'm 90% sure that's a person," it might say, "I'm only 40% sure, but I'm going to flag it anyway because it's better to be safe than sorry."

To make this happen, the team used a mathematical tool called a "normalizing flow." You can think of this as a very strict librarian who has memorized the exact look of every "good" book in the library (the training data). When a new book arrives, the librarian checks if it fits the pattern. If the book looks like a muddy, torn-up mess that doesn't belong on the shelf, the librarian raises a red flag. In their experiments, this librarian successfully spotted when images were darkened, covered in digital noise, blurred, or even simulated with fake lens flares and dirt. The system didn't just guess; it measured how "unlikely" a picture was to be a clean, standard image.

Once the system knows an image is low-quality, it triggers a safety switch. Normally, an AI might ignore a detection if it's not 100% sure, to avoid false alarms. But in this fail-degraded mode, the AI flips the script. It accepts lower confidence levels to ensure it doesn't miss anything critical. The researchers tested this on a pedestrian detection system, which is crucial for automated driving. They found that when the image quality dropped, their method significantly improved the number of pedestrians the AI actually noticed (a metric called "recall"). While this did mean the AI made a few more false alarms—thinking it saw a person where there was none—it successfully avoided the much more dangerous mistake of missing a real person entirely.

The paper suggests that this approach creates a smarter balance between accuracy and safety. In clear conditions, the AI focuses on being precise. In messy conditions, it focuses on not missing anything. The authors note that if the image were completely corrupted, the system would become so cautious that it might detect a pedestrian in every single frame, effectively telling the car, "I can't see anything clearly, so I won't move." While this sounds extreme, it highlights the core idea: it is safer for an automated system to be overly cautious in bad conditions than to confidently make a mistake. The team demonstrated this with a working prototype that runs in real-time, proving that an AI can adapt its own confidence levels on the fly, making it a more trustworthy partner for the road.

Technical Summary: Image Quality Dependent Degradation for AI Systems

Problem Statement
AI perception systems, particularly in safety-critical domains like automated driving, exhibit a heavy dependency on input image quality. While deep neural networks excel at interpolating within the distribution of their training data, they struggle with extrapolation when faced with corrupted or inferior input (e.g., due to noise, darkness, or sensor wear). A critical challenge is that not all errors carry equal weight; in pedestrian detection, a false negative (missing a pedestrian) is significantly more critical than a false positive (detecting a non-existent pedestrian). Traditional fault-tolerant approaches often rely on "fail-operational" systems that switch to redundant sensors (e.g., LiDAR) or shut down entirely. However, these fallback solutions are not always available or desirable. The authors propose a "fail-degraded" approach where the system maintains reduced but controlled functionality during faults, specifically prioritizing the avoidance of critical errors (missed detections) over general accuracy when image quality is poor.

Methodology
The proposed system operates as a plug-and-play module that adjusts the confidence threshold of an existing object detector based on real-time image quality estimation. The methodology consists of three main components:

Handcrafted Feature Extraction: Instead of relying on semantic features from deep backbones (which may ignore quality-affecting augmentations), the system extracts a vector of non-reference image quality assessment (IQA) properties. These include brightness, colorfulness, contrast, energy, over/underexposure, saturation, sharpness, signal-to-noise ratio (SNR), skewness, spectral center, and PSNR.
Quality Estimation via Normalizing Flows (NF): A Normalizing Flow network ( $\eta_\theta$ ) is trained on the handcrafted feature vectors of the training dataset. The NF learns the probability density of "good" image quality. During inference, the log-likelihood ( $L(x)$ ) of an incoming image's features is computed. High likelihood indicates the image quality is similar to the training distribution, while low likelihood suggests an out-of-distribution (OOD) or degraded state.
Fail-Degraded Threshold Conversion: The system dynamically adjusts the object detector's confidence threshold ($conf(x)$) based on the calculated log-likelihood. Using a phase-field representation approach, the threshold is defined as:
$conf(x) = \frac{1}{2} \tanh(\alpha(L(x) - \tau)) + \frac{1}{2}$
Where $\tau$ $τ$ is a threshold value derived from the training data's likelihood distribution (set to the median), and $\alpha$ $α$ controls the transition width.
- High Quality (High $L(x)$ ): The threshold remains high, prioritizing precision and accurate predictions.
- Low Quality (Low $L(x)$ ): The threshold is lowered, prioritizing recall to ensure critical objects (pedestrians) are not missed, even at the cost of increased false positives.

Key Contributions

Fail-Degraded System Design: The authors develop a strategy for perception subsystems to operate with reduced functionality (lowered confidence thresholds) during faults rather than shutting down or reverting to safe states, thereby extending the operational envelope of the AI component.
Novel Quality Monitoring: A new monitoring system is introduced that uses Normalizing Flows trained on handcrafted image statistics to detect image degradation. This approach is independent of the specific object detector architecture and does not require retraining the detector itself.
Empirical Validation: The method is validated on state-of-the-art object detectors (YOLOv11 and DETR) using the BDD100k dataset, demonstrating its ability to adapt to various image perturbations.

Experimental Results
The experiments were conducted using the BDD100k dataset for training and testing, with additional validation on the Zenseact OpenDataset (ZOD) and CARLA simulated data.

OOD Detection: The Normalizing Flow successfully distinguished between the training distribution (BDD100k) and other datasets. Simulated CARLA data showed minimal overlap with real-world data, while ZOD showed a broader range but significant overlap.
Augmentation Detection: The system was tested against six types of image augmentations: darkness, noise, dirt, blur, lens flare, and rolling shutter. The NF reliably detected all augmentation types, with log-likelihood decreasing as augmentation intensity increased. Notably, the authors found that using latent features from a ResNet-18 backbone failed to detect these augmentations reliably, validating their choice of handcrafted features.
Detector Performance: When applied to a YOLOv11 pedestrian detector:
- Recall: The fail-degraded system significantly improved recall for low-quality images (those with low log-likelihood), reducing the number of missed pedestrians.
- Precision: As expected, precision decreased slightly for low-quality images due to the lower threshold, leading to more false positives.
- Overall: The system improved recall for 377 images in the test set while maintaining high recall for high-quality images. The average recall per frame increased from 0.428 (fixed threshold) to 0.474 (fail-degraded), while average precision dropped marginally from 0.996 to 0.984.
Real-Time Capability: A ROS 2 implementation demonstrated that the image quality assessment and object detection can run in parallel, confirming the system's suitability for real-time applications.

Significance and Claims
The paper claims that this approach enhances the trustworthiness of AI-based systems in automated driving by preventing critical failures (missed pedestrians) in adverse conditions without resorting to complete system shutdowns. The authors frame this as a trade-off between trustworthiness and performance: in familiar, high-quality scenarios, the system maximizes accuracy; in unfamiliar, low-quality scenarios, it prioritizes safety by ensuring critical objects are detected.

The authors modestly note that their current approach focuses on global image quality. They acknowledge that for specific tasks like pedestrian detection, local quality (e.g., the quality of the specific bounding box region) might be more relevant. Future work is proposed to adapt the approach to local bounding box analysis and potentially incorporate asymmetric loss functions during detector training. The paper concludes that while a completely corrupted image might force the system to detect pedestrians everywhere (preventing the car from driving), the proposed method ensures the system only degrades when necessary, maintaining a balance between safety and utility.

Technical Summary: Image Quality Dependent Degradation for AI Systems

More like this