🤖 AI

Enhancing Visual Perception in Foggy Conditions via Multiclass Fog Density Modeling

This paper proposes a multiclass fog density modeling approach that trains separate perception models for five distinct fog levels using synthetically generated Waymo data, demonstrating that this specialized strategy significantly improves recall in very heavy fog conditions compared to a single unified model.

Original authors: Mohamad Mofeed Chaar, Galia Weidl

Published 2026-08-04

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohamad Mofeed Chaar, Galia Weidl

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to drive a car that thinks for itself. This "autonomous" car doesn't have a human behind the wheel; instead, it has a brain made of cameras and computers that constantly scan the road to find other cars, pedestrians, and signs. This field is called autonomous driving, and its biggest challenge is "perception"—the ability to see clearly. In perfect weather, these cameras work like human eyes. But when the weather turns nasty, like when a thick fog rolls in, the car's vision gets blurry and confused, just like ours would. Fog is essentially a wall of tiny water droplets that scatter light, making distant objects fade away and look gray. If the car can't see clearly, it can't drive safely. Scientists have been trying to teach these computer brains to see through the fog, but most of them try to teach the brain one single way to handle all kinds of fog at once, from a light mist to a heavy, blinding cloud.

This paper, titled "Enhancing Visual Perception in Foggy Conditions via Multiclass Fog Density Modeling," tackles that problem by asking a simple question: What if we stop trying to teach the car one general rule for fog, and instead give it a different set of "glasses" for every specific thickness of fog? The researchers, working with data from the Waymo autonomous driving dataset, decided to simulate five different levels of fog: clear, light, moderate, heavy, and very heavy. Instead of training one giant model to handle everything, they trained five separate, specialized models. They found that when a model is trained specifically on "very heavy fog," it gets much better at spotting things in those conditions. In fact, for the thickest fog, the model's ability to find objects (called "recall") jumped from a very low 0.076 to a much better 0.232. That's a huge improvement of 15.6 percentage points. However, the paper also suggests something interesting: training a model only on the absolute worst fog might actually make it a bit too specialized and less flexible. The authors propose that a smart system might just need three specialized models—one for clear weather, one for light fog, and one for moderate fog—to handle almost everything, including the very heavy stuff, without needing a separate brain for every single shade of gray.

The Story of the Foggy Road

Think of driving in fog like trying to find your way through a giant, invisible maze. In the real world, self-driving cars use cameras to "see" the road. But when fog rolls in, it's like someone has turned down the contrast on a TV and covered the screen with a white sheet. The car's computer gets confused because the objects it needs to see—like a red stop sign or a person crossing the street—start to look like gray smudges.

For a long time, scientists tried to fix this by building one super-smart computer brain that learned to handle all types of fog at once. They thought, "If we teach the brain to see through a little bit of fog and a lot of fog all together, it will be a master of the weather!" But the authors of this paper suspected that this "one-size-fits-all" approach was like trying to wear the same pair of sunglasses for a sunny beach day and a dark cave. It just doesn't work perfectly for either.

The Experiment: Five Glasses for Five Weathers

To test their idea, the researchers needed a lot of foggy pictures. Since real-world foggy driving data is hard to find (nobody wants to drive in a blinding storm just to take photos for a dataset), they created their own. They took clear, sunny pictures from the Waymo dataset (a huge collection of real driving footage) and used a special computer trick to add fog to them.

They didn't just add "fog"; they added specific amounts of it. They used a mathematical formula (an extension of something called the Koschmieder model) to control exactly how thick the fog was. They created five distinct levels:

Clear (No fog)
Light Fog (A little mist)
Moderate Fog (You can see a few cars ahead)
Heavy Fog (Visibility is getting low)
Very Heavy Fog (It's hard to see anything)

Here is the clever part: Instead of training one big model to handle all five, they trained five separate models. One model only saw clear pictures. Another only saw light fog. Another only saw heavy fog, and so on. It's like having five different detectives, each trained to solve a specific type of mystery, rather than one detective trying to solve every crime in the world.

The Results: Specialization Wins (But with a Twist)

When they tested these models, the results were exciting. The specialized models were much better at their specific jobs.

The Big Jump: For the "Very Heavy Fog" category, the model trained specifically for that condition found 0.232 of the objects it was supposed to see. Compare that to the baseline (a model trained on clear weather) tested on heavy fog, which only found 0.076. That is an improvement of 15.6 percentage points. It's the difference between a detective finding 7 out of 100 clues versus finding 23 out of 100. That extra visibility could be the difference between a safe stop and a crash.
The "Goldilocks" Discovery: However, the researchers found something surprising. When they looked at the "Very Heavy Fog" results, the model trained specifically for "Very Heavy Fog" wasn't always the absolute best in terms of detection accuracy (mAP). Sometimes, the model trained for "Moderate Fog" performed competitively, or even slightly better, in terms of mAP.

Why? The authors suggest that when the fog is too thick, the image becomes so blurry that the computer brain gets confused during training. It's like trying to learn to juggle while wearing a blindfold; the brain might get stuck on the extreme difficulty and fail to learn the general rules of the game. This suggests that maybe we don't need a separate "Very Heavy Fog" detective after all. A "Moderate Fog" detective might be smart enough to handle the heavy stuff too.

What This Means for the Future

The paper concludes that we should stop treating fog as a simple "yes or no" problem (foggy or not foggy). Instead, we should treat it like a spectrum. By using a "modular" approach—where a car switches between a few specialized models depending on how thick the fog is—we can make self-driving cars much safer.

The authors suggest that a system with just three models (Clear, Light Fog, and Moderate Fog) might be the sweet spot. It would be robust enough to handle the worst weather without getting too complicated or expensive. While this study focused on cameras (RGB images), the authors hint that this same idea could work for other sensors like LiDAR (lasers) and radar in the future, making our roads safer for everyone, no matter how thick the fog gets.

In short, the paper suggests that to see clearly in the fog, you don't need one giant brain; you need a team of specialists, each wearing the right glasses for the job.

Technical Summary: Enhancing Visual Perception in Foggy Conditions via Multiclass Fog Density Modeling

Problem Statement
Autonomous driving (AD) systems face a critical challenge in maintaining robust perception under adverse weather conditions, particularly dense fog. Fog significantly reduces scene visibility and image contrast, degrading the performance of optical sensors. A primary barrier to addressing this issue is the lack of publicly available datasets with explicit fog annotations and ground-truth depth information. Existing datasets, such as Waymo, do not provide per-pixel dense depth maps aligned with camera images, and few offer labeled fog data. Consequently, training and evaluating perception models under varying fog conditions remains difficult, hindering systematic benchmarking and the development of reliable AD systems for real-world deployment.

Methodology
To address these gaps, the authors propose a pipeline for synthesizing realistic fog on the Waymo Open Dataset and evaluating object detection performance across five distinct fog density levels: clear, light, moderate, heavy, and very heavy fog. The methodology consists of three core components:

Depth Estimation via Iterative Learning Approach (ILA): Since the Waymo dataset lacks ground-truth depth, the authors employ an Iterative Learning Approach (ILA) to generate depth maps from RGB images. This method progressively refines depth supervision by using a model's initial predictions to fill missing or invalid depth values in the training targets. Through multiple iterations, the system produces enhanced, structurally consistent depth maps without requiring additional sensors or external annotations.
Physics-Based Fog Synthesis: The authors extend the Koschmieder atmospheric model using a matrix-based representation to synthesize fog. The intensity of the synthesized fog is controlled by an attenuation coefficient ( $\beta$ ), which is applied as a function of the distance between the camera and scene content (derived from the ILA-generated depth maps). Five specific $\beta$ values are used to simulate the five fog density levels (ranging from 0.03 for light fog to 0.12 for very heavy fog).
Density-Specific Model Training: Instead of training a single unified model for all conditions, the study trains separate object detection models (using YOLOv12m) for each specific fog density level. This approach tests the hypothesis that condition-specific training yields better robustness than a general-purpose model.

Key Contributions

Synthetic Fog Generation Pipeline: The development of a reproducible pipeline that generates depth maps for the Waymo dataset using ILA and subsequently synthesizes physically motivated fog effects using an extended Koschmieder model.
Multiclass Fog Density Modeling: The proposal and evaluation of a strategy that discretizes fog into multiple density levels rather than treating it as a binary condition.
Open-Source Implementation: The provision of an open-source implementation of the fog synthesis pipeline to facilitate reproducibility.
Empirical Analysis of Density-Specific Training: A comprehensive cross-density evaluation demonstrating the performance trade-offs between models trained on specific fog levels versus those tested on mismatched conditions.

Experimental Results
The study utilized 8,100 training samples and 4,800 validation samples from the Waymo dataset. Models were evaluated using Recall and Mean Average Precision at IoU=0.50 (mAP@50).

Performance Gains: The results indicate that density-specific training significantly improves performance, particularly in severe conditions. For the "very heavy fog" class, training a model specifically on that density improved Recall from 0.076 (baseline) to 0.232, an absolute gain of 15.6 percentage points.
Cross-Density Evaluation: Models trained on light and moderate fog generally achieved the highest performance when tested on corresponding densities.
Observation on Extreme Fog: Interestingly, under "very heavy fog" conditions, a model trained on "moderate fog" achieved competitive, and in some cases slightly higher, mAP@50 scores compared to the model trained exclusively on "very heavy fog." The authors suggest that extreme visual degradation in the most severe training data may limit a model's generalization capability.
Practical Implication: The findings suggest that a modular architecture comprising three specialized models (for clear, light, and moderate fog) may offer an optimal balance between robustness and system complexity, maintaining competitive performance even under very heavy fog without requiring a dedicated model for the most extreme conditions.

Significance and Claims
The paper claims that explicitly modeling fog density is essential for reliable autonomous driving perception. The study demonstrates that deploying multiple specialized models, rather than a single general-purpose model, can enhance perception robustness under challenging visibility conditions. The authors emphasize that while density-specific training provides measurable gains, particularly for recall in severe fog, the inherent loss of visual information in extreme conditions remains a significant challenge.

The work positions itself as a step toward reducing reliance on purely simulated environments by applying these techniques to real-world driving data (Waymo), thereby mitigating the risk of overfitting to simulation artifacts. Future work is identified as extending this strategy to additional sensing modalities, such as LiDAR and radar, and evaluating generalization across diverse weather scenarios.

The Story of the Foggy Road

The Experiment: Five Glasses for Five Weathers

The Results: Specialization Wins (But with a Twist)

What This Means for the Future

Technical Summary: Enhancing Visual Perception in Foggy Conditions via Multiclass Fog Density Modeling

More like this