HG-Lane: High-Fidelity Generation of Lane Scenes under Adverse Weather and Lighting Conditions without Re-annotation

1. Problem Statement

Lane detection is critical for autonomous driving safety, yet current state-of-the-art (SOTA) models struggle in adverse weather and lighting conditions (e.g., rain, snow, fog, night, dusk).

• Data Scarcity: Existing datasets (CULane, TuSimple) are dominated by sunny, daytime scenes. Adverse condition samples are rare.
• Annotation Costs: Manually collecting and re-annotating data for extreme weather is prohibitively expensive and time-consuming.
• Limitations of Existing Solutions:
  • Image Restoration/Translation: Methods like CycleGAN or domain translation require retraining for each weather type and add inference overhead.
  • Diffusion Models: While capable of realistic generation, existing approaches often require fine-tuning the diffusion model or re-annotating generated data to preserve lane geometry, negating the efficiency gains.

2. Methodology: HG-Lane Framework

The authors propose HG-Lane, a training-free, high-fidelity generative framework that synthesizes realistic lane scenes under adverse conditions while strictly preserving lane annotations (no re-annotation required).

Core Architecture

The framework utilizes a dual-stage, control-guided diffusion pipeline based on pretrained Stable Diffusion components (VAE, UNet, ControlNets). It does not fine-tune the backbone models.

1. Control Information Fusion (Preprocessing):

   • Input: Normal-condition lane images and their ground-truth annotations.
   • Process: A semantic-preserving strategy fuses three elements into a unified control map ($C_0$):
     • Lane Annotations ($A$): The ground truth lane masks.
     • Color Masks ($M$): Extracted via color thresholding to isolate lane regions.
     • Canny Edges ($E$): Edge maps to capture structural boundaries.
   • Fusion Formula: $C_0 = (A \odot M) \oplus E$. This ensures lane positions are "hard-coded" into the control signal while edge cues fill gaps where annotations might be sparse.

2. Stage-I: Structure-Aware Reverse Diffusion:

   • Goal: Enforce strict lane geometry.
   • Mechanism: Uses a Canny-ControlNet to guide the reverse diffusion process in latent space.
   • Input: The fused control map ($C_0$) and text prompts.
   • Output: A geometry-consistent latent representation ($z_0^{(1)}$) that maintains the exact lane structure of the input.

3. Stage-II: Appearance-Aware Reverse Diffusion (Optional):

   • Goal: Refine lighting, style, and global tone.
   • Mechanism: Uses an InstructPix2Pix (iP2P) ControlNet.
   • Conditionality:
     • Night/Dusk: Activated to adjust global illumination and sky tones while respecting the Stage-I geometry.
     • Snow/Rain/Fog: Skipped. The authors found that a single Canny-ControlNet stage already produces satisfactory realism for these stochastic weather effects without the risk of over-smoothing or losing texture details.

4. Pixel-Space Conversion:

   • The final latent is decoded back to pixel space using the pretrained VAE decoder to generate the final image.

3. Key Contributions

1. HG-Lane Framework: A novel, re-annotation-free method for generating high-fidelity lane scenes. It preserves lane semantics exactly while altering weather/lighting.
2. Dual-Stage Strategy: A specific pipeline combining Canny-ControlNet (for structural integrity) and InstructPix2Pix (for style refinement), optimized for different weather types.
3. New Benchmark: Construction of a dataset containing 30,000 images across six categories (Normal, Snow, Rain, Fog, Night, Dusk), derived from 5,000 CULane images.
4. Training-Free Efficiency: The system reuses frozen pretrained models, eliminating the need for costly diffusion model fine-tuning or data re-labeling.

4. Experimental Results

The authors evaluated the framework using the generated benchmark and real-world data.

• Performance Improvement (CLRNet Benchmark):
  • Using the generated data for training significantly boosted lane detection performance.
  • Overall mF1: Increased by 20.87%.
  • F1@50 Gains:
    • Snow: +38.8%
    • Fog: +26.84%
    • Night: +21.5%
    • Rain: +14.96%
    • Dusk: +12.04%
    • Normal: +8.63%
• Generalization: Tested on 15+ SOTA models (including SCNN, RESA, CLRNet, LaneATT, FENet). All models showed consistent improvements, proving the dataset's compatibility.
• Real-World Validation: Tested on 200 real-world samples (40 per category). The model trained on HG-Lane data outperformed models trained on original CULane data by 15.51% in overall F1@50.
• Quality Comparison:
  • Compared against Stable Diffusion, PITI, CycleGAN, and Flux.1.
  • HG-Lane achieved the highest Human Score (4.15/5) for realism and maintained high F1@50 (80.1), whereas other methods either lost lane structure (Stable Diffusion) or lacked realism (PITI).
• Ablation Studies: Confirmed that the dual-stage approach (Canny first, then iP2P) and the fusion of annotations with Canny edges are critical. Reversing the order or using single-stage controls resulted in significant performance drops.

5. Significance

• Safety Critical: Directly addresses the "long-tail" problem in autonomous driving where models fail in extreme weather, potentially preventing accidents.
• Cost Efficiency: Eliminates the need for expensive data collection and manual re-annotation, making it feasible to build robust datasets for rare conditions quickly.
• Scalability: The training-free nature allows the framework to be easily plugged into existing pipelines and scaled to generate unlimited synthetic data for diverse scenarios.
• Benchmark Standard: The release of a 30k-image benchmark provides a standardized testbed for evaluating model robustness under adverse conditions, a gap previously unaddressed in the community.

In summary, HG-Lane offers a practical, high-performance solution to the data scarcity problem in lane detection, leveraging advanced diffusion control techniques to generate semantically consistent, photorealistic training data without human annotation.