⚡ electrical engineering

Learnable Blur Kernel for Single-Image Defocus Deblurring in the Wild

This paper proposes a novel unsupervised defocus deblurring framework that combines a learnable blur kernel for defocus map estimation with a Generative Adversarial Network (DefocusGAN) guided by a new defocus adversarial loss, achieving state-of-the-art performance in both map accuracy and perceptual image quality without requiring dual-pixel sensors.

Original authors: Jucai Zhai, Pengcheng Zeng, Chihao Ma, Yong Zhao, Jie Chen

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jucai Zhai, Pengcheng Zeng, Chihao Ma, Yong Zhao, Jie Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to look through a camera lens, but the lens is slightly out of focus. Some parts of the picture are sharp, but others are blurry because they are too close or too far away. This is called defocus blur.

For a long time, fixing this problem was like trying to un-mix a smoothie back into a strawberry and a banana. Most computer programs that tried to fix it either made the picture look too smooth (losing all the tiny details like skin texture or text on a magazine) or they needed special, expensive camera hardware to work at all.

This paper introduces a new, clever way to fix blurry photos using just a single picture, without needing special cameras. Here is how they did it, broken down into simple steps:

1. The "Map" Problem: Finding the Blur

To fix a blurry photo, a computer needs to know where the blur is and how bad it is.

The Old Way: Some methods used special "Dual-Pixel" cameras (like having two eyes) to figure out the blur. But getting these special camera views is hard and slow to use in real life. Other methods tried to guess the blur map but often got it wrong or needed a "teacher" (ground truth data) to learn from.
The New Trick (Learnable Blur Kernel): The authors created a "smart guesser." Imagine a student who doesn't have the answer key but is allowed to practice drawing the map over and over. They built a system that learns to create a Defocus Map (a guide showing which parts are blurry) by simulating the blur process itself. It's like a chef learning to recreate a dish by tasting it and adjusting the spices, rather than just following a recipe. This method is "unsupervised," meaning it teaches itself without needing perfect examples to copy.

2. The "Restoration" Problem: Fixing the Photo

Once they have the map, they need to actually fix the image.

The Old Way: Standard computer programs often treat the whole photo the same. They might fix the blurry parts but accidentally make the sharp parts look weird, or they might leave the blurry parts looking like plastic (smooth but fake).
The New Trick (DefocusGAN): The authors built a special AI team called DefocusGAN. Think of this team as having two roles:
- The Generator (The Artist): This AI tries to paint the sharp details back into the blurry spots.
- The Discriminator (The Critic): This AI looks at the result and says, "That doesn't look real; the texture is wrong."
- The Secret Sauce (Defocus Adversarial Loss): Usually, the Critic judges the whole picture equally. But the authors told the Critic to pay extra attention to the blurry areas. It's like a teacher grading a test but giving extra points for getting the hardest questions right. This forces the Artist to focus its energy on fixing the messy parts while keeping the clear parts clear.

3. The "Multi-Scale" Strategy: Zooming In and Out

Blurry spots come in different sizes. A tiny blur needs a small fix; a huge blur needs a big fix.

The Solution: The network looks at the photo at different "zoom levels" (scales). It has different "branches" for different sizes of blur.
- For small blurs, it looks closely at just a few pixels.
- For huge blurs, it steps back to look at a wider area to understand the context.
- This is like a detective solving a crime: for a small clue, they look through a magnifying glass; for a big pattern, they look at the whole crime scene map.

4. The Result: A "Magic" Single-Image Fix

The coolest part of this method is that while the AI uses the map to learn how to fix things, it doesn't need the map when it's actually working on a new photo.

Training: The AI learns with the help of the map (like training wheels).
Inference (Real Use): Once it's trained, you can take the training wheels off. You just feed it a single blurry photo, and it knows exactly how to fix it on its own.

Why This Matters (According to the Paper)

Better Details: Unlike previous methods that made photos look smooth and plastic-like, this method brings back high-frequency details (like the texture of a brick wall or the letters in a book).
No Special Hardware Needed: You don't need a Dual-Pixel camera to use this; it works on regular photos.
Efficiency: It does all this with a relatively small number of computer "parameters" (it's not a bloated, slow system).
Performance: In their tests, their method produced sharper, more realistic images than other top methods, scoring higher on standard quality measurements.

In short: They taught a computer to draw its own map of the blur, then used a "tough critic" AI to force a "painter" AI to fix only the blurry parts while keeping the sharp parts sharp, resulting in a photo that looks like it was taken with a perfect focus.

Technical Summary: Learnable Blur Kernel for Single-Image Defocus Deblurring in the Wild

Problem Statement
Defocus blur occurs when scene points lie outside the lens's depth of field, resulting in a Circle of Confusion (CoC) on the image plane. While recent advances in dual-pixel (DP) sensors have improved defocus map estimation and deblurring, extracting real-time DP views remains complex and troublesome for algorithm deployment. Existing single-image defocus deblurring methods often fail to provide reliable defocus clues, leading to poor handling of large-scale blur and a lack of high-frequency (HF) details, which results in unsatisfactory perceptual quality. Furthermore, traditional two-stage methods (estimating a map then deconvolving) often rely on hand-crafted features, ground truth, or calibrated blur kernels, making them difficult to apply in the wild.

Methodology
The authors propose a novel framework comprising two main components: an unsupervised defocus map estimation module using a learnable blur kernel, and a defocus map-guided Generative Adversarial Network (DefocusGAN) for single-image deblurring.

Learnable Blur Kernel (BK) for Defocus Map Estimation:
- Unsupervised Approach: Since ground truth defocus maps are unavailable, the authors propose an unsupervised method to estimate the map using DP views as input.
- Blur Reconstruction: The method learns a network $f$ to estimate the defocus map ( $I_{DM}$ ) from DP views. This map guides a reconstruction network $g$ to blur an all-in-focus (AiF) image, producing a synthetic blurred image ( $I^*_{OoF}$ ).
- Learnable Kernel: To avoid the difficulty of calibrating real blur kernels, the authors introduce a learnable BK. The kernel is initialized using a Butterworth high-pass filter (mimicking the camera's defocus characteristics) convolved with a Gaussian kernel. The parameters are optimized alternately with the defocus map estimation network to minimize the geometric loss ( $L_{gem}$ ) between the reconstructed blur and the actual DP blurred image.
- Regularization: A prior regularization term is added to the loss function to penalize gradients, ensuring the estimated defocus map is smooth.
DefocusGAN (Defocus Map Guided Deblurring):
- Architecture: The generator ( $G$ ) is a multi-scale network guided by the estimated defocus map. It employs a Defocus Map Guide Block (DGB) that splits the processing into multiple branches based on the CoC radius.
- Multi-Scale Processing: The network processes features at different resolutions (1x, 1/4x, 1/8x). Branches handling larger CoC radii utilize a U-Net-like structure with Residual Channel Attention Modules (RCAB) to aggregate features over a larger receptive field, while smaller radii branches use fully convolutional layers.
- Simulated Annealing: During training, the guidance of the defocus map is gradually reduced using a simulated annealing strategy. This allows the network to learn the deblurring priors independently, enabling inference using only a single image without the defocus map.
- Defocus Adversarial Loss: A specialized loss function is introduced where the discriminator ( $D$ ) applies weights to different regions based on the defocus map. Areas with larger CoC radii (more blurred) are assigned higher weights, forcing the generator to focus on restoring details in heavily blurred regions.
- Total Loss: The generator is trained using a combination of L1 content loss ( $L_c$ ), perceptual loss ( $L_p$ ) based on VGG19 features, and the proposed defocus adversarial loss ( $L_{defocusadv}$ ).

Key Contributions

Learnable Blur Kernel: The paper introduces a self-supervised method to estimate defocus maps using a learnable blur kernel, eliminating the need for kernel calibration and achieving performance comparable to supervised methods.
DefocusGAN: The authors propose the first defocus map-guided multi-scale GAN for single-image defocus deblurring. This architecture effectively maintains information in clear areas while recovering textures and details in blurred regions.
Defocus Adversarial Loss: A novel loss function that directs the adversarial training to focus specifically on defocused regions, improving the restoration of HF details.
Efficiency and Performance: The method achieves state-of-the-art results with a relatively small number of parameters (4.59M), outperforming existing methods in both quantitative metrics and perceptual quality.

Experimental Results
The method was evaluated on the DPDD dataset (500 image sets) and tested for generalization on the CUHK and Google PixelDP datasets.

Quantitative Performance: On the DPDD test set, the proposed method achieved a PSNR of 25.56 dB and an LPIPS of 0.111. This outperforms previous state-of-the-art single-image methods like MDPNet (PSNR 25.35, LPIPS 0.225) and IFAN (PSNR 25.18, LPIPS 0.156).
Comparison with DP-based Methods: While methods using raw DP views as input (e.g., DPDNet) achieve high PSNR, the proposed single-image method achieves competitive results (25.56 dB vs. 25.13 dB for DPDNet) while being more deployable as it does not require access to dual-pixel views during inference.
Qualitative Improvements: Visual comparisons show significant improvements in recovering large-scale blur, object contours, and fine textures (e.g., text in magazines, soldier's glasses) compared to methods that produce artifacts or lack HF details.
Ablation Studies: Experiments confirmed the necessity of each component: the defocus map guide (DG), multi-scale (MS) modules, perceptual loss, and the specific defocus adversarial loss all contributed to the final performance. The learnable BK was shown to outperform fixed Gaussian or Butterworth kernels.

Significance and Claims
The paper claims that the proposed method effectively addresses the challenges of single-image defocus deblurring in the wild by leveraging unsupervised defocus map estimation and a specialized GAN architecture. The significance lies in:

Deployability: By removing the reliance on real-time DP view extraction during inference, the method simplifies algorithm deployment.
Perceptual Quality: The integration of GANs and defocus-specific losses successfully recovers high-frequency details and realistic textures that are often lost in L1/L2 loss-based methods.
Robustness: The method handles large-area defocus blur effectively, a known weakness in previous single-image approaches.
Efficiency: It achieves state-of-the-art performance with a compact model size, making it suitable for practical applications.

The authors conclude that their work provides a robust framework for defocus deblurring that balances performance with practical deployment constraints, while noting that future work could explore new frameworks involving mask learning and diffusion models.