💻 computer science

Response-decomposed residual filtering for joint low-light enhancement and deblurring

This paper proposes a response-decomposed residual filtering module integrated into LEDNet's dynamic-kernel interfaces to jointly enhance low-light images and remove blur, achieving significant improvements in structural similarity and perceptual quality on a test set while incurring a modest increase in parameters and latency without boosting PSNR.

Original authors: Minhui Kang, Peizhe Long, Depei Cheng, Yiyao Zhang, Jiayun Zheng, Yuxin Zhu

Published 2026-11-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Minhui Kang, Peizhe Long, Depei Cheng, Yiyao Zhang, Jiayun Zheng, Yuxin Zhu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to read a street sign in the middle of a foggy night. The darkness makes the letters hard to see, but the fog also blurs their edges, smearing the sharp lines into a soft, indistinct haze. Fixing such an image is a double challenge for computers: they must brighten the scene to reveal the content while simultaneously sharpening the edges to restore the detail. This is the problem of low-light restoration, a task that requires balancing two competing needs. If a computer simply turns up the brightness, it often amplifies the blur, making the image look washed out and muddy. If it tries to sharpen the image too aggressively, it can introduce harsh artifacts or noise. The goal is to recover both visibility and spatial detail at the same time, a process that demands a delicate, localized adjustment rather than a single, blanket fix.

In the world of digital imaging, researchers have developed various tools to tackle this, but a specific approach called LEDNet has shown promise by using a flexible, adaptive method to process images. Instead of applying the same filter to the entire picture, LEDNet predicts unique, small filters for every tiny patch of the image, allowing it to react differently to shadows, lights, and textures. However, the researchers behind this new study asked a specific question: could they add a layer of correction to this existing system to make it even better, without breaking the careful work the original system had already learned? They wanted to know if they could teach the computer to make a more precise, two-part adjustment to its own filters, improving the final picture while keeping the original engine running smoothly.

The team, led by researchers from Jiangxi Electric Vocational and Technical College and Shandong Agricultural University, proposed a method they call response-decomposed residual filtering. To understand what this means, picture the computer's filter as a tool that modifies the image. The researchers realized that this tool has two main behaviors: it can shift the overall brightness of a small area, and it can change the contrast or shape of the details within that area. In their new design, they split these two behaviors apart. They added a specific pathway that allows the computer to explicitly decide how much to shift the brightness, and a separate pathway that decides how to reshape the details. Crucially, the detail-shaping part is restricted to only four learned patterns that sum to zero, meaning they change the shape without accidentally brightening or darkening the whole patch. This separation ensures that the computer doesn't confuse a need for more light with a need for sharper edges.

To make this work, the system uses two different types of information to guide these decisions. For the brightness shift, it looks only at the internal features the computer has already extracted from the dark image. For the shape adjustments, it looks at those same features but also brings in the original, degraded red, green, and blue colors of the image itself. This allows the system to use the raw color information to fine-tune the details, while keeping the brightness adjustment based on the deeper understanding of the scene. The researchers built this new pathway to start with zero effect, so that when they began training the system, it behaved exactly like the original, proven version. As the training progressed, the new pathways learned how to make small, helpful corrections.

The results of this experiment were measured against a strict set of 1,800 test images taken from 30 different videos. When the new system, which the authors call LEDNet-P3, was compared to the original LEDNet, it showed a clear improvement in how similar the restored images were to the original sharp targets in terms of structure and human perception. Specifically, the structural similarity score increased, and the perceptual difference score decreased, indicating that the images looked more natural and retained more of the original scene's layout. These improvements were consistent across the vast majority of the test videos. However, the researchers were careful to note that this did not translate to a measurable improvement in the standard peak signal-to-noise ratio, a common metric for image quality that focuses on pixel-by-pixel accuracy. In fact, the difference in that specific score was so small it was effectively negligible, suggesting that while the images looked better to the eye, the raw pixel data was not significantly more accurate.

There was a cost to this improvement. The new system required more computer memory and took about 17.7 percent longer to process each image than the original version. The researchers also compared their work to another public system known as DarkIR-m, which uses a different training method and architecture. That system still outperformed their new model on all major metrics, indicating that while their specific addition helped the LEDNet system, it did not make it the best possible solution available. The study also tested whether the specific way they split the brightness and shape tasks was necessary. They found that removing the brightness-shifting part or changing the way the system used the color information led to worse results, confirming that their specific design choices were indeed helpful.

Ultimately, this work demonstrates a successful way to upgrade an existing image restoration system by adding a specialized, decomposed correction layer. The researchers showed that by separating the decision to change brightness from the decision to change shape, and by using the raw image colors to guide the shape adjustments, they could make the images look structurally superior. While the system is not a magic fix that solves every problem or outperforms all other methods, it provides a clear, measurable benefit within its own framework. It proves that even in a highly optimized system, there is still room for a more thoughtful, decomposed approach to refining how a computer sees and corrects the world.

Technical Summary: Response-Decomposed Residual Filtering for Joint Low-Light Enhancement and Deblurring

Problem Statement
Joint low-light enhancement and deblurring presents a dual challenge: increasing image intensity to reveal scene content while simultaneously reconstructing displaced edges and recovering spatial detail. Standard enhancement often fails to address motion blur, while deblurring alone cannot recover visibility in dark regions. Existing systems, such as LEDNet, utilize native dynamic filtering interfaces to address this joint task. However, the specific challenge addressed in this work is how to introduce a supplementary correction mechanism that improves restoration quality without disrupting the original generator's learned aggregation capabilities or requiring a complete architectural overhaul.

Methodology
The authors propose LEDNet-P3, a model that integrates a structured residual correction directly into LEDNet's native dynamic-kernel interfaces. The core innovation is response-decomposed residual filtering, which organizes the added correction into specific functional components while retaining the original generator.

Architecture and Integration:
- The model retains LEDNet's original enhancement encoder, pyramid pooling modules (PPM), curve nonlinear units (CurveNLU), and deblurring decoder.
- The correction operates at three native filtering scales (1/8, 1/4, and 1/2 resolution).
- Instead of replacing the original kernel predictor ( $k^0$ ), the model adds a residual increment ( $\Delta k$ ) such that the final kernel is $k = k^0 + \Delta k$ .
Response Decomposition:
The residual increment $\Delta k$ is decomposed into two distinct components:
- Signed Kernel-Sum Coordinate: A feature-conditioned branch predicts a scalar increment ( $d$ ) per channel. This represents a constant-neighborhood response. It is calculated as $\Delta k_{sum} = \frac{d}{25}\mathbf{1}$ , where $\mathbf{1}$ is a vector of ones. This component is derived solely from encoder features.
- Restricted Zero-Sum Shape Expansion: A second branch predicts coefficients for four learned, shared 2D basis vectors ( $b_r$ ) per channel. These bases are "centered" (zero-sum) to ensure they only respond to local variations, not constant neighborhoods. The coefficients are predicted using a combination of encoder features and the degraded RGB image (projected to the feature resolution). The shape term is calculated as $v = \sum a_r \tilde{b}_r$ , where $\tilde{b}_r$ are the zero-sum bases.
Kernel Synthesis:
The final kernel increment is the sum of the constant response and the shape response: $\Delta k = \frac{d}{25}\mathbf{1} + v$ . This design ensures that for a constant neighborhood, the shape component vanishes, leaving only the sum coordinate, while local variations are handled by the shape component.
Training Strategy:
- Function-Preserving Initialization: The weights for the sum head ( $D_s$ ) and coefficient head ( $H_s$ ) are initialized to zero, and the bases are initialized with small random noise. This ensures $\Delta k = 0$ at the start of training, preserving the unmodified backbone's function.
- Joint Training: The backbone, heads, projections, and bases are trained jointly from scratch using the original LEDNet loss function (L1 + VGG perceptual loss + side-output loss). No new loss targets are introduced.

Key Contributions

Native Interface Correction: The paper introduces a method to augment dynamic filtering networks by adding a residual route directly at the kernel prediction stage, rather than as a post-processing CNN or an additional output skip connection.
Explicit Response Decomposition: The authors explicitly separate the kernel correction into a "sum" coordinate (handling constant neighborhood responses) and a "shape" component (handling local variations via zero-sum bases).
Conditional Routing: The design employs a specific conditioning strategy where the shape coefficients utilize both encoder features and the degraded RGB image, while the sum coordinate relies solely on encoder features.
Paired Evaluation Protocol: The study utilizes a strict paired evaluation protocol, training models from scratch for a fixed number of updates (60,000) on the same data splits to ensure fair comparison between the baseline LEDNet and the proposed LEDNet-P3.

Experimental Results
The model was evaluated on the LOL-Blur dataset using 1,800 official test pairs from 30 videos.

Performance vs. Paired LEDNet:
- SSIM: Increased by +0.004656 (improvements in 26/30 videos).
- LPIPS: Decreased by 0.011445 (improvements in 30/30 videos), indicating better perceptual similarity.
- PSNR: The mean difference was -0.045441 dB, meaning a PSNR improvement was not established. Improvements were mixed (favorable in 16 videos, unfavorable in 14).
Comparison with Public Systems:
- When compared to the public DarkIR-m system, LEDNet-P3 performed worse across all metrics (PSNR: -2.92 dB, SSIM: -0.095, LPIPS: +0.106). The authors note that DarkIR-m uses a different training protocol, limiting direct architectural attribution.
Ablation Studies:
- Removing the kernel-sum branch (Shape-only) resulted in lower PSNR, SSIM, and LPIPS, supporting the necessity of the sum branch.
- Replacing the uniform carrier with a center-tap carrier resulted in worse performance across all metrics.
- Using RGB conditioning for shape coefficients yielded better SSIM/LPIPS than using a feature-summary control.
Computational Cost:
- LEDNet-P3 adds 226,880 parameters (totaling ~7.64M vs. 7.41M for LEDNet).
- Forward latency increased by approximately 17.7% (352.98 ms vs. 299.88 ms per image on an RTX 5060).

Significance and Claims
The paper modestly claims that the proposed method establishes a bounded, within-LEDNet benefit. Specifically:

It demonstrates that response-decomposed residual filtering can improve structural similarity (SSIM) and learned perceptual image patch similarity (LPIPS) over the native LEDNet baseline under a fixed training protocol.
It validates the specific design choices regarding the separation of sum and shape responses and the use of RGB conditioning for shape coefficients.
The authors explicitly state that the method does not establish a general PSNR improvement and does not outperform the stronger public DarkIR-m system under different training conditions.
The work is presented as a specific contribution to the design of supplementary corrections within dynamic filtering interfaces, rather than a universal solution for low-light restoration that supersedes existing state-of-the-art systems.

Technical Summary: Response-Decomposed Residual Filtering for Joint Low-Light Enhancement and Deblurring

More like this