BiDeMem: Bidirectional Degradation Memory for Explainable Image Restoration

Based on the provided paper, here is a detailed technical summary of BiDeMem: Bidirectional Degradation Memory for Explainable Image Restoration.

1. Problem Statement

Image restoration aims to recover clean images from observations degraded by noise, haze, rain, blur, or compression. While modern deep learning approaches have improved restoration quality (often measured by PSNR/SSIM), the semantic role of degradation-aware conditions (prompts, latent priors, memory states) remains poorly understood.

Current evaluation methods are weak tests of semantics because they rely solely on endpoint performance. A condition might improve PSNR by acting merely as:

• Extra model capacity.
• A global correction bias.
• A dataset-specific shortcut.
• A dense prompt vector.

The paper argues that for a degradation prior to be truly explainable, it must be verifiable. Specifically, it should be input-aligned, sensitive to "wrong" priors, dependent on specific selected memory slots, and capable of supporting a reverse-direction signal (reconstructing the degradation from the clean target).

2. Methodology: BiDeMem

The authors propose BiDeMem, a bidirectional degradation memory framework designed to make the degradation prior an auditable object. The method is built upon a controlled backbone (NAFNet) and consists of the following components:

A. Evidence-Guided Query

Instead of using a free-form image identifier, the system constructs a query ($q$) from observable degradation evidence:

1. Bottleneck Features: Spatial and frequency-band descriptors (low, mid, high-frequency FFT statistics) from the restoration backbone.
2. Input Statistics: A 22-dimensional vector containing RGB/luminance stats, gradient magnitudes, and frequency ratios.
   These are mapped to a shared 128-dimensional query vector.

B. Compact Top-k Memory

The system maintains a memory bank of $N=12$ slots (dimension $d=128$).

• Retrieval: The query retrieves a compact subset of $k=4$ slots using normalized dot products.
• Activation: Only the selected slots are active; the rest are zeroed out. This creates a sparse, top-k route rather than a dense condition.

C. Bidirectional Paths

The core innovation is the shared slot identity used in two distinct paths:

1. Restoration Path (Inference & Training): The retrieved slots generate a prior ($p_r$) that modulates the restoration decoder via FiLM (Feature-wise Linear Modulation) and conditions a slot-routed correction head. This produces the final restored image $\hat{y}$.
2. Forward-Degradation Explanation Path (Training Only): The same selected slot identity generates a degradation prior ($p_d$). A forward-degradation head takes the clean target $y$ and $p_d$ to predict the degraded observation $\hat{x}$. This branch is supervised to ensure $\hat{x}$ matches the actual input $x$ in pixel, frequency, and statistic spaces.

D. Objectives

The model is trained with a composite loss:

• Restoration Loss ($L_{pix}$): Pixel-wise difference between restored and clean images.
• Ranking Loss ($L_{rank}$): Encourages the native prior to score higher than "wrong" priors from a cross-degradation queue.
• Degradation & Explanation Losses ($L_{deg}, L_{explain}$): Ensure the forward-degradation branch accurately reconstructs the input degradation using the same slots.

3. Key Contributions

1. Verifiable Prior Formulation: The paper formulates degradation memory not just as a performance booster, but as a verifiable prior assessed by restoration utility, intervention sensitivity, and route consistency.
2. Bidirectional Memory Architecture: A compact top-k memory where the selected slot identity is shared between the restoration decoder and a training-only forward-degradation consistency branch.
3. Control Suite: A rigorous set of controls to isolate the memory mechanism from alternative explanations (e.g., extra backbone capacity, residual heads, dense prompts, or static global priors).
4. Intervention-Based Evaluation: Demonstrating that the retrieved prior is sensitive to counterfactual interventions (e.g., swapping priors, shuffling slots), framing the memory as a falsifiable explanation mechanism.

4. Experimental Results

The study uses a controlled multi-degradation setting (denoising, deraining, dehazing) based on NAFNet.

• Restoration Performance:

  • BiRank Memory (the full bidirectional model) achieves 29.7529 dB average PSNR and 0.8865 SSIM across eight seen benchmarks.
  • It outperforms the baseline NAFNet and a "Rank Memory" variant (unidirectional) by small but consistent margins.
  • Control Variants: Models using only a correction head, dense query priors, or static global priors perform 0.25–0.28 dB lower than BiRank, proving the gain comes from the specific memory mechanism, not just added capacity or heads.

• Intervention Sensitivity (The Core Claim):

  • Wrong-Prior Sensitivity: When the correct prior is replaced with a wrong one, BiRank Memory's performance drops by 1.0430 dB, compared to only 0.2365 dB for the unidirectional Rank Memory.
  • Native vs. Non-Native Gap: The performance gap between using the native prior and a non-native prior increases from 0.3484 dB (Rank) to 0.6134 dB (BiRank).
  • Active Slot Masking: Masking the active (selected) slots significantly reduces performance, while masking inactive slots has negligible effect, confirming the reliance on the specific top-k subset.

• Generalization & Efficiency:

  • Generalization: The model shows gains (+0.20 dB) on unseen/mixed degradations (JPEG, mixed rain/haze).
  • Low-Data Adaptation: With only 10% of training data, BiRank Memory improves over a scratch-trained NAFNet by +0.7954 dB.
  • Efficiency: The model is parameter-compact (18.02M params) but introduces latency (approx. 42ms vs. 26ms for baseline) due to memory and correction heads.

• External Backbones: When applied to AirNet and PromptIR, BiRank Memory consistently improves PSNR over matched controls, though the gains are setting-dependent.

5. Significance and Claims

The paper positions BiDeMem not as a universal, state-of-the-art restoration operator, but as a verification-oriented design for explainable image restoration.

• Auditable Mechanism: By forcing the same slot identity to support both restoration and a reverse degradation explanation, the model makes the "degradation prior" an auditable object. The system can be tested by masking, shuffling, or replacing slots to see if the output degrades predictably.
• Beyond Endpoint Metrics: The authors argue that a useful degradation prior must be input-aligned and intervention-sensitive. The bidirectional design ensures that the prior is not just a latent variable improving PSNR, but a route that can be falsified.
• Modest Scope: The authors acknowledge limitations, noting that the design adds computational latency and that the compact global slot design may not be optimal for every backbone. The primary contribution is the protocol for making degradation memory testable, rather than claiming a new SOTA for all restoration tasks.

In summary, BiDeMem demonstrates that a compact, bidirectional memory can serve as a meaningful, verifiable degradation prior, shifting the focus from "does it restore better?" to "does the restoration depend on the correct, interpretable degradation evidence?"