⚡ electrical engineering

Beware of Aliases -- Signal Preservation is Crucial for Robust Image Restoration

The paper introduces BOA-Restormer, a transformer-based image restoration model that utilizes frequency-domain downsampling and upsampling to create alias-free paths, thereby significantly improving model robustness with minimal impact on restoration performance.

Original authors: Shashank Agnihotri, Julia Grabinski, Janis Keuper, Margret Keuper

Published 2026-06-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shashank Agnihotri, Julia Grabinski, Janis Keuper, Margret Keuper

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Broken Photocopier" Problem

Imagine you have a high-resolution photograph of a city skyline. You want to shrink this photo down to a tiny thumbnail (downsampling) and then blow it back up to full size (upsampling) so a computer can understand it and fix any blur or rain on the lens.

The problem the authors found is that standard computer vision models act like a broken photocopier. When they shrink the image, they accidentally throw away important details (like the sharp edges of buildings) and introduce weird, repeating patterns called "aliases" (think of a moiré pattern you see when filming a TV screen).

Usually, these models are so good at their job that they can "fake" the missing details when looking at normal, clean photos. It's like a magician who can make a rabbit appear from an empty hat because the audience isn't looking closely. But, if you poke the magician with a stick (an adversarial attack—a tiny, almost invisible change to the image designed to confuse the AI), the trick fails. The model reveals that it didn't actually learn the shape of the rabbit; it just learned a shortcut. The result is a restored image full of weird, grid-like noise or "spectral artifacts."

The Solution: The "Smart Filter" (BoA-Networks)

The authors propose a new way to handle shrinking and growing images, which they call Beware of Aliases (BoA). Instead of just blindly shrinking the image, they use a "frequency domain" approach.

Think of an image not just as pixels, but as a symphony of sounds:

Low Frequencies: The deep bass notes. These are the big shapes, the general colors, and the smooth areas of the image.
High Frequencies: The sharp cymbal crashes. These are the fine details, the sharp edges, and the textures.

Standard methods often try to silence the cymbals (high frequencies) to avoid noise, but in image restoration (like removing blur), you need those cymbals to make the image sharp again. If you silence them, the image stays blurry.

How BoA works:

The Down-Step (Shrinking): Instead of just throwing away the high notes, the model splits the music. It keeps the bass (low frequencies) safe and also keeps a separate track of the cymbals (high frequencies). It mixes them together carefully so nothing is lost, but the "noise" is filtered out.
The Up-Step (Growing): When the model blows the image back up, it doesn't just guess. It uses a special method to bring back the cymbals it saved earlier, ensuring the sharp edges return exactly where they belong.

The authors call their two new tools FrequencyPreservedPooling (for shrinking) and FreqAvgUp (for growing). Together, they create a "safe path" for the image data to travel through the computer, ensuring no important details get lost in the shuffle.

Why This Matters: The "Stress Test"

The paper argues that we shouldn't just judge these models by how they look on normal photos. We need to stress-test them.

The Analogy: Imagine two bridges.
- Bridge A (Standard Model): Looks perfect when no cars are on it. But if a strong wind blows (an adversarial attack), it starts shaking and reveals cracks.
- Bridge B (BoA Model): Also looks good when empty. But when the wind blows, it stands firm because it was built with better engineering (preserving the signal).

The authors tested their new BoA models on tasks like removing rain from photos and fixing blurry photos.

On clean photos: The new models performed just as well as the best existing models.
On "attacked" photos: When they added tiny, invisible distortions to trick the AI, the old models fell apart, producing images with strange grid patterns and artifacts. The BoA models, however, remained stable and produced clean, sharp images.

The Takeaway

The paper claims that by respecting the fundamental rules of signal processing (specifically, not losing high-frequency details when shrinking images), we can build AI that is much more robust.

They aren't just making the images look prettier; they are making the AI's "brain" more reliable. By ensuring the AI actually learns the true structure of the image rather than taking shortcuts, the model becomes harder to fool and produces fewer weird visual glitches, even when things go wrong.

In short: They fixed the photocopier so it doesn't lose the sharp edges when shrinking the picture, ensuring that when it blows the picture back up, the result is real and reliable, not a fake shortcut.

Technical Summary: Beware of Aliases – Signal Preservation is Crucial for Robust Image Restoration

Problem Statement
Deep neural networks (DNNs) used for pixel-wise prediction tasks, such as image restoration, often overlook fundamental signal processing principles, specifically regarding sampling and aliasing. While anti-aliasing methods have improved robustness in image classification, their application to image restoration is complex. Standard downsampling techniques (e.g., strided convolutions, Pixel Unshuffle) often discard high-frequency information or introduce spectral artifacts (e.g., grid, checkerboard, or ringing artifacts) due to sampling below the Nyquist rate.

In image restoration tasks like deblurring, high-frequency information (edges, shapes) is essential for reconstruction. Conversely, tasks like denoising require removing high-frequency noise. Existing approaches that strictly apply low-pass filtering to prevent aliasing often degrade restoration performance by removing necessary signal details. Furthermore, while these models may perform well on clean, in-domain data, they frequently learn "shortcuts" that result in hidden spectral artifacts. These artifacts often remain invisible to the human eye under normal conditions but become apparent and degrade performance significantly under adversarial attacks (e.g., PGD), revealing the model's reliance on sub-optimal feature representations.

Methodology
The authors propose Beware of Aliases (BoA)-Networks, a modification to state-of-the-art (SotA) image restoration architectures (both CNN-based like NAFNet and Transformer-based like Restormer). The core innovation lies in replacing standard downsampling and upsampling operations with frequency-domain-aware methods that ensure alias-free paths while preserving relevant signal information.

The proposed architecture consists of two primary modules:

FrequencyPreservedPooling (FP) for Downsampling:
- Instead of simple low-pass filtering (which removes high frequencies) or standard subsampling (which causes aliasing), FP operates in the frequency domain.
- It applies a Discrete Fourier Transform (DFT) to the input feature maps.
- The frequency spectrum is split into low-frequency and high-frequency components.
- The low-frequency component is inverse transformed and upsampled in channel dimension (mimicking Pixel Unshuffle).
- The high-frequency component is isolated (via subtraction of the low-pass filtered version) and downsampled using standard Pixel Unshuffle, which is noted as perfectly invertible for high frequencies.
- These two paths are combined using a learnable parameter $\alpha$ (initialized to 0.3, biasing toward low frequencies), allowing the network to balance stability and detail preservation.
FreqAvgUp for Upsampling:
- This module mirrors the downsampling logic to ensure symmetry between the encoder and decoder.
- It separates the input feature maps into low and high-frequency components in the frequency domain.
- The low-frequency path involves averaging sets of four channels (to reverse the concatenation in downsampling) and zero-padding in the frequency domain before inverse transformation.
- The high-frequency path separates the high-frequency components, applies a Pixel Shuffle operation, and inverse transforms.
- A learnable parameter $\beta$ (initialized to 0.3) mixes these two paths to reconstruct the feature map.

The authors also introduce a symmetry mechanism where, for alternating layers, the spatial dimensions are transposed before FFT operations to ensure isotropic frequency handling.

Key Contributions

Novel Sampling Methods: The introduction of FrequencyPreservedPooling and FreqAvgUp, which provide alias-free paths for both downsampling and upsampling without discarding critical high-frequency signal details required for restoration.
Architectural Integration: The successful modification of both CNN-based (NAFNet) and Vision Transformer-based (Restormer) SotA models with these BoA modifications.
Robustness via Signal Preservation: Demonstrating that providing alias-free paths improves model robustness against adversarial attacks (specifically PGD and CosPGD) with minimal cost to clean data performance.
Task-Specific Analysis: A detailed frequency-domain analysis showing that image deblurring requires high-frequency preservation, while image deraining occupies a middle ground, necessitating a balanced approach that BoA provides.

Experimental Results
The authors evaluated their method on image deblurring (GoPro dataset) and image deraining (Rain100H/L, Test100/1200 datasets).

Adversarial Robustness: Under PGD and CosPGD attacks, standard models (Restormer, NAFNet) and models with simple anti-aliasing (FLC Pooling) exhibited severe spectral artifacts and significant drops in PSNR/SSIM. In contrast, BoA-modified models maintained higher PSNR and SSIM scores and produced visually cleaner images with significantly reduced artifacts (e.g., checkerboard patterns).
Clean Data Performance: There was a slight trade-off in performance on clean, non-adversarial data compared to the baseline Restormer, but the BoA models outperformed models using strict low-pass filters (FLC Pooling) which suffered from excessive blurring.
Ablation Studies: Experiments showed that blindly dropping high frequencies (simulating strict low-pass filtering) degraded deblurring performance. The learnable mixing parameters ( $\alpha, \beta$ ) were crucial for balancing the retention of high-frequency details with the stability of low-frequency features.
Visual Evidence: Qualitative results (Fig. 1, Fig. 7) demonstrated that while other methods produced restored images with visible grid artifacts under attack, BoA-Restormer and BoA-NAFNet produced artifact-free reconstructions.

Significance and Claims
The paper argues that the reliability of image restoration models in real-world settings is compromised by spectral artifacts arising from inadequate sampling. The authors claim that:

Adversarial attacks serve as a diagnostic tool to reveal underlying spectral artifacts that are otherwise invisible to the human eye on clean data.
Signal preservation is critical: Simply removing high frequencies to prevent aliasing is detrimental to restoration tasks that require edge recovery.
Theoretical motivation yields robustness: By theoretically motivated sampling operations (BoA) that preserve signal integrity, models can learn more meaningful feature representations without the computational cost of adversarial training.
Limitations: The authors modestly note that while BoA reduces artifacts, performance on clean blurry images is not yet ideal, and current evaluation metrics (PSNR/SSIM) may not fully capture the visual quality improvements regarding spectral artifacts. They suggest that current metrics often overlook the human-perceivable artifacts that their method addresses.

The Big Picture: The "Broken Photocopier" Problem

The Solution: The "Smart Filter" (BoA-Networks)

Why This Matters: The "Stress Test"

The Takeaway

Technical Summary: Beware of Aliases – Signal Preservation is Crucial for Robust Image Restoration

More like this