💻 computer science

Phase-Aware Wavelet-Based-Scattering Encoder-Decoder for Dense Predictions

This paper proposes a Phase-Aware Scattering Encoder-Decoder that enhances dense prediction tasks like image denoising and skin lesion segmentation by explicitly preserving phase information in skip connections, thereby restoring spatial structure lost in traditional scattering transforms and significantly improving performance metrics.

Original authors: Ghassen Marrakchi, Basarab Matei

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ghassen Marrakchi, Basarab Matei

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, mathematically perfect librarian (the Scattering Transform) whose job is to organize a massive library of images. This librarian is famous for two superpowers:

Stability: If you slightly move a book on a shelf, the librarian doesn't panic; they know it's the same book.
Invariance: If you move a book from the left side of the room to the right, the librarian treats it exactly the same as if it were still on the left. They ignore where things are, focusing only on what they are.

For a long time, this was great for classification (e.g., "Is this a cat or a dog?"). But for dense prediction tasks like denoising (cleaning up a blurry photo) or segmentation (drawing a line around a tumor), this "ignoring location" superpower is actually a curse. You can't clean a photo or draw a line if you don't know exactly where the pixels are.

The authors of this paper asked: "Can we keep the librarian's stability superpower but fire their 'ignore location' superpower?"

Here is how they did it, using simple analogies:

1. The Problem: The "Blurry Map"

Standard scattering transforms take a high-resolution image and shrink it down, averaging everything out. It's like taking a detailed city map and squishing it until all the streets merge into one big blob. You know the city exists, but you can't find the specific street corner anymore. This is great for saying "This is New York," but terrible for saying "Fix the pothole on 5th Avenue."

2. The Solution: The "Phase-Aware" Upgrade

The authors built a new system called the Phase-Aware Wavelet-Based-Scattering Encoder-Decoder. Think of it as a three-part assembly line:

The Encoder (The Smart Scanner): Instead of squishing the image into a blob, they changed the scanner to keep every single pixel in its original spot (called Stride-1 Equivariance). They stopped the "global averaging" that caused the blur.
- The Result: They kept the mathematically perfect stability (the librarian still knows a moved book is the same book) but stopped ignoring the location. This alone added a huge boost to image quality (+2.17 dB).
The Secret Sauce: Phase Preservation: When the scanner reads an image, it gets two types of data:
- Magnitude (The Brightness): How strong the signal is.
- Phase (The Location): The exact timing and position of the edges and corners.
- The Analogy: Imagine a choir. Magnitude is how loud the singers are. Phase is the exact rhythm and timing of their voices. If you only know how loud they are, you hear a mess. If you know the timing (phase), you hear a beautiful song.
- The authors realized that previous systems threw away the "timing" (phase) data. They built a special "skip connection" (a direct pipeline) to carry this phase information from the scanner all the way to the output, ensuring the decoder knows exactly where the edges are. This added another significant boost (+1.03 dB).
The Decoder (The Artist): This part takes the stable, location-aware data and tries to reconstruct the clean image. They added a "gating" mechanism (like a dimmer switch) to help the artist decide where to focus, though they found the artist mostly just needed the phase data itself to do a good job.

3. The Results: A Trade-Off

The paper tested this on cleaning up noisy images (denoising).

The Good News: Their new method works much better than the old "blurry map" scattering methods. It recovered sharp edges and details that were previously lost.
The Catch: It still didn't beat the "black box" deep learning models (like DnCNN) that learn everything from scratch without math rules. The black box models got slightly higher scores.
The Why: The authors admit this is the "price of admission." They sacrificed a tiny bit of raw performance to keep the system mathematically interpretable. You can look at their system and understand why it works (because of the wavelets and phase), whereas the black box models are mysterious.

4. What They Learned (The "Aha!" Moments)

Phase is King: They did a weird experiment where they shuffled the "phase" data around randomly. The image quality crashed. When they shuffled the "magnitude" (brightness) data, the image barely changed. This proved that location (phase) is 5 times more important than brightness for reconstructing a clear image.
Data Hungry: This new system needs a lot of training data to work well. If you only give it a few pictures, it struggles. The black box models are better at learning from very few examples.
Medical Imaging Warning: Because this system needs lots of data, the authors warn it might not be the best choice for medical imaging right now, where doctors often have very few labeled patient scans to train on.

Summary

The paper is about taking a mathematically rigid, stable system and teaching it to care about location again. They did this by keeping the "timing" (phase) information of the image alive throughout the process. It's not the absolute best performer in the race, but it's the most honest and understandable runner, proving that you can have mathematical stability without losing the ability to see the fine details.

Technical Summary: Phase-Aware Wavelet-Based-Scattering Encoder-Decoder for Dense Predictions

1. Problem Statement

Scattering transforms, introduced by Mallat (2012) and Bruna & Mallat (2013), provide mathematically grounded signal representations characterized by Lipschitz stability and translation invariance. These properties are highly valuable for classification tasks. However, they present a fundamental conflict with dense prediction tasks (e.g., image denoising, segmentation), which require pixel-level spatial correspondence.

The core mechanism of standard scattering—global averaging and downsampling by a factor of $2^J$ —fundamentally discards the fine spatial information necessary for reconstruction. While the modulus operations ensure stability regardless of resolution, the subsampling required for translation invariance destroys spatial equivariance. The authors posit that for dense prediction, it is possible to decouple these properties: preserving the mathematical stability of the scattering transform while breaking global invariance to recover spatial structure.

2. Methodology

The authors propose a Phase-Aware Scattering Encoder-Decoder architecture. This design replaces learned convolutional encoders with fixed, mathematically grounded wavelet filters while introducing specific architectural innovations to enable dense prediction.

2.1. Encoder: Stride-1 Equivariance and Phase Preservation

The encoder modifies the standard scattering transform in two critical ways:

Stride-1 Equivariance: The authors remove the global averaging and subsampling ( $2^J$ ) typically used to achieve translation invariance. Instead, they employ stride-1 processing with scale-dependent smoothing windows ( $\phi_j \propto 2^j$ ). This preserves full spatial resolution throughout the hierarchy, ensuring the bottleneck receives fully resolved spatial structures. This sacrifices global translation invariance but retains Lipschitz stability, which is derived from the cascaded modulus operations and remains valid at any resolution.
Phase Preservation: Standard scattering discards the phase of complex wavelet coefficients, retaining only the modulus (magnitude). The authors argue that magnitude alone cannot capture structural details like edges and singularities. The encoder explicitly extracts and preserves both magnitude ( $A$ ) and phase ( $\Phi$ ) from the pre-modulus complex coefficients. These are stored separately and routed to the decoder via skip connections.

2.2. Bottleneck

A single $3 \times 3$ convolutional layer acts as a bridge, compressing the high-dimensional scattering coefficients (which can exceed 1000 channels) into a compact latent representation ( $C_{bn}$ ) while maintaining full spatial resolution. This layer is the only learnable component in the encoder path.

2.3. Decoder: Phase-Aware Gating and Fusion

The decoder follows a U-Net-like structure with $J$ upsampling levels. It integrates three information sources at each level:

Bottleneck Features: Up-sampled latent representations.
Magnitude Skip Connections: Texture and energy information.
Phase Skip Connections: Geometric structure and spatial localization.

The decoder employs a Phase-Informed Gating Mechanism. A learned gating network processes concatenated magnitude and phase to generate a spatial mask. This mask modulates the upsampled bottleneck features, theoretically allowing the network to focus on regions where phase encodes significant structure (edges, singularities). Finally, a fusion network combines the gated bottleneck features with the magnitude and phase skip connections to produce the final pixel-level prediction.

3. Key Contributions

The paper advances the integration of wavelet theory and deep learning through four specific contributions:

Decoupling Stability and Invariance: Extending scattering from classification to dense prediction by replacing global invariance with stride-1 equivariance, thereby recovering spatial structure.
Phase-Aware Architecture: Integrating explicit phase preservation into a mathematically grounded scattering architecture, demonstrating that complex-valued skip connections are essential for reconstruction.
Polar Representation: Utilizing polar form (magnitude and phase) for skip connections to decouple gradients, isolating phase as an independent learned signal.
Interpretability: Providing an interpretable alternative to black-box learned methods (like DnCNN or SwinIR) by using fixed, theoretically defined wavelet filters, offering robustness guarantees for safety-critical applications.

4. Experimental Results

The method, referred to as M6, was evaluated on the BSD68 image denoising benchmark (Gaussian noise $\sigma=25$ ) and a preliminary study on ISIC 2018 skin lesion segmentation.

4.1. Denoising Performance (BSD68)

Baseline Comparison: M6 achieves 27.41 dB PSNR. This underperforms fully learned baselines: DnCNN (29.27 dB), BM3D (28.57 dB), and DSWN (29.61 dB).
The Trade-off: The authors explicitly frame the $\sim$ 2 dB performance gap as the "principled cost" of using a fixed, mathematically interpretable encoder with Lipschitz stability guarantees, rather than a black-box learned encoder.
Ablation Studies:
- Stride-1 Equivariance: Transitioning from standard invariant scattering (24.19 dB) to stride-1 equivariant scattering (26.36 dB) yields a +2.17 dB gain, validating that spatial equivariance is critical for dense prediction.
- Phase Preservation: Adding complex-valued skip connections (magnitude + phase) to the stride-1 model yields an additional +1.03 dB (27.39 dB).
- Spatial Shuffling: Randomizing the spatial indices of phase coefficients causes a −1.26 dB penalty, whereas shuffling amplitude causes only a −0.25 dB penalty. This confirms that phase encodes location-dependent structural information essential for reconstruction (5.14× more important than amplitude).
- Polar Form: Using polar representation over raw complex values adds a marginal +0.02 dB, primarily aiding optimization stability.

4.2. Robustness and Stability

Geometric Perturbations: M6 demonstrates superior robustness to geometric transformations compared to learned methods. Under 10° rotation, M6 degrades by 10.4 dB, whereas DnCNN degrades by 16.1 dB. M6 is 1.55× more robust, empirically validating the Lipschitz stability of the scattering encoder.
Data Efficiency: M6 is less robust to data scarcity than DnCNN. At 10% of the training data, M6 performance drops by 5.29% compared to 1.71% for DnCNN. The authors note this limits applicability in domains with very few annotated samples (e.g., medical imaging with $N < 50$ ).

4.3. Segmentation (Preliminary)

A preliminary study on ISIC 2018 skin lesion segmentation (training set only, no validation) showed consistent Dice coefficients of 0.886–0.887. The study found that explicit phase regularization provided no benefit, suggesting that implicit phase learning via the task loss is sufficient.

4.4. Negative Findings

The paper reports "honest null findings" regarding specific design choices:

Gating Mechanism: The learned gating networks converged to spatially constant scaling masks rather than meaningful spatial attention, contributing negligibly (+0.02 dB) to performance.
Phase Regularization: Explicit constraints (Phase Total Variation, Phase Alignment) harmed performance in both denoising and segmentation, suggesting that hand-tuned mathematical constraints can interfere with task-specific learning.

5. Significance and Claims

The paper claims to advance the principled integration of wavelet-based deep learning by demonstrating that Lipschitz stability and spatial invariance can be decoupled. The significance lies in:

Theoretical Rigor: Showing that stability (robustness to deformations) can be maintained even when invariance is broken to serve dense prediction.
Phase Importance: Providing empirical evidence that phase information is the primary carrier of location-dependent structural content in wavelet representations, far outweighing magnitude in reconstruction tasks.
Interpretability vs. Performance: Establishing a clear trade-off where the proposed method sacrifices peak reconstruction accuracy (approx. 2 dB) to gain mathematical interpretability, fixed multi-scale decomposition, and provable stability guarantees.

The authors conclude that while learned wavelet methods (like DSWN) offer superior performance, the proposed phase-aware scattering encoder-decoder offers a viable, interpretable alternative for applications where mathematical transparency and robustness are prioritized over absolute peak performance, provided sufficient training data is available.