📄 radiology and imaging

FreqFuseNet: Resolving Feature-Scale Mismatch in Dual-Frequency Fusion for Thin-Wall Head-and-Neck OAR Segmentation

FreqFuseNet addresses a critical 863-fold activation-scale mismatch in dual-frequency feature fusion by normalizing the FcaNet branch to the FFT scale, thereby enabling a stable 5% residual injection that significantly improves the segmentation accuracy of thin-wall head-and-neck organs-at-risk while maintaining a lightweight architecture.

Original authors: Chen, W.-Y., Wan, S.-Y., Lin, G.-Y.

Published 2026-07-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chen, W.-Y., Wan, S.-Y., Lin, G.-Y.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to build the perfect radiotherapy plan to zap a tumor without hurting the tiny, delicate organs nearby—like the cochlea or the middle ear. These organs are as thin as a sheet of paper (about 0.5 to 2 mm thick) and are buried deep inside the skull. To map them out, doctors use AI, but the AI keeps getting the edges wrong.

The researchers behind this paper, FreqFuseNet, decided to investigate why a specific AI strategy was failing. They were trying to combine two different "superpowers" to see the boundaries of these tiny organs:

The FFT Branch: Think of this as a high-speed scanner that looks at the image in terms of waves and frequencies. It's great at spotting sharp edges.
The FcaNet Branch: This is a smart attention mechanism that focuses on specific patterns, like a detective zooming in on clues.

The Great Volume Mismatch

The team tried to mix these two superpowers together. They expected the FFT scanner to be the main voice, with the FcaNet detective adding just a tiny, helpful whisper (about 5% of the volume) to refine the edges.

But here is where things went sideways. When they turned on the computer's "fast mode" (a setting called FP16 to save memory), something strange happened. The FFT scanner's voice became incredibly quiet—so quiet it was almost a whisper. Meanwhile, the FcaNet detective was shouting at full volume.

The paper measured this mismatch and found the FcaNet branch was 863 times louder than the FFT branch.

Because of this, when the researchers tried to add the "5% whisper" from the FcaNet branch, it didn't act like a whisper at all. Because the FFT was so quiet, that "5%" ended up being 43 times louder than the main scanner! It was like trying to add a pinch of salt to a soup, but the spoon you used was actually a firehose. The result? The AI stopped listening to the FFT scanner entirely and let the shouting FcaNet branch take over, ruining the delicate edge detection they needed.

The "Naive" Mistake

The researchers tested a simple fix: just letting the computer learn how to mix the two voices on its own. They hoped the AI would figure out, "Oh, the FcaNet is too loud, I'll turn it down."

But the paper shows that this didn't work. Even after 100 rounds of training, the AI's mixing knob stayed stuck right where it started. The computer couldn't figure out how to balance the volumes because the difference was so huge. The paper argues that simply trying to learn a weight to fix this is a dead end in this specific setup.

The Solution: The Volume Knob

So, how did they fix it? They didn't try to teach the AI to guess the right volume. Instead, they added a volume knob right before the two branches met.

They built a special step called Scale-Normalized Residual Fusion. Before the FcaNet branch could shout its message, the system measured how loud the FFT branch was and turned the FcaNet volume down to match it perfectly.

Once the volumes were equal, the "5% coefficient" finally worked as intended. The FcaNet branch became a gentle, helpful whisper that refined the edges without drowning out the main scanner.

The Results

When they tested this new system, FreqFuseNet, on a benchmark of head-and-neck CT scans (specifically 180 samples of 10 different tiny organs), the results were impressive:

It achieved a Dice score of 0.849 (a measure of how well the AI's outline matches the real organ).
It reduced the HD95 error to 0.824 mm (meaning the furthest point of error was less than a millimeter away).
It did this with only 29.7 million parameters, which is about 92.8% fewer than a previous heavy-duty model that used 414.6 million parameters.

The paper suggests that this new method is statistically better than older models like 3D U-Net and MedNeXt-S. For example, it showed a statistically significant improvement over 3D U-Net with a p-value of less than 0.01.

What This Means (and What It Doesn't)

The paper suggests that the key to fixing these tiny organ outlines wasn't a new, complex brain, but simply fixing the volume mismatch between two existing tools.

However, the authors are careful to note a few things:

They tested this on a specific set of 18 cases (180 samples total). While the results are strong, they suggest that larger studies are needed for definitive proof.
They only tested on non-contrast CT scans.
They didn't actually measure the radiation dose impact on patients yet; they only measured how well the AI drew the lines.

In short, the paper suggests that if you want to map out the tiniest, thinnest parts of the body, you have to make sure your AI's different "voices" are speaking at the same volume, or the loudest one will drown out the truth.

Technical Summary: FreqFuseNet for Thin-Wall Head-and-Neck OAR Segmentation

Problem Statement
Accurate segmentation of thin-wall organs-at-risk (OARs)—specifically the cochlea, vestibular semicircular canals, internal auditory canal (IAC), tympanic cavity, and middle ear—is critical for head-and-neck radiotherapy planning. These structures, with wall thicknesses of 0.5–2 mm and volumes under 1.5 cm³, are often at the resolution limit of standard clinical CT. While state-of-the-art spatial-domain architectures (e.g., 3D U-Net, MedNeXt) perform well on volumetric context, they often struggle with the high-frequency boundary transitions required for these small structures.

The authors investigated a dual-frequency fusion approach combining a spatial encoder with frequency-domain branches (FFT and FcaNet). However, they identified a critical failure mode in the investigated FP16 (mixed-precision) training setting: a severe feature-scale mismatch. The FFT branch, initialized with small values for numerical stability, exhibited an activation standard deviation ( $\sigma_{FFT} \approx 0.0007$ ) roughly 863 times smaller than the FcaNet branch ( $\sigma_{FcaNet} \approx 0.586$ ).

In a naive residual fusion design ( $F_{fused} = F_{FFT} + \beta F_{FcaNet}$ ) with a nominal coefficient $\beta=0.05$ , this scale disparity caused the "residual" FcaNet term to dominate the fusion by a factor of approximately 43 ( $\rho_{naive} \approx 43$ ). This inverted the intended design (FFT-dominant with minor FcaNet refinement), effectively turning the architecture into an FcaNet-dominant model. Furthermore, under FP16 autocast, learnable fusion weights failed to adapt, remaining near their initialization after 100 epochs, indicating that optimization alone could not overcome the mismatch.

Methodology: FreqFuseNet
To resolve this, the authors propose FreqFuseNet, which introduces scale-normalized residual fusion. The core mechanism involves normalizing the FcaNet branch to match the FFT activation statistics before residual injection.

Scale Normalization: The FcaNet features are rescaled using the ratio of the standard deviations of the two branches:
$\hat{F}_{FcaNet} = F_{FcaNet} \times \frac{\sigma_{FFT}}{\sigma_{FcaNet} + \epsilon}$
This ensures the residual branch operates on the same amplitude scale as the primary FFT branch.
Residual Injection: The normalized features are fused with a fixed, low-amplitude coefficient:
$F_{fused} = F_{FFT} + \beta \hat{F}_{FcaNet}, \quad \text{where } \beta = 0.05$
Post-normalization, the residual magnitude ratio becomes $\rho_{norm} \approx \beta = 0.05$ , restoring the intended design where FcaNet acts as a calibrated 5% refinement rather than a dominant term.
Architecture: The model utilizes a 3D U-Net spatial encoder feeding a bottleneck. The bottleneck is processed in parallel by a 3D real-valued FFT branch and a DCT-based FcaNet branch. The scale-normalized fusion is followed by cross-domain attention and an adaptive router (inherited from the authors' prior AFS-DSN work) to produce the final segmentation mask.
Training Protocol: The method was evaluated on the SegRap2023 benchmark using a controlled binary per-OAR crop protocol (128×128×128 voxels) across 10 thin-wall OARs. Training used FP16 mixed-precision, AdamW optimizer, and a combined Dice + Cross-Entropy loss.

Key Contributions

Identification of Feature-Scale Mismatch: The authors quantify an ~863× activation-scale disparity in FP16 FFT–FcaNet fusion, demonstrating how a nominal 5% residual coefficient effectively becomes a ~43× dominant term, inverting the architectural intent.
Analysis of Naive Fusion Failure: They formalize the residual magnitude ratio ( $\rho_{naive} \approx 43$ ) and provide empirical evidence that learnable fusion weights fail to converge to meaningful values under this mismatch in FP16 settings.
Scale-Normalized Residual Fusion: They propose FreqFuseNet, a single-operation correction that normalizes the residual branch to the primary branch's scale, allowing a fixed $\beta=0.05$ to function as an interpretable low-amplitude coefficient.
Performance Efficiency: The proposed method achieves competitive performance with significantly fewer parameters (29.7 M) compared to a full wavelet baseline (414.6 M).

Results
Evaluated on 180 binary per-OAR test samples (18 cases × 10 OARs) from the SegRap2023 dataset:

Primary Metrics (Seed 2): FreqFuseNet achieved a Dice score of 0.849, HD95 of 0.824 mm, and SDice@1mm of 0.959.
Reproducibility: An independent second seed yielded comparable results (Dice 0.843, HD95 0.823 mm).
Comparative Performance:
- FreqFuseNet showed statistically significant case-level aggregate improvements over 3D U-Net (Wilcoxon $p < 0.01$ ) and MedNeXt-S (Wilcoxon $p < 0.05$ ) across Dice, HD95, and SDice@1mm.
- It outperformed the DWT-Full baseline (414.6 M parameters) while using 92.8% fewer parameters.
- Per-OAR analysis showed consistent Dice improvements across all 10 structures, with the largest gains on the most geometrically complex structures (Tympanic Cavity and IAC).
Ablation: The "FixedFusion" (naive fusion without normalization) performed worse than the FcaNet-only branch on HD95, confirming the failure of naive fusion under scale mismatch.

Significance and Claims
The paper claims that the primary barrier to effective dual-frequency fusion in this context is not the lack of informative frequency features, but the incompatibility of activation scales between branches, particularly under mixed-precision training.

The significance of FreqFuseNet lies in:

Restoring Design Intent: By normalizing scales, the method ensures the residual branch functions as a calibrated refinement, preventing it from overwhelming the primary pathway.
Boundary Sensitivity: The improvements in HD95 and surface Dice (SDice) suggest that the FFT-dominant pathway, when correctly fused with FcaNet, captures fine-grained boundary transitions that spatial convolutions miss, which is crucial for thin-wall OARs.
Parameter Efficiency: The method demonstrates that high-performance boundary-sensitive segmentation can be achieved with a lightweight architecture (29.7 M parameters) compared to heavy wavelet-based baselines.

The authors remain modest regarding clinical deployment, noting that the study is limited to non-contrast CT, a specific binary crop protocol, and a small test set (18 cases). They emphasize that while the method improves geometric metrics relevant to dosimetry, direct clinical dosimetric impact (dose-volume histogram analysis) remains to be validated in future work.