💻 computer science

CHASM: Cross-frequency Harmonized Axis-Separable Mixing for Spectral Token Operators

The paper introduces CHASM, a spectral token mixer that improves global interaction modeling in visual feature maps by learning a shared channel eigenbasis across all frequencies while maintaining frequency-specific spectral gains, thereby achieving consistent performance gains over existing baselines in tasks like MRI reconstruction and natural-image processing.

Original authors: Pengcheng Fang, Hongli Chen, Yuxia Chen, Tengjiao Sun, Jiaxin Liu, Xiaohao Cai

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pengcheng Fang, Hongli Chen, Yuxia Chen, Tengjiao Sun, Jiaxin Liu, Xiaohao Cai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to fix a blurry, scrambled photograph or a medical scan (like an MRI) that is missing pieces of information. To do this, computers use a special tool called a Spectral Token Mixer.

Think of this tool as a master chef trying to understand a complex soup. Instead of tasting the soup spoon-by-spoon (looking at one pixel at a time), the chef uses a special "frequency filter" to taste the whole pot at once. This helps the computer understand how different parts of the image relate to each other globally.

However, existing methods have a problem. They are like chefs who either:

Stick to a rigid recipe: They taste the soup the same way for every ingredient, regardless of the flavor intensity.
Go rogue: They invent a completely new, unique way to taste every single flavor note, but they never agree on what the "flavor scale" even means. This makes it hard to compare one part of the soup to another.

Enter CHASM.

The paper proposes a new method called CHASM (Cross-frequency Harmonized Axis-Separable Mixing). Here is how it works, using simple analogies:

1. The Shared "Compass" (The Shared Basis)

Imagine you are exploring a city with many different neighborhoods (these are the different "frequencies" or patterns in the image).

Old methods: Every neighborhood had its own map, drawn by a different cartographer. One map might call "North" "Up," while another calls it "Right." It was chaotic and hard to navigate between neighborhoods.
CHASM's approach: CHASM gives every neighborhood the exact same compass. It learns one "master map" (a shared channel eigenbasis) that everyone agrees on. Now, when the computer looks at a pattern in one part of the image, it knows exactly how to interpret that pattern in another part because they are speaking the same "directional language."

2. The "Volume Knobs" (Frequency-Specific Gains)

Just because everyone uses the same compass doesn't mean every neighborhood is the same. Some areas are loud and bright; others are quiet and dark.

CHASM's approach: While the directions (the compass) are shared, CHASM gives each frequency its own set of volume knobs (positive spectral gains). It can turn the volume up for a specific pattern in one area and down in another. This keeps the system flexible and adaptable to local details.

3. The "Two-Step Dance" (Axis-Separable Mixing)

CHASM doesn't try to fix the whole image in one giant, messy step. Instead, it dances in two simple steps:

First, it looks at the image row-by-row (height).
Then, it looks at the image column-by-column (width).
By breaking the problem down this way, it keeps the math efficient and fast, acting like a "drop-in" replacement that can swap into existing computer vision systems without needing to rebuild the whole machine.

What Did They Prove?

The authors tested CHASM in three main scenarios, acting like a stress test for their new "compass":

Fixing Scrambled MRIs: They tried to reconstruct fast, blurry MRI scans of brains and knees. CHASM did a better job than the old methods, creating clearer images with fewer artifacts (ghosting or blurring).
Segmenting Tumors: They used it to help computers identify tumor boundaries in undersampled brain scans. CHASM was more accurate.
Fixing Regular Photos: They even tested it on natural images (like landscapes), and it still worked better than the competition.

The "Why" Behind the Success

The paper ran some "what-if" experiments (ablations) to prove their theory:

If you remove the shared compass: The system gets confused and performance drops. This proves that having a common language between different frequencies is crucial.
If you scramble the sampling pattern: When they used random, chaotic sampling instead of the structured patterns found in real MRI machines, the advantage of CHASM disappeared. This suggests that CHASM is specifically designed to work well with the structured, coherent way medical data is actually collected.

In Summary

CHASM is a smarter way for computers to mix image data. It strikes a perfect balance: it forces different parts of the image to agree on a common "direction" (the shared basis) so they can talk to each other, while still allowing each part to have its own unique "volume" (the gains) to handle specific details. This makes it a powerful tool for cleaning up medical scans and reconstructing images from incomplete data.

Technical Summary: CHASM

Problem Statement
Spectral token mixers, which utilize Fourier transforms to model global interactions in visual feature maps, have become a standard mechanism for tasks involving undersampling and aliasing, such as accelerated MRI reconstruction. However, existing designs face a structural dilemma in how they handle channel mixing across different frequencies:

Filter-wise methods apply different spectral responses to different frequencies but operate along fixed channel axes, limiting adaptivity.
Fully adaptive methods allow frequency-indexed channel transformations but learn independent channel directions for each frequency. This lacks an explicit mechanism to align these directions across the spectrum, potentially leading to incoherent representations.

The paper identifies a gap for a "structured middle ground" that learns useful channel directions, makes them comparable across frequencies, and still allows each frequency to respond differently.

Methodology: CHASM
The authors propose CHASM (Cross-frequency Harmonized Axis-Separable Mixer), a structured operator designed as a drop-in replacement for existing spectral mixers. The core innovation is the separation of shared and frequency-specific components:

Shared Eigenbasis: CHASM enforces that all frequencies share a single learned orthogonal channel eigenbasis ( $U$ ). This defines a common channel coordinate system across the entire spectrum, ensuring that channel directions are comparable.
Frequency-Specific Gains: While the directions are shared, each frequency retains its own positive spectral gain vector ( $\lambda_k$ ). This preserves local spectral adaptivity, allowing the model to weight the shared channels differently at different frequencies.
Axis-Separable Application: The operator is applied separably along the height-channel (CH) and width-channel (CW) planes. For each axis, the process involves:
1. Real-valued FFT along the spatial axis.
2. Application of the harmonized operator $M(k) = U \text{diag}(\lambda_k) U^\top$ to the Fourier coefficients.
3. Inverse FFT back to the spatial domain.
4. Composition of the two passes (e.g., $\Phi_{CW} \circ \Phi_{CH}$ ) followed by lightweight local refinement and $1\times1$ fusion.
Parameterization: The shared basis $U$ is parameterized via the exponential map of a skew-symmetric matrix (mapping to $SO(C)$), initialized as the identity. The gains are learned via a table interpolated to the input resolution and passed through a softplus function to ensure positivity.
Theoretical Justification: The paper provides a structural characterization of the shared-basis operator family, proving that the parameterization matches the intrinsic local degrees of freedom. It also notes that the positive shared-basis form is compatible with the Hermitian positive-semidefinite structure of normal operators ( $A^*A$ ) found in inverse problems like MRI.

Key Contributions

Design Principle: The identification of cross-frequency harmonization as a critical design principle for spectral token mixers, addressing the lack of explicit structure tying frequency-indexed channel operators together.
Architecture: The proposal of CHASM, an axis-separable mixer that shares a learned channel eigenbasis across frequencies while retaining frequency-specific positive gains.
Validation: A structural characterization of the shared-basis operator family and extensive evaluation through controlled same-backbone comparisons. The study isolates the effects of the shared-basis constraint and coherent sampling geometry through ablation studies.

Experimental Results
CHASM was evaluated as a drop-in replacement in HiFi-Mamba-family, U-Net, and ViT backbones across three domains:

Accelerated MRI Reconstruction: On CC359 and fastMRI datasets (4x and 8x undersampling), CHASM consistently outperformed same-backbone spectral-mixer baselines (including GFNet, AFNO, FFC, and FNO-D) in both PSNR and SSIM.
Cross-Domain Transfer: CHASM improved performance on natural-image reconstruction (ADE20K, ImageNet) and undersampled brain tumor segmentation (BraTS), demonstrating that the benefits extend beyond a single dataset or task.
Ablation Studies:
- Shared-Basis Constraint: Removing the shared-basis constraint (allowing independent bases per frequency) significantly weakened performance, confirming that the shared coordinate system is a primary driver of the gain.
- Sampling Geometry: Randomizing the sampling mask (disrupting coherent spectral relationships) substantially reduced the performance gain of CHASM compared to structured masks. This supports the claim that cross-frequency harmonization is most effective when the measurement process contains coherent spectral structure.

Significance and Claims
The paper claims that CHASM provides a simple yet effective way to coordinate spectral channel mixing without sacrificing local frequency adaptivity. By enforcing a shared eigenbasis, the model learns a coherent channel representation that generalizes well across different frequencies and architectures. The authors position CHASM not as a new end-to-end architecture, but as a modular, drop-in spectral mixer that improves upon existing backbones. The work suggests that cross-frequency harmonization is a useful inductive bias for spectral token operators, particularly in inverse imaging problems where spectral structure is explicit. The authors note limitations to image-domain reconstruction and segmentation, suggesting future work could explore broader dense prediction and real sensor-noise restoration.