A Compact Hybrid Convolution--Frequency State Space Network for Learned Image Compression

This paper proposes HCFSSNet, a compact hybrid network for learned image compression that combines convolutional layers with a novel Vision Frequency State Space block to effectively model both local details and long-range dependencies while preserving 2D neighborhood continuity and enhancing frequency awareness.

The Gist

Imagine you have a massive, high-resolution photo album that you want to send to a friend over a slow internet connection. You need to shrink the files (compress them) so they send quickly, but you don't want your friend to receive blurry, pixelated garbage. This is the challenge of Learned Image Compression (LIC).

For a long time, computers used "hand-crafted" rules (like JPEG) to do this. But recently, scientists started using AI to learn the best way to compress images. The problem? The AI models getting the best results are either:

1. Too heavy: Like a giant truck that takes forever to drive (too much computing power).
2. Too clumsy: They flatten the 2D photo into a long 1D line to process it, which breaks the natural "neighborhood" of pixels (like cutting a map into a single strip of paper and trying to find your way).

Enter HCFSSNet, the new model proposed in this paper. Think of it as a smart, compact delivery service that uses a hybrid strategy to pack your photos perfectly.

Here is how it works, broken down into simple analogies:

1. The Two-Track Team (The Hybrid Design)

Most AI models try to do everything with one big brain. HCFSSNet splits the work into two specialized teams that work together:

• The Local Detail Team (CNN): Imagine a microscope. This team zooms in on tiny, specific details like the texture of a cat's fur or the edge of a brick wall. It's great at seeing what's right in front of it but doesn't care about the whole room.
• The Big Picture Team (State Space Model): Imagine a drone flying overhead. This team looks at the whole scene to understand the context (e.g., "this is a street, so there should be cars and sidewalks"). It connects distant parts of the image efficiently without getting bogged down in math.

The Magic: HCFSSNet combines these two. It uses the microscope for the fine details and the drone for the big picture, ensuring nothing is lost.

2. The "All-Direction" Scanner (VONSS)

Here is the tricky part: How do you scan a 2D photo (a square grid) with a 1D scanner (a line)?

• Old Way: Imagine reading a book line-by-line (left to right, top to bottom). If you are at the top-left corner and want to talk to the pixel just below you, you have to wait until the scanner loops around. The "distance" in the scanner is huge, even though the pixels are neighbors. This confuses the AI.
• The HCFSS Way (VONSS): Imagine a security guard checking a room. Instead of just walking in a straight line, the guard checks horizontally, vertically, diagonally, and even backwards.
  • By scanning in all directions (like an "Omni-directional" scan), the AI ensures that pixels that are neighbors in the photo stay neighbors in the data stream. It preserves the natural shape of the image much better.

3. The "Frequency Tuner" (AFMM)

Think of an image like a symphony.

• Low frequencies are the deep drums and bass (the big shapes, the sky, the background).
• High frequencies are the cymbals and violins (the sharp edges, the fine textures, the noise).

Traditional AI often treats all notes the same. HCFSSNet has a Frequency Tuner (Adaptive Frequency Modulation).

• It looks at the image, breaks it down into its musical notes (using a math tool called DCT), and then turns up the volume on the important notes and turns down the volume on the ones that don't matter as much.
• This allows the AI to be very smart about what data to keep and what to throw away, saving space without losing quality.

4. The "Side Note" Manager (FSTAM)

When you compress an image, you also need to send a "cheat sheet" (called a hyperprior) to help the decoder understand how to unpack the image.

• Usually, this cheat sheet is just a rough summary.
• HCFSSNet adds a Frequency Tuner to this cheat sheet too. It makes sure the instructions sent to the decoder are also "frequency-aware," ensuring the decoder knows exactly how to reconstruct the high-frequency details (like sharp edges) when it receives the file.

The Result: A Compact, Efficient Solution

The authors didn't try to build the biggest, most powerful AI possible (which would be slow and expensive). Instead, they built a compact, efficient hybrid.

• The Analogy: If other top models are like a heavy-duty semi-truck (great capacity, slow, expensive), HCFSSNet is a high-performance sports sedan. It's smaller, uses less fuel (fewer parameters), but still gets you to the destination (high-quality image) just as fast and efficiently for most trips.

In summary:
HCFSSNet is a new way to shrink photos for the internet. It uses a two-team approach (local details + big picture), scans images in all directions to keep neighbors together, and tunes the frequencies of the image to keep only the most important data. It achieves excellent results without needing a supercomputer to run it.

Technical Summary

1. Problem Statement

Learned Image Compression (LIC) aims to optimize image compression through end-to-end deep learning. While recent advancements have utilized Transformers (for long-range dependencies) and State Space Models (SSMs) (for linear complexity), significant challenges remain:

• Transformer Limitations: They incur quadratic computational complexity relative to sequence length, making them inefficient for high-resolution images.
• SSM Limitations: Existing Vision SSMs (e.g., Vim, VMamba) typically flatten 2D feature maps into 1D sequences using horizontal and vertical scans. This approach disrupts neighborhood continuity, particularly failing to explicitly model diagonal spatial relationships, which are crucial for preserving local image structures.
• Entropy Modeling Gaps: Most entropy models operate primarily in the spatial and channel domains. There is a lack of frequency-aware modeling for side information (hyperpriors), despite the fact that frequency-domain analysis is fundamental to traditional compression (e.g., DCT in JPEG).
• Design Trade-offs: Many high-performance models are becoming increasingly complex and heavy, lacking a compact, unified design that balances local detail, long-range context, and frequency sensitivity.

2. Methodology

The authors propose HCFSSNet, a compact hybrid architecture that integrates convolutional local modeling, state-space global modeling, and frequency-aware modulation.

A. Overall Architecture

HCFSSNet follows a standard hyperprior-based LIC framework consisting of an analysis transform, synthesis transform, hyperprior pathway, and a channel-wise entropy model. The core innovation lies in the Hybrid Convolution–Frequency State Space (HCFSS) block used in the main and hyperprior transforms.

B. Key Components

1. Hybrid Convolution–Frequency State Space (HCFSS) Block
This block splits feature channels into two parallel branches to combine local and global modeling:

• CNN Branch: Uses standard convolutional layers to refine local spatial details.
• Vision Frequency State Space (VFSS) Branch: Handles complementary long-range contextual aggregation.

2. Vision Omni-directional Neighborhood State Space (VONSS)
To address the mismatch between sequential scanning and 2D spatial proximity in SSMs:

• Mechanism: Instead of relying solely on horizontal/vertical scans, VONSS scans features along eight directions: horizontal, vertical, diagonal, and anti-diagonal (including reverse orders).
• Benefit: This ensures that spatially adjacent pixels (including diagonal neighbors) remain close in the sequential domain, preserving 2D neighborhood relations and improving the modeling of local structures like edges and textures.

3. Adaptive Frequency Modulation Module (AFMM)
Integrated into the VFSS branch to introduce frequency awareness:

• Mechanism: It applies a fixed Discrete Cosine Transform (DCT) to local spatial windows. Instead of learning a new basis, AFMM learns adaptive weights to reweight the DCT frequency coefficients.
• Benefit: This allows the network to perform compression-oriented frequency selection (e.g., suppressing high-frequency noise or enhancing specific textures) without the overhead of learning a complex transform basis. The weighted coefficients are then transformed back to the spatial domain via Inverse DCT (IDCT).

4. Frequency Swin Transformer Attention Module (FSTAM)
Designed for the hyperprior pathway (side information modeling):

• Mechanism: Combines a Swin Transformer block with the AFMM.
• Benefit: Enhances the modeling of side information by incorporating frequency-aware refinement into the entropy estimation process, allowing for more accurate bitrate prediction.

3. Key Contributions

1. HCFSSNet Architecture: A compact hybrid design that unifies CNN-based local refinement, SSM-based long-range modeling, and frequency-aware modulation within a single framework.
2. VONSS Module: A novel scanning strategy that extends SSMs to 2D features via omni-directional (8-way) scanning, explicitly preserving diagonal neighborhood relations often lost in standard SSMs.
3. Frequency-Aware Refinement: The introduction of AFMM in the main transform and FSTAM in the hyperprior path, demonstrating that frequency-domain modulation improves both feature representation and entropy modeling.
4. Efficiency vs. Performance: Achieves competitive rate-distortion performance with significantly fewer parameters than recent high-capacity SSM and Transformer-based codecs.

4. Experimental Results

The model was evaluated on Kodak, Tecnick, and CLIC Professional Validation datasets.

• Rate-Distortion Performance:
  • HCFSSNet achieves BD-rate savings of 18.06% (Kodak), 24.56% (Tecnick), and 22.44% (CLIC) compared to the VTM anchor.
  • It performs competitively against heavy baselines like MambaIC and MLIC++, despite having fewer parameters.
• Model Complexity:
  • Parameters: 80.97M (compared to 116.72M for MLIC++ and 123.81M for MambaIC).
  • Trade-off: While it does not achieve the absolute lowest BD-rate or the fastest decoding time (due to the overhead of omni-directional scanning and DCT), it occupies a "compact-but-competitive" region in the design space.
• Ablation Studies:
  • VONSS vs. CSM: Replacing the standard Cross-Scan Module (CSM) with VONSS improved PSNR by up to 0.15 dB.
  • AFMM: Adding frequency modulation improved PSNR by up to 0.22 dB and reduced bitrate.
  • FSTAM: Frequency-aware refinement in the hyperprior provided consistent, albeit small, gains over standard Swin attention.

5. Significance and Conclusion

This paper addresses the critical gap in current LIC architectures regarding the explicit modeling of 2D neighborhood continuity and frequency-domain characteristics.

• Scientific Impact: It demonstrates that extending SSMs to include omni-directional scanning and integrating frequency modulation into both the main transform and entropy model yields significant performance gains.
• Practical Value: HCFSSNet offers a balanced solution for scenarios where model size and parameter efficiency are as important as compression performance. It proves that a "compact" hybrid design can rival much larger, more complex models.
• Future Work: The authors plan to explore lighter variants for lower-latency applications and extend the design to video compression tasks.

In summary, HCFSSNet represents a strategic shift from simply scaling up model size to architectural refinement, leveraging the strengths of convolution, state-space models, and frequency analysis to achieve efficient and high-quality image compression.